How powerful laptop do I need to run spark

Question
1

I’m considering replacing my 6 year old macbook air laptop for a new one. The top 13 inch MacBook pro has 16 Gbyte memory, and a quad-core intel I7 processor. Is that big enough to do interesting things with Spark, or am I throwing my money down the toilet by even attempting it? The problem (as everyone knowns with Apple) is that you can’t upgrade the processor or the memory later, so you have to make the right decision up front.

From what I’ve read, yes, it is possible to run Spark on the MacBook, but only toy projects. So, I’m wondering if it is better just to save 600 euros and get the 8GByte machine with the intel i5 processor?

2

Are you planning to run Spark in local mode? Or on a cluster with your laptop as master node?

Most people would run Spark on their laptop only for development and testing, and on some separate cluster for production.

1 Like
3

Alternatively, save all your euros and just fire up an AWS EMR instance with Spark installed for $0.1 - $0.3 per hour. That will also save you the Spark setup/configure learning curve.

1 Like