[Python] Wide and Narrow transformations in Spark

I have a question : Wide and Narrow transformations python Spark, are found both in RDDs and Structured APIs, right ?

I mean, I guess I got the difference between wide and narrow transformations. My point is, are they exclusive for one or the other python Spark API (whether it's RDD or not)?

Wide Transformations:

• 
  These are the operations that require shuffling data across partitions.
  
  
• 
  This means that the data needs to be moved between executor or worker nodes.
  
  
• 
  Some examples of wide transformations in Spark include :
  
  • groupBy
  • groupByKey()
  • reduceByKey()
  • aggregate()
  • aggregateByKey()
  • distinct()
  • join()
  • repartition()

Narrow transformations :

• 
  compute data that live on a single partition meaning there will not be any data movement between partitions to execute narrow transformations
  
  
• 
  map()
  
  
• 
  filter()
  

Thank you a lot!

Continue reading...