[Python] Spark write parquet not writing any files, only _SUCCESS

The application includes

val stats = sqlContext.sql("select id, n from myTable")

stats.write.parquet("myTable.parquet")


This creates dir myTable.parquet with no contents other than an empty _SUCCESS file, even that

stats.show // illustration only here, original size motivates parquet use

+-----+----+
| id | n |
+-----+----+
| a | 1 |
| b | 2 |
+-----+----+

stats.printSchema

root
|-- id: string (nullable = true)
|-- n: long (nullable = true)


How to make write.parquet to write the actual contents of the dataframe ? What is missing ?

Note This occurs also with saveAsTextFile.

Continue reading...