Export DF data to Text file without Folder and part files


#1

Hi ,

I am trying to write a single text file but not able to do.It is generating as a folder and folder contains part files.
This is expected behaviour. Is there any way to do with Scala to export DF (Or) RDD data to text file?
I am new to Scala coding.

Kindly let me know how to do?

Many thanks for your help.


#2

A bad idea is to collect or use single reducer (partition) while saving the file. Alternative way is to have part files and do a hdfs dfs -cat * > output.txt later.