Hi,
I have to pull some data from hive table based on some filtering criteria in spark scala code.
Wanted to use the dropduplicates functions to remove duplicate data from aet of columns.
Please anyone let me know how dropDuplicates really works,asit deletes duplicate record randomly or based on any criteria…
Thanks,
KA