Hi,
I have read text file convert to dataframe tables using scala spark.
The table looks like,
±-----------±---------±—±---+
|value |col1 |col2|col3|
±-----------±---------±—±---+
|FIRST: |FIRST: |null|null|
|erewwetrt=1 |erewwetrt |1 |null|
|wrtertret=2 |wrtertret |2 |null|
|ertertert=3 |ertertert |3 |null|
|; |; |null|null|
|FIRST: |FIRST: |null|null|
|asdafdfd=1 |asdafdfd |1 |null|
|adadfadf=2 |adadfadf |2 |null|
|adfdafdf=3 |adfdafdf |3 |null|
|; |; |null|null|
|SECOND: |SECOND: |null|null|
|adfsfsdfgg=1|adfsfsdfgg|1 |null|
|sdfsdfdfg=2 |sdfsdfdfg |2 |null|
|sdfsdgsdg=3 |sdfsdgsdg |3 |null|
|; |; |null|null|
|FIRST: |FIRST: |null|null|
|asdasdas=1 |asdasdas |1 |null|
|adasfasdf=2 |adasfasdf |2 |null|
|dfsdfsdf=3 |dfsdfsdf |3 |null|
|; |; |null|null|
±-----------±---------±—±---+
My problem is, how can I remove the rows SECOND: to ;.
So, final dataframe table looks as,(needs to contains only FIRST portion…)
±-----------±---------±—±---+
|value |col1 |col2|col3|
±-----------±---------±—±---+
|FIRST: |FIRST: |null|null|
|erewwetrt=1 |erewwetrt |1 |null|
|wrtertret=2 |wrtertret |2 |null|
|ertertert=3 |ertertert |3 |null|
|; |; |null|null|
|FIRST: |FIRST: |null|null|
|asdafdfd=1 |asdafdfd |1 |null|
|adadfadf=2 |adadfadf |2 |null|
|adfdafdf=3 |adfdafdf |3 |null|
|; |; |null|null|
|
|FIRST: |FIRST: |null|null|
|asdasdas=1 |asdasdas |1 |null|
|adasfasdf=2 |adasfasdf |2 |null|
|dfsdfsdf=3 |dfsdfsdf |3 |null|
|; |; |null|null|
±-----------±---------±—±---+
How can achieve it in scala spark?.
thank you