Hello everyone !
I’m new here so i hope i will respect the chart.
Hope i’ll be clear.
I have a question about scala (i’m working on spark scala actually).
I have on data where each row is an array of string initially.
To apply my schema, i need to convert some values at some positions in my array to double (5 to 18), position 1 to Int, and let the other in string.
val churn_app1 = churn_app.map(x=> {for(i<-0 to 19) yield { if (i==1) x(i).toInt else if ((5 to 18).toSet contains i) x(i).toDouble else x(i).toString }}.toArray)
I “succeed” doing that but now, i have an array of any and when i apply my scheme, it doesn’t recognize the types because every elements of my arrays are of type any.
This is the object i want to apply to every array but i always have " not enough arguments" but i’m sure it’s related to the any type.
case class Account(state: String, len: Integer, acode: String,
intlplan: String, vplan: String, numvmail: Double,
tdmins: Double, tdcalls: Double, tdcharge: Double,
temins: Double, tecalls: Double, techarge: Double,
tnmins: Double, tncalls: Double, tncharge: Double,
timins: Double, ticalls: Double, ticharge: Double,
numcs: Double, churn: String)
When i do it manually :
churn_app.map(line=> Account(line(0),line(1).toInt,line(2),line(3),line(4),line(5).toDouble,line(6).toDouble,line(7).toDouble,line(8).toDouble,line(9).toDouble,line(10).toDouble,line(11).toDouble,line(12).toDouble,line(13).toDouble,line(14).toDouble,line(15).toDouble,line(16).toDouble,line(17).toDouble,line(18).toDouble,line(19)))
it works perfectly obv, but what will happen when i’ll have 1000 elements in my array … ?
My ultimate goal is to get a data frame in order to apply some algorithms.
Thank you in advance for your help.
If things aren’t clear enough, i can help.