Process - using Pipe

Question
#1

When I pipe a String to another process, I get an ‘Unmatched quote’ exception - but only if the number of quotes (either ’ or ") within the String is odd:

import scala.language.postfixOps
import scala.sys.process.stringToProcess

val odd = "odd number of 'qu'otes' - this does not work"
s"""echo $odd""" #| "cat -" !
scala.sys.process.Parser$ParseException: Unmatched quote
  at scala.sys.process.Parser$.tokenize(Parser.scala:111)
  at scala.sys.process.ProcessCreation.apply(Process.scala:95)
  at scala.sys.process.ProcessCreation.apply$(Process.scala:94)
  at scala.sys.process.ProcessImplicits.stringToProcess(Process.scala:46)
  at scala.sys.process.ProcessImplicits.stringToProcess$(Process.scala:214)
  at scala.sys.process.package$.stringToProcess(package.scala:207)
  ... 31 elided

When the number of quotes is even, it works:

val even = "even number of 'quotes' - this works"
s"""echo $even""" #| "cat -" !

I would assume that Scala (2.13.8 / 3.1.2) does not parse the String, because ist is wrapped with “”". Is this correct?

1 Like
#2

As shown in the stack trace, process.Parser tokenizes the string and handles quoting.

You can flip quotes:

scala> """echo "ain't it real?"""" #| "cat -" !
ain't it real?
val res7: Int = 0

It respects escaping without stripping it.

scala> """echo 'ain\'t it real?'""" #| "cat -" !
ain\'t it real?
val res8: Int = 0

I’m not sure why it does that, as a quick test shows bash and zsh just dump you into the secondary prompt.

By coincidence, the other day I was doing some spring cleaning and among a stack of papers I tossed was the sheet on which I’d diagrammed sys.process. That was a lot of ink. The details escape me. Oh, pun alert!

Edit: guillotine is still active, and offers an interpolator sh"" that also strips backslash from escaped quotes. Development on sys.process is blocked by the library getting stuck between Scala 2 and 3, as when cars try to beat the drawbridge, or does that only happen in movies. I see the PR for the string DSL ticket did not make it.

1 Like
#3

Flipping quotes does not really help. The problem is the same with double quotes when the count is odd:

scala> """echo "ain"t it real?"""" #| "cat -"
scala.sys.process.Parser$ParseException: Unmatched quote
  at scala.sys.process.Parser$.tokenize(Parser.scala:111)
  at scala.sys.process.ProcessCreation.apply(Process.scala:95)
  at scala.sys.process.ProcessCreation.apply$(Process.scala:94)
  at scala.sys.process.ProcessImplicits.stringToProcess(Process.scala:46)
  at scala.sys.process.ProcessImplicits.stringToProcess$(Process.scala:214)
  at scala.sys.process.package$.stringToProcess(package.scala:207)
  ... 31 elided

The problem is that the string comes from a parsed PDF (PDFBox) or another file, so I do not want to mess with it before piping it to another process.
I think the process.Parser should leave the string alone - at least when it is a variable like in the original example. What is the purpose of parsing it?

#4

The ticket was here, where it mentions you can supply a Seq, as shown in the copious documentation.

Worth noting that a DSL for humans will always be at odds with simple machine processing.

#5

I tried the Seq solution but ran into another problem. The receiving program complained about the string structure - maybe echo did something funky. I ended up writing the string to a temporary file and using cat to pipe it to the next command. This works as expected. Thanks for your help.

#6

Oh that’s good thinking. Command line args are overrated. I was recently reminded that the scala shell script supplies -classpath "" to ensure it doesn’t default to -classpath ., but that actually supplies a path with one entry named "" which is not the empty string. That ticket was about unicode file names.

Currently everything I do is at the command line because it reminds me of my childhood, but I intend to change that this year, and this conversation inspires me to aspire.

#7

You got me curious. What is the problem with the command line, and what is the alternative? I find it to be a convenient way to pass parameters to a program. In fact, I wrote my own little command-line parser years ago that gives me more power than the standard Java version. It provides arguments both by name and by order.

#8

One thing the command line does well is scalac -help. They just added scalac -h --help -? alternatives on the assumption that you don’t know what you’re doing. (Tab completion helps with help.)

They also added scalac @file to Scala 3.

Options are just program configuration and suffer from all the challenges of config files except expressed in one dimension, backwards and in high heels.

That is aside from issues mentioned above about expansion and quoting and special options for JVM such as -D, -J, -toolcp, etc, which are about component configuration.

The scalac conventions are stretched thin: supporting -opt:inline:** to mean “inline from anywhere” was hairy while supporting deprecated -opt:l:method.

I don’t know what the alternative is, but I would like to see my options vertically, in two dimensions, as I’ve gotten used to in sbt files.

#9

I certainly agree that command-line arguments have their limits, but I can’t imagine trying to do without them. If I have a script that processes a file, how would I specify the file other than on the command line? And I often find that a program has certain basic parameters, each with a default value, but I want to be able to conveniently specify alternative values on the command line. Sometimes you want a config file, and sometimes you want the convenience of the command line so you don’t have to edit a file. And sometimes you want both options.