Running external processes in Scala

Introduction

In one of my home projects, I needed to write a small external process manager. The application should be able to start an external daemon, periodically monitor its state, when it is necessary to turn off, turn on, change settings, etc. The existing functionality in Java for such tasks is very scarce, and since I was simultaneously dealing with Scala, I decided to see how she was doing with it. And I was pleasantly surprised: Scala offers in my opinion a good API for working with external processes.
In this article, I would like to talk more about this.

Process start

Two traits are the cornerstone of work with processes: these are scala.sys.process.Process and scala.sys.process.ProcessBuilder .
Process allows you to work with an already running process, and ProcessBuilder allows you to configure startup parameters.
These entities are in the package scala.sys.process . To run a simple example, execute the code:

scala> import scala.sys.process._
scala> val process: Process = Process("echo Hello World").run()
scala> println(process.exitValue())

The run method is the main method for starting a process whose declaration is located in the ProcessBuilder tray . Returns a reference to an object of type Process . The running process runs in the background, data is output in the console. There are two methods declared in the Process tray :

• exitValue() - Waits for the completion of the process and returns a completion code;
• destroy() - destroys the running process.

This trait is very similar to the standard Java class java.lang.Process.
There are more specialized methods for starting processes in the ProcessBuilder tray . I will give a brief description of the main ones:

• ! - starts the process, waits for completion, executes the data to the console, and returns the process completion code as a result;
• !! - starts the process, waits for completion, displays the data in the console, if the completion code is other than zero - throws an exception, as a result it returns the output of the process as a string;
• lines- starts the process, returns Stream [String]. This thread allows you to read process data in parallel with the process. In case the information is not available, Stream is blocked and will wait until the information reappears, or the process completes execution. If the process termination code is non-zero, the method will throw an exception. To avoid an exception, you should call lines_ !;
• run- starts the process and returns a link to the Process .

In my project, I did not need to store links to external processes, so I almost did not use the run method. But the method was !just right for me.
The previous example can be rewritten as follows:

scala> Process("echo Hello World!").!
Hello World!
res1: Int = 0
scala> Process("echo Hello World!").!!
res2: String = "Hello World!"
scala> Process("echo Hello World!").lines
res3: Stream[String] = Stream(Hello World!, ?)

Implicit casting

There are methods for implicitly casting strings ( java.lang.String ) and sequences ( scala.collection.Seq ) to the ProcessBuilder trait .
We can write our code like this:

scala> "echo Hello World!".!
Hello World!
res2: Int = 0

or so:

scala> Seq("echo", "Hello", "World!").!
Hello World!
res3: Int = 0

To a large extent, this reduces and simplifies writing, and the code becomes more understandable.
And this in turn reduces the number of errors in the future.

Process Combination (Pipe)

Process calls can be combined into chains similar to linux command chains.

scala> "ls".!
11.txt
1.txt
2.txt
3.txt
res2: Int = 0
scala> ("ls" #| "grep 1").!
11.txt
1.txt
res6: Int = 0

The output from the ls command was directed to grep input. Grep filtered out the received information on occurrence 1.
You can perform conditional operations, for example:

scala> ("find . -name *.txt -exec grep 0 {} ;"  #|  "xargs test -z"  #&&  "echo 0-free"  #||  "echo 0-exists").!
0-exists
res23: Int = 0

Here, if there are files in the directory with the * .txt extension and in any of them, there is 0 in the text - it will output 0-exists to the console, otherwise 0-exists.
#&&- executes the next command if the previous one is executed correctly;
#||- executes the following command if the previous command is executed with errors.
I like this feature most of all, it allows you to use a linux-like pipe inside Scala and write small sh scripts right inside your code.

Overriding I / O Streams

All our code is inconvenient and useless without the functionality of redefining the input / output of external processes.
Often, it is required to monitor the information displayed in order, for example, to decrypt the error that has occurred, or to make sure that everything works correctly.
In the ProcessBuilder tray, in each of the run,!, !!, lines methods, you can transfer the instance of the ProcessLogger trait , which allows you to redirect the program's output streams to a file or line.
Here's how you can use ProcessLogger to count the number of lines printed by a process:

scala> var normalLines = 0
normalLines: Int = 0
scala> var errorLines = 0
errorLines: Int = 0
scala> val countLogger = ProcessLogger(line => normalLines += 1,
 	| line => errorLines +=1)
countLogger: scala.sys.process.ProcessLogger = scala.sys.process.ProcessLogger$$anon$1@459c8859
scala> "ls" ! countLogger
res0: Int = 0
scala> println("normalLines: " + normalLines + ", errorLines: " + errorLines)
normalLines: 4, errorLines: 0

ProcessLogger allows you to override output streams. The scala.sys.process.ProcessIO class is also used to override both input and output .
A small example:

Seq("grep", "1") run new ProcessIO((output: java.io.OutputStream) => {
	output.write("1.txt\n2.txt\n3.txt\n11.txt".getBytes)
	output.close()
  }, (input: java.io.InputStream) => {
  	println(Source.fromInputStream(input).mkString)
 	input.close()
  }, _.close())

The first parameter is the input stream to the process: here we write the initial data.
The second parameter is the standard output, and the last is the error output.
Parameters are functions that handle the necessary threads.
Earlier, I said that the execution of external commands can be combined, in addition, using the same form of recording, you can transfer data to the process, or read them from there.
You can transfer data from a file to a process using the method #<, and write using the method #>:

scala> ("echo -e 1.txt\\n2.txt\\n3.txt" #> new java.io.File("1.txt")).!
res21: Int = 0
scala> ("grep 1" #< new java.io.File("1.txt")).!!
res22: String =
"1.txt"

In the same way, for example, you can copy information from one file to another:

scala> (new java.io.File("1.txt") #> new java.io.File("2.txt")).!
res23: Int = 0
scala> "cat 2.txt".!
1.txt
2.txt
3.txt
res24: Int = 0

Conclusion

In the article, I talked about the basics of working with external processes in Scala. In Java, to implement this, I would have to write a bunch of wrappers, and in the end, I still could not get closer to such simplicity. You can read more about the API at http://www.scala-lang.org or dig into the source (which I did, for example, took some examples from there). In jdk1.7, we expanded the class java.lang.ProcessBuilder a bit , and in Java has become more convenient to run and execute external commands. But to the simplicity of Scala, jdk is far away.