Execute (real) shell commands from Groovy.
This post is about running shell commands from within Groovy, specifically bash but it is easy to adapt to other shells. You can already run commands with syntax like:
"ls -l".execute()
That is about as simple as it gets and works great for many situations. However, execute() runs the given command passing it the list of options, the options are NOT passed through the shell (e.g. bash) for expansion and so on. As a result, you can NOT do something like:
"ls *.groovy".execute()
In this case, no shell sees the * to expand it, and so it just gets passed to ls exactly as it is. To address this, we can create a shell process with ProcessBuilder and pass the command to the shell for execution. A common use case for me is to want to just pipe the shell command's output to stdout. With some Groovy meta-object programming we can make this a method of GString and String so that you can execute any kind of string simply by calling, for example, a .bash() method on the string. Below is a class that does that. This class (including improvements) is included in durbinlib.jar. With this class, one can not only properly execute the ls *.groovy example above, but can even execute shell scripts like:
"""
for file in \$(ls);
do
echo \$file
done
""".bash()
To turn on this functionality it is necessary to call RunBash.enable() first. So a full example using the durbinlib implementation is:
#!/usr/bin/env groovy
RunBash.enable()
"""
for file in \$(ls);
do
echo \$file
done
""".bash()
A skeleton of the class itself follows:
static boolean bEchoCommand = false;
// Add a bash() method to GString and String
static def enable(){
GString.metaClass.bash = {->
RunBash.bash(delegate)
}
String.metaClass.bash = {->
RunBash.bash(delegate)
}
}
static def bash(cmd){
cmd = cmd as String
// create a process for the shell
ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd);
pb.redirectErrorStream(true); // use this to capture messages sent to stderr
Process shell = pb.start();
shell.getOutputStream().close();
InputStream shellIn = shell.getInputStream(); // this captures the output from the command
// at this point you can process the output issued by the command
// for instance, this reads the output and writes it to System.out:
int c;
while ((c = shellIn.read()) != -1){
System.out.write(c);
}
// wait for the shell to finish and get the return code
int shellExitStatus = shell.waitFor();
// close the stream
try {
shellIn.close();
pb = null;
shell = null;
} catch (IOException ignoreMe) {}
}
}
{
Labels: bash, groovy, ProcessBuilder
If the bash process created within RunBash by pb.start produces a sufficiently large amount of output, the buffer for the stream will fill up, and the bash process will pause, waiting for your program to read from shellIn.
If this happens, shell.waitFor() will wait forever, since the bash process won't die, since it is waiting for your program to finish reading it's output before it can run to completion and exit.
To avoid the risk of this kind of deadlock happening, you should call shell.waitFor() after reading all of shellIn.
Also, you might want to add "shell.getOutputStream().close();" before reading from shellIn, to avoid the risk that the bash process will attempt to read from it's standard input stream, and hang forever, waiting for you to write something to it.
Of course, these are not really Groovy issues, or Java ones, but are simply a matter of understanding how processes and pipes work. (October 12, 2012 at 7:17 PM) top
Thanks! I have made those changes above. I had apparently discovered and fixed this error in durbinlib but I forgot about fixing it in the post. (October 12, 2012 at 8:22 PM) top