java shell execute

[brothman01]

I am writing a java program that needs to execute shell commands, so I wrote a function that would take the command to execute as a string (ie: "mkdir ~/Folder1") and execute that command with the shell. Here is the function:

   try
   {
    Runtime run = Runtime.getRuntime();
    Process pr = run.exec(cmd);
    pr.waitFor();
    BufferedReader buf = new BufferedReader(new InputStreamReader(pr.getInputStream()));
    String line = "";

    while ((line = buf.readLine()) != null) {
     System.err.println(line); // show any errors returned by the command executed on the error console

    }

   } catch (Exception ee) {}

for some weird reason this function is not executing any commands. Did I do this wrong? It seems like a simple thing to execute shell commands, but it is not working.

[JamesCherrill]

First thing is to fix that terrible catch clause.
If/when there is an error you just told Java that you didn't want to know anything about it, and please discard the detailed error message that Java just created for you.
ALWAYS put an e.printStackTrace(); in your catch blocks until/unless you have a good reason to do something else.

Second thing: You read the standard output from your Process, but you ignore its error stream, so you won't see any error messages it generates. Best to read/display that as well.

There's a good chance that these two things will give you the info you need.

Last thing: Runtime.exec was replaced over 10 years ago by ProcessBuilder, which is much more powerful and avoids many problems. You should replace this code with the ProcessBuilder equivalent. Google for details and examples.

[brothman01]

oh ok, I will definitely print the stacktrace from now on.

cmd = "mkdir ~/Desktop/yellow/home"

    InputStream is = null;
    ByteArrayOutputStream baos = null;
    ArrayList<String> commands = new ArrayList<String>( Arrays.asList(cmd.split(" ") ) );

    ProcessBuilder pb = new ProcessBuilder(commands);
    try {
        Process prs = pb.start();
        is = prs.getInputStream();
        byte[] b = new byte[1024];
        int size = 0;
        baos = new ByteArrayOutputStream();
        while((size = is.read(b)) != -1){
            baos.write(b, 0, size);
        }
        System.out.println(new String(baos.toByteArray()));
    } 
    catch (IOException e) 
    {
        e.printStackTrace();
    } 
    finally
    {
        try {
            if(is != null) is.close();
            if(baos != null) baos.close();
        } catch (Exception ex){}
    }
}

Output from this is blank.I am just trying to tell my program to execute the command in 'cmd'. Why is this still not working?

[JamesCherrill]

I will definitely print the stacktrace from now on.
See line 26

You're still not displaying the Process's error stream - which is where error messages may be waiting. (all you need is a pb.redirectErrorStream();)

Which OS are you using?

ps Just a question: Why all the faffing about with a byte array stream?

[brothman01]

You're still not displaying the Process's error stream - which is where error messages may be waiting. (all you need is a pb.redirectErrorStream();

but I have the e.printStackTrace() to print the error. Do I need to replace it with "pb.redirectErrorStream()"?

ps Just a question: Why all the faffing about with a byte array stream?

I have never done this before programmatically and when I looked this up the examples used byte array streams so I assumed that was the way to do it. If you know of an easier or better way I will use that instead.

@JamesCherrill

[JamesCherrill]

You are confusing exceptions and process errors. You need the catch for your java program errors. Your process may issue its own error messages which will go to its error stream. For those you have to capture the error stream or redirect it to std out which you are already handling
I'm just on my phone right now, so I'll answer the array stream question later.

[JamesCherrill]

If you want to see the output and the errors from your process the the simplest way is to tell your process to inherit its output and error streams from your java program...

    ProcessBuilder pb =  new ProcessBuilder(... etc
    pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
    pb.redirectError(ProcessBuilder.Redirect.INHERIT);

This will make your process send its normal output to System.out and its error messages to System.err. That's all you need.

ps: If my guess is right there is a message about the "~" directory waiting for you in the process's error stream ;)

[brothman01]

actually when I try to add
pb.redirectOutput(ProcessBuilder.Redirect.INHERIT); pb.redirectError(ProcessBuilder.Redirect.INHERIT);

I get the error:
File: /repo/java/mySync/mySync.java [line: 145]
Error: /repo/java/mySync/mySync.java:145: cannot find symbol
symbol : variable Redirect
location: class java.lang.ProcessBuilder

As I'm sure you can tell I haven't used the ProcessBuilder class before. I am just trying to run shell commands from my java application. Do you know of an easier way to execute shell commands?

[JamesCherrill]

This really is the easiest way... for example here's all the code to run a ls (list directory contents) command and display the output and any errors.

    ProcessBuilder pb =  new ProcessBuilder("ls");
    pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
    pb.redirectError();
    pb.start(); 

(plus 4 lines if you wrap that in a try/catch/printStackTrace())

You don't need all that
ArrayList<String> commands = new ArrayList<String>( Arrays.asList(cmd.split(" ") ) );
stuff. Just
ProcessBuilder pb = new ProcessBuilder(cmd.split(" "));

Re your latest errors:
Are you using an obsolete version of Java? You should be on Java 8 by now, but the above code just needs Java 7. If you're not on a current Java version then you are danger to yourself and those around you - you are missing years and years of critical security fixes and expose yourself and everyone around you to highly malignant malware.

[brothman01]

woah, maybe. I used to be a comp sci student but now I just write java code for fun so I am using the same programs that I was using in school. I used drjava to write most code, and I would switch to Eclipse, but now I cannot run files in Eclipse because when I press the debug button it says "the ant file cannot be found". I know Eclipse is the better program, but I am trapped using DrJava and based on what you just said it is probably using an outdated version of Java. How can I make Eclipse work (not do anything special, but function the way it is supposed to)

[JamesCherrill]

Yes, I did come on strong about obsolete versions of Java, but I'm not going to apologise. There really are hideous exploits for older versions of Java in the wild, and if you don't replace them with a currently-supported version you are definitely at risk.

Anyway....
A clean download/install of the the current JDK and Eclipse versions should run perfectly. You could also try NetBeans, which is my personal preference for Java development.
What OS are you on? Looks like Linux or MacOS from your use of the ~

[brothman01]

Yes, Mac os Sierra. I will try Netbeans, thanks. Is there anything I need to know for setup?

[JamesCherrill]

Go to http://www.oracle.com/technetwork/java/javase/downloads/jdk-netbeans-jsp-142931.html and download the JDK/NetBeans bundle for Mac OSX. That's everything you need in one package.
It's a standard Mac disk image installer, coudn't be simpler.

When you get your code running with all the errors displayed you will find that the ~ is a problem. Come back here when you get to that stage and I'll walk you through the fix.

JC

[brothman01]

oh NetBeans seems great but I am still having problems executing the shell commands. When I run my program I get the following error:

Sync has begun. 12:45:39 12/15/2016 ==========
Directions:
[sync ~/.bash_profile -> !!/home/test/debug/folder/bash_profile]
mkdir /Users/ben/Desktop/synx/home
Exception in thread "main" java.lang.RuntimeException: Uncompilable source code - unreported exception java.io.IOException; must be caught or declared to be thrown
    at mySync.shell(mySync.java:148)
    at mySync.<init>(mySync.java:67)
    at mySync.main(mySync.java:169)
/Users/ben/Library/Caches/NetBeans/8.2/executor-snippets/run.xml:53: Java returned: 1
BUILD FAILED (total time: 0 seconds)

Line 148 of mySync.java is pb.start();:

 private void shell(String cmd)
 {
    ProcessBuilder pb =  new ProcessBuilder("ls");
    pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
    pb.redirectError();
    pb.start(); 
  }

What is the problem here and how do I fix it?

[AssertNull]

Scroll down to near the bottom for the start() function.

https://docs.oracle.com/javase/8/docs/api/java/lang/ProcessBuilder.html

public Process start()
throws IOException

Look at your error...

Exception in thread "main" java.lang.RuntimeException: Uncompilable source code - unreported exception java.io.IOException; must be caught or declared to be thrown

You aren't catching the IOException in your code. Try this...

   private void shell(String cmd)
   {
        ProcessBuilder pb =  new ProcessBuilder("ls");
        pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
        pb.redirectError();
        try 
        { 
            pb.start();
        }
        catch (IOException ex) 
        {
            System.out.println(ex);
        }
    }

The nice thing about NetBeans is that it'll give you a yellow lightbulb with a red exclamation point next to the problem line. Hover over that for the error. Then click on it and it gives you some options to fix it. I clicked "Surround statement with Try-Catch" and it created the code to fix it (note: I added the System.out.println(ex) line).

[brothman01]

I used your code for the shell() function but it is not executing the command.

  private void shell(String cmd)
   {
        ProcessBuilder pb =  new ProcessBuilder( cmd );
        pb.redirectOutput( ProcessBuilder.Redirect.INHERIT );
        pb.redirectError();
        try 
        { 
            pb.start();
            System.out.println( "Command: " + cmd );
        }
        catch (IOException ex) 
        {
            // System.out.println(ex);
        }
    }

as you can see I added a print statement in the try-block, but that print statement never fires which makes me think that the ProcessBuilder instantiation fails. I call the shell() function above with the following code:

shell("mkdir /Users/ben/Desktop/synx/home");

why does this not make the directory 'home'? BTW there is no output from the System.out.println(ex) part.

[JamesCherrill]

Seriously - you commented out line 13? Are we wasting our time here?

[brothman01]

my fault, I was trying to debug but I guess that was the exact line I needed.. mistake.

anyway, the output:

run:
Sync has begun. 17:10:21 12/15/2016 ==========
java.io.IOException: Cannot run program "mkdir /Users/ben/Desktop/synx/home": error=2, No such file or directory
java.io.IOException: Cannot run program "mkdir /Users/ben/Desktop/synx/home/test": error=2, No such file or directory
java.io.IOException: Cannot run program "mkdir /Users/ben/Desktop/synx/home/test/debug": error=2, No such file or directory
java.io.IOException: Cannot run program "mkdir /Users/ben/Desktop/synx/home/test/debug/folder": error=2, No such file or directory
BUILD SUCCESSFUL (total time: 0 seconds)

I see that the error says that the directory does not exist, but that is not true for the first command, which would make the directory structure required for the second command and so on...
Essentially, why am I getting that error if the '/Users/ben/Desktop/synx' directory does exist and has files in it already?

[rproffitt]

And there it is. I was wondering if your OS actually has a binary called mkdir. On many systems I have to launch a shell (bash or other) then add the command to the bash command line.

My second thought over all this is the folder manipulation could be done in native Java. Why shell out and break your app from running everywhere?

[AssertNull]

Quick experiment. Put any command you're running into a shell script. Run the shell script. Any errors? If yes, then not a Java-related problem.

It can't hurt to have the full path to ls, mkdir, etc. just to preclude a path problem.

Once it all works in the shell program, have the Java ProcessBuilder command run the shell program. See if that works. Again, check all paths and hardcode as needed. Also check permissions.

[brothman01]

And there it is. I was wondering if your OS actually has a binary called mkdir. On many systems I have to launch a shell (bash or other) then add the command to the bash command line.

I saw this too and it looked fishy to me. Based on the error output, Cannot run program "mkdir /Users/ben/Desktop/synx/home" the program seems to be trying to run a program called 'mkdir /Users/ben/Desktop/synx/home' rather than run the mkdir program with the parameter ' /Users/ben/Desktop/synx/home'. Did I interperet the error correctly? If so, how do I tell the terminal to execute mkdir and then pass arguments to it?

My second thought over all this is the folder manipulation could be done in native Java. Why shell out and break your app from running everywhere?

I thought of that too, and I know folder creation can be done from native java, but I also need to execute other commands with this shell() function such as cp -R somefile so I am trying to just execute strings of directions that would work in a terminal window so that my code will be more versatile/reusable in the future.

[rproffitt]

Sorry but I'll try to pair this down. Look in your bin folder (varies with the OS) for a mkdir app. Here there is no such thing as that's built into my bash (Unix console) and the system I looked at didn't need an external mkdir in the bin.

Also open a shell and try
type mkdir

It may tell you it's builtin or where the bin path is and then you can use the full path or know if you need to shell to bash with a command line.

https://www.tutorialspoint.com/execute_bash_online.php is an example we can kick around a little.
Try
type eval
there.

[brothman01]

I got it to work correctly. Thanks

   private void shell(String cmd) {
        ProcessBuilder pb;
           String param1 = "";

           if ( cmd.startsWith("mkdir ") ) {

                param1 = cmd.substring( 6, cmd.length() ); // make 'param1' the parameter for the mkdir
                pb =  new ProcessBuilder("mkdir", param1 );
           } else {
               pb =  new ProcessBuilder("mkdir", cmd ); // DEBUG
           }

        pb.redirectOutput( ProcessBuilder.Redirect.INHERIT );
        pb.redirectError();
        try 
        { 
            pb.start();
            System.out.println( "Command: " + cmd ); // DEBUG
        }
        catch (IOException ex) 
        {
            System.err.println(ex);
        }
    }

I knew the 'mkdir' binary was there and could be executed from my terminal, but the error was saying that java was unable to execute 'mkdir /Users/ben/Desktop/synx/home'. It was trying to execute that entire string as one file. If I added a file called 'mkdir /Users/ben/Desktop/synx/home.sh' to my bin and tried to run it from the terminal, it probably would have worked, lol. ProcessBuilder requires that arguments be separate Strings and separated by commas which I am now doing. Ok, thank you everyone.

[JamesCherrill]

You always need to split up the command and it parameters. Don't use special-case code like that, you should split up every command, ie

 new ProcessBuilder(cmd.split(" "));

In summary, the easiest general way to execute a command under OSX from Java is to use something like: (also deals with ~ like you would expect)

   void runCommand(String command) {
        command = command.replaceAll("~", System.getProperty("user.home"));
        try {
            new ProcessBuilder(command.split(" ")).
                    redirectError(ProcessBuilder.Redirect.INHERIT).
                    redirectOutput(ProcessBuilder.Redirect.INHERIT).
                    start();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }