Running Mobius Spark – The system cannot find the path specified error

Mobius is a project to enable using C# to develop against Apache Spark. The project page on GIT has the directions to set up and use Mobius. However, when I tried those, I ran into “The system cannot find the path specified” error. Essentially, the error is because the various directory and file paths referred to in the scripts did not exist on my machine.

The entry point for running a Mobius project is the “sparkclr-submit.cmd” found in the “..\runtime\scripts” folder.  This script gets the various PATH variables and in turn calls the “spark-submit.cmd” command found in the bin folder of the Spark installation directory.

Obviously, you need to have Spark installed. Then, ensure that you have set up all the PATH variables referred to in the “sparkclr-submit.cmd”; these inlude the following variables among others

  1. SPARK_HOME
  2. JAVA_HOME
  3. HADOOP_HOME
  4. SPARKCLR_HOME

Next, ensure that the various paths implied in the script are valid on your machine. For example, the sparkclr-submit.cmd file uses the JAVA_HOME variable to call and submit to the java program. And the command looks something like this:

“%JAVA_HOME%\bin\java” -cp “%LAUNCH_CLASSPATH%”

On my machine though, the path “%JAVA_HOME%\bin\java” did not exist and the actual path on my machine was “%JAVA_HOME%\java” (notice that “bin” is not present). This could have been because of something I did when I first installed java.

Likewise, ensure that all the other paths implied in the sparkclr-submit.cmd script are valid. If you still keep getting the “cannot find the path specified” error, you may have to hunt through the various paths implied in the child scripts that the sparkclr-submit.cmd calls.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s