Explicitly state Databricks runtime version

[jacobtomlinson]

Closes #246

The 12.2 LTS runtime doesn't seem to work with the databricksruntime/gpu-conda:cuda11 base image. Updating the instructions to explicitly recommend users choose the 10.4 LTS runtime.

[jacobtomlinson]

I tried creating a notebook using the cluster and am now seeing another error.

java.lang.Exception: Cannot run program "virtualenv" (in directory "/local_disk0/pythonVirtualEnvDirs"): error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.apache.spark.util.Utils$.executeCommand(Utils.scala:1507)
	at org.apache.spark.util.Utils$.executeAndGetOutput(Utils.scala:1533)
	at org.apache.spark.util.DatabricksUtils$.createPythonVirtualEnv(DatabricksUtils.scala:602)
	at org.apache.spark.util.DatabricksUtils$.$anonfun$getOrCreatePythonVirtualEnvDir$1(DatabricksUtils.scala:722)
	at org.apache.spark.util.DatabricksUtils$.withStripedLock(DatabricksUtils.scala:411)
	at org.apache.spark.util.DatabricksUtils$.getOrCreatePythonVirtualEnvDir(DatabricksUtils.scala:708)
	at com.databricks.backend.daemon.driver.DBUtilsPythonEnvManager.getPythonExecutable(DBUtilsPythonEnvManager.scala:27)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.launchPython(PythonDriverLocal.scala:315)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.startPython0(PythonDriverLocal.scala:956)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.startPython(PythonDriverLocal.scala:1009)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.<init>(PythonDriverLocal.scala:1032)
	at com.databricks.backend.daemon.driver.PythonDriverWrapper.instantiateDriver(DriverWrapper.scala:697)
	at com.databricks.backend.daemon.driver.DriverWrapper.setupRepl(DriverWrapper.scala:335)
	at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:224)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Throwable: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 15 more

[jacobtomlinson]

I'm seeing that same error with the databricksruntime/gpu-conda:cuda11 base image to perhaps it just isn't compatible with Databricks anymore?

[jacobtomlinson]

Things still aren't working with the changes, it looks like Databricks is looking for virtualenv but isn't finding it in the place it is expecting. Converting this to a draft while we explore how to resolve.

[skirui-source]

@jacobtomlinson I was able to reproduce the databricks instructions on dt08 (not sure why workstation was taking so long).
But In testing Rapids notebook, I am also seeing the same error as yours:

Failure starting repl. Try detaching and re-attaching the notebook.

java.lang.Exception: Cannot run program "virtualenv" (in directory "/local_disk0/pythonVirtualEnvDirs"): error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.apache.spark.util.Utils$.executeCommand(Utils.scala:1507)
	at org.apache.spark.util.Utils$.executeAndGetOutput(Utils.scala:1533)
	at org.apache.spark.util.DatabricksUtils$.createPythonVirtualEnv(DatabricksUtils.scala:602)
	at org.apache.spark.util.DatabricksUtils$.$anonfun$getOrCreatePythonVirtualEnvDir$1(DatabricksUtils.scala:722)
	at org.apache.spark.util.DatabricksUtils$.withStripedLock(DatabricksUtils.scala:411)
	at org.apache.spark.util.DatabricksUtils$.getOrCreatePythonVirtualEnvDir(DatabricksUtils.scala:708)
	at com.databricks.backend.daemon.driver.DBUtilsPythonEnvManager.getPythonExecutable(DBUtilsPythonEnvManager.scala:27)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.launchPython(PythonDriverLocal.scala:315)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.startPython0(PythonDriverLocal.scala:956)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.startPython(PythonDriverLocal.scala:1009)
	at com.databricks.backend.daemon.driver.PythonDriverLocal.<init>(PythonDriverLocal.scala:1032)
	at com.databricks.backend.daemon.driver.PythonDriverWrapper.instantiateDriver(DriverWrapper.scala:697)
	at com.databricks.backend.daemon.driver.DriverWrapper.setupRepl(DriverWrapper.scala:335)
	at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:224)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Throwable: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 15 more