Configuring SKIL Notebooks for Large Neural Networks

More ambitious SKIL projects require larger neural networks. Neural networks are memory-hungry, so the default settings in Zeppelin notebooks may not suffice. Hitting memory limits produces errors like this:


To fix memory issues in Zeppelin, increase the amount of JVM RAM available. Zeppelin's default configuration provides 8 GB of RAM on the JVM, and 4 GB of off-heap space. You can expand these by changing the DEFAULT_ZEPPELIN_JVM_ARGS environment variable and deleting the Zeppelin interpreter process in the Processes tab.

To set the environment variable, either specify it when you launch the Docker container (with -e), or add it to /etc/skil/ (or /etc/profile.d/ if /etc/ doesn't exist)

The following line shows the maximum settings available if you have a K80 GPU:

DEFAULT_ZEPPELIN_JVM_ARGS=-Xmx16g -Dorg.bytedeco.javacpp.maxbytes=16G -Dorg.bytedeco.javacpp.maxphysicalbytes=16G -Dorg.nd4j.versioncheck=false -Dorg.deeplearning4j.config.custom.enabled=false

If you are training on CPUs alone, you are only limited to your RAM. After you set the environment variable, you'll need to stop and re-start the SKIL system with the commands:

sudo systemctl stop skil
sudo systemctl start skil