Deploying Python models with the SKIL command line interface

This post introduces skil-python, a Python library created for the Skymind Intelligence Layer (SKIL). skil-python can be used to create data science experiments in SKIL, deploy machine-learning models via REST, and query models with new data. We will go over the steps required to start a SKIL server, train a model on the MNIST data, and deploy and query the model using skil-python. The code used for this blog post is here.

Starting the SKIL Server

Before using skil-python, there must be a SKIL server to access. So let's start by setting up a temporary SKIL server. We will use Docker, so first install Docker if it is not already installed on your machine. Look here to get Docker for your machine. Note that the Docker virtual machine should be configured with 8 gigabytes of RAM or more before starting the SKIL server. Once Docker is up and running, enter the following command on the command line. This will start the temporary SKIL server.

docker run --rm -it -p 9008:9008 -p 8080:8080 skymindops/skil-ce:1.1.4-2 bash /start-skil.sh

Let's unpack this command. The --rm flag tells Docker to automatically delete the Docker container when it is exited. The first -p flag publishes the 9008 port from the host to the 9008 port within the Docker container. This means if you connect to the host using port 9008, you will be routed to the container via port 9008. The second -p flag does the same, except with the 8080 port. skymindops/skil-ce:1.1.4-2 specifies the Docker image that contains all the software to run your code. Lastly, bash /start-skil.sh specifies the instructions to run to start the SKIL server, so you don't have to do this yourself. Once the command is finished running, the SKIL UI can be accessed at http://localhost:9008.

Note that you may need to pull the Docker image from the Skymind repo with the following command before using docker run:

docker pull skymindops/skil-ce:1.1.4-2 

Configuring SKIL access with the pyskil CLI

Now that the SKIL server has started, we will go over the code to connect to our SKIL machine, process the data, train the model, and add the model to a SKIL experiment. These things can be done within the SKIL UI, but we will show how to do it all within a Python script using skil-python. To do so, you first install skil-python from Python's package index like this:

pip install tensorflow==1.10 keras==2.2.2 skil==0.2.8

You'll notice that we pin the Keras and TensorFlow versions you will use for training to make ensure you can run this example in the future. Apart from the skil-python library itself, this will also expose an easy to use command line interface (CLI) called pyskil that you can use to configure your SKIL access - just run

pyskil configure

and pyskil will automatically walk you through providing SKIL credentials (user name and password) and the SKIL connection string (host and port). For your first run you can simply accept all defaults and confirm, which should result in the following JSON configuration:

{
    "username": "admin", 
    "host": "localhost", 
    "password": "admin", 
    "port": "9008"
}

pyskil has other functionality as well that we will see later on in this post. For now, let's turn to training a simple model in Python that we can deploy.

Training a simple Keras model on MNIST

We will use the MNIST dataset to train our model. Recall, the MNIST dataset consists of images containing handwritten digits from 0 to 9. Each image contains 28 by 28 pixels or 784 pixels in total. The task at hand is to create a model to read in all the pixels of an image and predict the digit that is contained in the image.

The MNIST dataset is already part of the keras.datasets library. Simply use the load_data function from the mnist class in keras.datasets to read in the data. We can then reshape the 28 x 28 images to a 784 dimensional vector and scale the pixels to have values from 0 to 1 by dividing by 255. Lastly, we change the integer labels to one-hot categorical vectors.

import keras
from keras.datasets import mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

The data is now ready to be passed as input to a neural network. The architecture of the model is specified below using the Keras library in python. There are two dense layers with Relu activations and an output dense layer with a softmax activation over all the 10 classes. Thus, the classifier will output ten scalars, each representing the probability that the input image contains one of the ten digits.

from keras.models import Model
from keras.layers import Dense, Dropout, Input

inp = Input((784,))
x = Dense(512, activation='relu')(inp)
x = Dropout(0.2)(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.2)(x)
out = Dense(num_classes, activation='softmax')(x)
model = Model(inp, out)

You can then compile the model and run the fit function to train the network on our training data.

model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
model.fit(x_train, y_train,
          batch_size=128,
          epochs=10,
          verbose=1,
          validation_data=(x_test, y_test))

Adding the Trained Model to a SKIL Experiment

Now that our model is fully trained, we can save the model to a SKIL experiment, which you can create with pyskil like this:

pyskil init-experiment -f exp.json

This command will create a SKIL experiment for you and store it in exp.json in JSON format. You can now easily load this experiment from Python and use your Keras model to create a SKIL model that we can deploy.

from skil import Experiment, Model as SkilModel

experiment = Experiment.load('exp.json')
skil_model = SkilModel(model, experiment=experiment)

Model Deployment with SKIL

To deploy a SKIL model, all you need is a SKIL Deployment, which you can also quickly create with the pyskil CLI:

pyskil init-deployment -f dep.json

You can now load the deployment and use it to create a SKIL service by calling deploy on you SKIL model.

from skil import Deployment

deployment = Deployment.load('dep.json')
service = skil_model.deploy(deployment=deployment)

Querying the Model

Let's query the model using the MNIST test set. We use a for loop to iterate through each example in the test set, obtain the deployed model probabilities for each digit class using the predict_single function of service, and compare it to the actual label.

num_correct = 0
for i in range(10000):
    result = service.predict_single(x_test[i])
    num_correct += np.argmax(result) == np.argmax(y_test[i])

We can then output the test accuracy, which concludes this blog post!

print("Test Accuracy: %s" % (float(num_correct) / 10000,))

Running it all again

If you now run the whole script one more time, your Keras model will be trained again, but SKIL is smart enough to reuse your pre-configured experiment, deployment and service instances. This makes model deployment fairly simple - you configure it once and use it whenever you want.

Recap

Today we've shown basic tasks that can be done using skil-python, such as connecting to a SKIL server, creating workspaces and experiments, and deploying models. This was done by showing how to add a MNIST classifier to Skil, deploying the model, and querying it using the test set. You should now be ready to integrate SKIL into your data science workflow using your own python scripts!