Getting started with the SKIL Deployment Client

Previously, we showed how to interact with SKIL through endpoints with raw JSON requests. In this article, we'll use the SKIL endpoints through the SKIL deployment client. This is the recommended method, since it is less error prone and easier to use than raw JSON requests.

Getting started

If you're new to SKIL, take a look at the Quickstart guide to get SKIL CE running.

The source code used below can be found in the SKIL examples repo.

What is the SKIL deployment client?

The SKIL deployment client is a Java API for handling SKIL client requests. It's intuitive and easy to use compared to raw JSON requests. It provides builder patterns for building a requests object. The responses have their own class objects.

Adding the Maven dependency

To add the SKIL deployment client in your Maven project, copy and paste the code snippet below in your pom.xml under the <dependencies> tag:

<dependency>
    <groupId>io.skymind</groupId>
    <artifactId>skil-daemon-client</artifactId>
    <version>LATEST</version>
</dependency>

Imports

You'll need the following imports to follow along:

import io.skymind.auth.rest.LoginResponse;
import io.skymind.deployment.client.SKILDeploymentClient;
import io.skymind.deployment.model.ModelEntity;
import io.skymind.deployment.rest.CreateDeploymentRequest;
import io.skymind.deployment.rest.DeploymentResponse;
import io.skymind.deployment.rest.ImportModelRequest;
import io.skymind.deployment.rest.UpdateModelStateRequest;
import io.skymind.skil.client.errors.ClientException;

import java.text.MessageFormat;
import java.util.List;

Connecting the client to the SKIL server

To connect your Java client to SKIL, you need to pass a path string to the constructor of SKILDeploymentClient class, specifying the host and port of the running SKIL server as:

String host = "localhost";
String port = "9008";
String path = MessageFormat.format("http://{0}:{1}", host, port);

SKILDeploymentClient skilDeploymentClient = new SKILDeploymentClient(path);

Endpoints covered here

In this blog, we'll explore the following endpoints:

  1. Login
  2. Deployment
  3. Model
  4. Transform
  5. KNN

1. Login

Not all requests require authorization, but to get a prediction returned, you need to pass along an authorization token in the request headers.

The authorization token is a JWT token that stores your credentials as a user in an encrypted form and lets the server know that you're an entity authorized to carry out the requested operation.

Use the following code to login and receive an authorization token from the SKIL server:

String userId = "admin";
String password = "admin";

/* Getting authorized in the deployment client */
// The following code will authenticate you as a user in the SKIL Server
LoginResponse loginResponse = skilDeploymentClient.login(userId, password);

You'll receive a LoginResponse object with the authorization token present. To read the authorization token, do the following:

// To get the token...
print(MessageFormat.format("Authorization Token: {0}", loginResponse.getToken()));

2. Deployment

A SKIL deployment is “a logical group of models, transforms, and KNN endpoints”. DL4J models are usually associated with data pipeline transforms that convert data to a format that a neural network can ingest. So a SKIL deployment just deploys the model and its transforms and KNNs, and exposes it through endpoints to client apps.

The following shows how to use endpoints related to a SKIL deployment:

A. Listing the deployments

To receive a list of all the deployments:

// Getting the list of deployments in the SKIL server
List<DeploymentResponse> deployments = skilDeploymentClient.getDeployments();

A DeploymentResponse object has all information related to a single deployment. You can read the data from it as:

if(deployments.size() > 0) {
    DeploymentResponse deploymentResponse = deployments.get(0);
    
    print(MessageFormat.format("Deployment ID: {0} | Name: {1} | Slug: {2} | Status: {3} | Other Details:\n{4}", 
        deploymentResponse.getId(),
        deploymentResponse.getName(),
        deploymentResponse.getDeploymentSlug(),
        deploymentResponse.getStatus(),
        deploymentResponse.getBody()));
}
B. Get deployment by ID

To get a deployment by ID:

// Getting a deployment by ID
if(deployments.size() > 0) {
    long deploymentId = Long.parseLong(deployments.get(0).getId());
    DeploymentResponse deployment = skilDeploymentClient.getDeployment(deploymentId);
}
C. Adding a deployment

For adding a deployment, you need to pass a CreateDeploymentRequest object by passing it the new deployment's name. Such as:

String newDeploymentName = "New-Deployment";
DeploymentResponse addedDeploymentResponse = null;
try {
    addedDeploymentResponse = skilDeploymentClient.addDeployment(new CreateDeploymentRequest(newDeploymentName));
} catch (ClientException e) {
    e.printStackTrace();
}

Once you do that, you'll have the newly created deployment inside the addedDeploymentResponse variable.

D. Get all deployment models

To get all the models in a deployment:

// Getting all the models in a deployment
if(deployments.size() > 0) {
    long deploymentId = Long.parseLong(deployments.get(0).getId());
    List<ModelEntity> modelEntities = skilDeploymentClient.getModels(deploymentId);
}

A ModelEntity contains all the model-related information. For example:

if(deployments.size() > 0 && modelEntities.size() > 0) {
    ModelEntity modelEntity = modelEntities.get(0);
    print(MessageFormat.format("Model ID: {0} | Name: {1} | Scale: {2} | State: {3} | File location:\n{4}",
            modelEntity.getId(),
            modelEntity.getName(),
            modelEntity.getScale(),
            modelEntity.getState(),
            modelEntity.getFileLocation()));
}

3. Model

Models are contained within a deployment, so you'll have to specify the deployment with each model-related code. Let's see what model features are available.

String transformFileLocation = "file:///tmp/transform.json",
        reimportedTransformFileLocation = "file:///tmp/reimported_transform.json";
String knnFileLocation = "file:///tmp/knn.bin",
        reimportedKnnFileLocation = "file:///tmp/reimported_knn.bin";
A. Add a model

To add a model, you need to build a model request object using ImportModelRequest.builder():

// Adding a new model to a deployment
String modelFileLocation = "file:///tmp/model.zip";
String newModelName = "New-Model";
String singleModelEndpointUrl = "new_model_endpoint";
ModelEntity addedModelEntity = null;
try {
    addedModelEntity = skilDeploymentClient.addModel(deploymentId,
            ImportModelRequest.builder()
                    .name(newModelName)
                    .fileLocation(modelFileLocation)
                    .scale(1)
                    .uri(singleModelEndpointUrl)
                    .build());
} catch (ClientException e) {
    e.printStackTrace();
}
B. Reimport a model

To reimport a model:

// Reimporting a model to a model ID in a deployment
String reimportedModelFileLocation = "file:///tmp/reimported_model.zip";
String reimportedModelName = "Reimported-Model";
ModelEntity reimportedModelEntity = null;
if (addedModelEntity != null) {
    reimportedModelEntity = skilDeploymentClient.reimportModel(
            deploymentId,
            (Long) addedModelEntity.getId(),
            ImportModelRequest.builder()
                    .name(reimportedModelName)
                    .fileLocation(reimportedModelFileLocation)
                    .build());
}
C. Delete a model

To delete a model:

String deletedModelResponse = null;
if (addedModelEntity != null) {
    deletedModelResponse = skilDeploymentClient.deleteModel(
            deploymentId,
            (Long) addedModelEntity.getId()
    );
}
D. Setting model state

To set a model state, use UpdateModelStateRequest.builder():

ModelEntity stateChangedModel = null;
if (addedModelEntity != null) {
    stateChangedModel = skilDeploymentClient.setModelState(
            deploymentId,
            (Long) addedModelEntity.getId(),
            UpdateModelStateRequest.builder()
                    .state(ModelEntity.SetState.STOP)
                    .build()
    );
}

4. Transform

Transform works the same way. Just specify the subType in ImportModelRequest.builder().

For transforms, it's ModelEntity.ModelType.TRANSFORM.name().

5. KNN

Ditto KNNs. Just specify the subType in ImportModelRequest.builder().

For KNNs, it's ModelEntity.ModelType.KNN.name().