ts.torch_handler.request_envelope package

Submodules

ts.torch_handler.request_envelope.base module

Base class for all RequestEnvelope.

A request envelope reformats the inputs/outputs of a call to a handler. It translates from formats specific to a model orchestrator like Seldon or KServe to a set of flat Python items, and vice versa.

class ts.torch_handler.request_envelope.base.BaseEnvelope(handle_fn)

Bases: ABC

Interface for all envelopes. Derive from this class, replacing the abstract methods

abstract format_output(data)

handle(data, context)

The Input Requests and Response are handled here.

abstract parse_input(data)

ts.torch_handler.request_envelope.body module

Default class for all handlers. Grabs the data out of the request body.

class ts.torch_handler.request_envelope.body.BodyEnvelope(handle_fn)

Bases: BaseEnvelope

Gets the key “body” from the input data, returns a raw list

format_output(data)

parse_input(data)

ts.torch_handler.request_envelope.json module

Uses JSON formatted inputs/outputs, following the structure outlined in https://www.tensorflow.org/tfx/serving/api_rest

class ts.torch_handler.request_envelope.json.JSONEnvelope(handle_fn)

Bases: BaseEnvelope

Implementation. Captures batches in JSON format, returns also in JSON format.

format_output(data)

parse_input(data)

ts.torch_handler.request_envelope.kserve module

The KServe Envelope is used to handle the KServe Input Request inside Torchserve.

class ts.torch_handler.request_envelope.kserve.KServeEnvelope(handle_fn)

Bases: BaseEnvelope

This function is used to handle the input request specified in kserve format and converts it into a Torchserve readable format.

Parameters:

Format (data - List of Input Request in kserve) –

Returns:

Returns the list of the Input Request in Torchserve Format

Return type:

[list]

format_output(data)

Returns the prediction response and captum explanation response of the input request.

Parameters:

outputs (List) – The outputs arguments is in the form of a list of dictionaries.

Returns:

The response is returned as a list of predictions and explanations

Return type:

(list)

parse_input(data)

ts.torch_handler.request_envelope.kservev2 module

Module contents