Test Session-based preproc with Triton about transformers4rec HOT 2 CLOSED

Errors when trying to export the model to ONMX:

1. When using the original code to convert NVT offset representation of sequence features to sparse features:
   "getting an error: RuntimeError: Exporting the operator repeat_interleave to ONNX opset version 9 is not supported."

2. Sara/Ronay changed the torch.repeat_interleave b np.repeat and the error changed to:
   "RuntimeError: Exporting the operator sparse_coo_tensor to ONNX opset version 9 is not supported."

Next steps:

• [] Julio was about to create an option to generate dense tensors for list columns. Follow-up on that and check if we are able to export the model to ONMX using the latest NVTabular version (which now provides dicts of tensors)
• [] Triton team has suggested us to try newest version of ONMX (current opset is 9)
• [] Try export using TorchScript instead of ONMX. That requires us to change our ensemble on Triton to use TorchScript instead of ONMX
• [] Use Python backend for Triton

P.s. We need the NVT preproc module on Triton to be able to work with multi-hot (list columns) - Issue on NVTabular

from transformers4rec.

I tried the ecom-rees NVT ETL notebook to generate config file and serve the model_nvt to triton.

• One error is about datetime columns. when we have datetime object columns as input or output, we cannot generate config files, so so we need to convert these columns to timestamp column as integer dtypes.
• Once we get rid of datetime columns, we can generate the config file, Model is loaded to the server. But when I send a client request it gives error. It looks like it complains about lambda and filter ops.

Will create a GH issue on NVTabular and followup on that.

from transformers4rec.