Cant build python+onnx+ternsorrtllm backends r24.04 about server HOT 3 OPEN

Tracking ticket: [DLIS-6397]

from server.

Hi @gulldan, compose.py doesn't currently support the TensorRT-LLM backend (DLIS-6397).

You should be able to achieve something similar by using build.py with:

--backend tensorrtllm:r24.04
--backend python:r24.04
--backend onnxruntime:r24.04

https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/customization_guide/build.html#building-with-docker

Let us know if this helps for your use case.

from server.

thank you.

i tried

./build.py --backend tensorrtllm:r24.04 --backend python:r24.04 --backend onnxruntime:r24.04 --enable-gpu --build-type Release --target-platform linux --endpoint grpc --endpoint http

but its failed
build_log.txt

Host info
Linux 6.5.0-35-generic #35-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 26 11:23:57 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Docker version 26.1.2, build 211e74b
cmake version 3.28.4
python 3.11.6
GeForce RTX 4090
Driver Version: 550.54.15

from server.