Comments (2)
deadeyegoodwin
commented on May 26, 2024
1
Note that we just released the 0.9.0 version of TRTIS on the r18.12 branch (but the functionality you mention was also in 18.11).
The documentation for modifying the model repository while the server is running is here: https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/model_repository.html#modifying-the-model-repository
When you add or remove a model from the repository what was printed in the TRTIS console?
from server.
bezero
commented on May 26, 2024
@deadeyegoodwin Thank you very much for the quick response. TRTIS console did not change when new model was added. The problem I was having is due to my TRTIS version. I was using older version (r18.09).
from server.
Related Issues (20)
- Use Dynamic Batching on ONNX Model Exported with Half Precision HOT 1
- Use Dynamic Batching on ONNX Model Exported with Fixed Batch Dim HOT 3
- non stable inference response time with tensorflow saved model HOT 4
- Triton ensemble pipeline high CPU usage HOT 1
- Missing request params, while using ensemble model HOT 1
- Does python model support venv-pack for Creating Custom Execution Environments HOT 2
- Reload a model even when there are no changes to it in EXPLICIT mode when using the Load API HOT 4
- /v2/health/ready fails when loading a model in explicit mode HOT 1
- PermissionError: [Errno 13] Permission denied: '/home/triton-server' HOT 3
- Segmentation fault when loading new version of model HOT 1
- PyTorch Backend deterministic algorithms flag HOT 1
- Sending two "load" requests to server makes it load twice HOT 1
- Failed to allocated memory for requested buffer of size X HOT 1
- after calling unload_model capi, the memory is not completely released HOT 1
- Issue with FP16 Output for Half Precision Model HOT 10
- tritonserver:24.02-py3 cannot use onnxruntime? HOT 2
- How to deploy Triton using Docker on Jetson Orin NXīŧ HOT 3
- null
- why concurrency multiply batch_size same in perf_analyzer but throughtput are very different HOT 6
- triton inference server does not have onnx backend HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
đ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. đđđ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google â¤ī¸ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from server.