A collection of tools to measure inference latency for foundations models in Amazon Bedrook.
- Measure latency for LLMs bedrock-latency-benchmark.ipynb. Measure across scenarios like: model latency with different input/output lengths, model1 vs model2, same model different AWS Regions.
- Measure latency for text-to-image models - Coming soon.