Pinned Loading

1. ai-dynamo/modelexpress ai-dynamo/modelexpress Public

   Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.

   Python 151 77

2. ai-dynamo/aiperf ai-dynamo/aiperf Public

   AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

   Python 662 180

3. NVIDIA-NeMo/Guardrails NVIDIA-NeMo/Guardrails Public

   NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

   Python 7.1k 835

4. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 8.1k 1.6k

5. triton-inference-server/perf_analyzer triton-inference-server/perf_analyzer Public

   Python 160 50

6. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python