Skip to content
#

hpml

Star

Here are 6 public repositories matching this topic...

Language: All
Filter by language
All 6 Jupyter Notebook 3 Julia 1 Python 1 Shell 1

rutujaingole / Optimizing-LLM-Inference-using-NVIDIA-Dynamo-and-TorchDynamo

Star 1

The goal of the project is to benchmark and optimize BERT inference using different backends—PyTorch eager mode, TorchDynamo (Inductor backend), and NVIDIA Triton Inference Server. We use GLUE SST-2 samples for evaluation and compare performance through profiling, kernel timing, and latency analysis.

machine-learning machine-learning-algorithms pytorch high-performance-computing profiling bert nvidia-gpu hpml torchdynamo llm-inference nvidia-dynamo
  • Updated May 10, 2025
  • Jupyter Notebook

alex-is-busy-coding / speculative-rag

Star 0

An implementation of Speculative RAG exploring latency-quality trade-offs in multi-draft retrieval. Features batched parallel drafting via vLLM and log-probability verifier selection for fast, high-quality QA on a single A100 GPU.

nlp benchmarking gpu latency high-performance-computing throughput quantization faiss rag vector-search hpml a100 llm vllm retrieval-augmented-generation speculative-rag
  • Updated Mar 14, 2026
  • Python

Improve this page

Add a description, image, and links to the hpml topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the hpml topic, visit your repo's landing page and select "manage topics."

Learn more