NVIDIA 550 DKMS patches for kernel 7.x compatibility + apt hook
-
Updated
Jun 7, 2026 - Shell
#
rtx-2070
Here are 8 public repositories matching this topic...
Real PyTorch inference server comparing FCFS, EagerContBatch, and ChunkedPrefill on RTX 2070. Key finding: ChunkedPrefill is not a throughput optimizer but a fairness mechanism — it reduces worst-case decode stalls from 58.6ms to 34.1ms while EagerContBatch maximizes mean TTFT and throughput.
python gpu latency scheduling inference pytorch throughput serving llm continuous-batching chunked-prefill rtx-2070
-
Updated
Jul 12, 2026 - Python
Measures actual GPU costs for LLM prefill and decode on RTX 2070 (8GB) to validate simulation parameters. Key findings: prefill converges to 30-70 us/token at long sequences; decode is memory-bandwidth-bound (constant with prefix length, 5300-10800 us/token single-request); simulation defaults are correct for server-level amortized batching.
python benchmark gpu inference pytorch decode profiling memory-bandwidth prefill kv-cache llm rtx-2070
-
Updated
Jul 10, 2026 - Python
Measures real KV cache costs on GPU: formula validation (36 KB/token exact), decode memory-bound (3% variation), compaction bandwidth 19-67 GB/s, and counter-intuitive finding: compaction during active decode runs 30% faster due to CUDA stream overlap.
-
Updated
Jul 12, 2026 - Python
Empirical study of speculative decoding with Qwen2-0.5B (draft) and Qwen2-1.5B (target) on a single RTX 2070. Measures token acceptance rate, speedup, KV sync overhead, and validates the Leviathan et al. (2023) analytical model. Shows why cost_ratio=1.18x (not 3.1x as expected) causes failure and what hardware conditions make it viable.
benchmarking transformers pytorch analytical-model kv-cache llm-inference speculative-decoding qwen2 consumer-gpu rtx-2070
-
Updated
Jul 15, 2026 - Python
Profiles 6 attention variants on RTX 2070 (sm75): naive O(n²), SDPA auto, FlashAttention, math backend, mem_efficient, sliding window, and GQA. Key findings: sdpa_default is 7.64x faster than naive at seq=2048; memory grows 25x (naive) vs 2.4x (sdpa) for 32x longer sequences; kernel choice matters more than model architecture at long sequences.
python kernel gpu pytorch transformer attention profiling memory-efficient gqa llm flash-attention rtx-2070
-
Updated
Jul 12, 2026 - Python
Measures real speculative decoding speedup using the official HuggingFace assistant_model API across 4 model pairs and output lengths up to 512 tokens. Best result: distilgpt2->gpt2-medium achieves 1.747x speedup at 512 output tokens. Validates that cost_ratio and output_length are the key parameters.
python gpu transformers inference pytorch throughput llm speculative-decoding draft-model rtx-2070 assisted-generation
-
Updated
Jul 13, 2026 - Python
Measures real prefix cache costs on GPU across 3 experimental versions. Key findings: 2.41x speedup with prefix=512; multi-turn speedup grows to 2.06x over 10 turns; batch sharing breakeven at n=2 (prefix=512) vs n=12 (prefix=128); LFU cache with Zipf alpha=2.0 achieves 82% hit rate with only 4 cache slots.
python gpu inference pytorch transformer profiling serving multi-turn kv-cache llm prefix-cache rtx-2070
-
Updated
Jul 13, 2026 - Python
Improve this page
Add a description, image, and links to the rtx-2070 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the rtx-2070 topic, visit your repo's landing page and select "manage topics."