LLM speculative inference server for heterogeneous hardware & consumer GPUs
-
Updated
Nov 1, 2026 - C++
LLM speculative inference server for heterogeneous hardware & consumer GPUs
BestBuy Bot is an Add to cart and Auto Checkout Bot. This auto buying bot can search the item repeatedly on the ITEM page using one keyword. Once the desired item is available it can add to cart and checkout very fast. This auto purchasing BestBuy Bot can work on Firefox Browser so it can run in all Operating Systems. It can run for multiple ite…
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
Pre-built wheels for llama-cpp-python across platforms and CUDA versions
Best Buy Bullet Bot, abbreviated to 3B Bot, is a stock checking bot with auto-checkout created to instantly purchase out-of-stock items on Best Buy once restocked. It was designed for speed with ultra-fast auto-checkout, as well as the ability to utilize all cores of your CPU with multiprocessing for optimal performance.
Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090
Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with hot MoE experts on the GPU and cold experts computed on the CPU, INT8 KV cache, Docker, OpenAI-compatible API.
The Universal 245K Agent Microkernel for Local LLMs.
Open source DIY AI computing platform: Build a powerful RTX 3090 GPU rig under €1,300 for local LLM inference, training, and AI development. Complete with parts list, assembly guide, Ubuntu setup, and remote access configuration. Ideal for students, researchers, and hobbyists wanting affordable AI hardware without cloud dependencies.
🌐 Automate and manage BitBrowser tasks for efficient Google One student discount processing in one streamlined system.
NVIDIA vGPU on a consumer GeForce RTX 3090 (GA102) via legacy mdev. Working config, the exact kernel requirement, ~80 measured dead ends, and a tool to port the fix to other driver builds. Fixes the runlist-disable leak behind 'Immediate pteblit timed out' and silent no-render guests.
Full benchmark traces: Laguna S 2.1 INT4 vs DFlash speculative decoding on 4x RTX 3090 (vLLM 0.25, TP4, 96GB VRAM). 152 measurements, raw JSON + GPU telemetry.
PlantCare AI | High-performance Plant Disease Detection system optimized for RTX 3090. Identifies 38 diseases with 93% accuracy. Features a futuristic Glassmorphism UI, real-time diagnosis, and AI-driven treatment recommendations. 🌿🌱
GLM-5.3-Flash on 2× RTX 3090 + 128 GB RAM: full 256K context, up to 1,155 tok/s prefill · 47–50 tok/s decode. vLLM + ExLlamaV3 (EXL3 3.05 bpw) runtime with CPU expert offload, Docker, OpenAI-compatible API.
poolside Laguna-S-2.1 INT4 + DFlash speculative decoding on 4x RTX 3090: 200K context, 282 tok/s peak decode, gate-proven with a 190K-token prompt. Full levers menu + failure catalog.
Qwen3.8 Flash Next on 4x RTX 3090: full-GPU PLE, CUDA Graph, IQ4_NL, 262K context, multimodal / 四卡3090部署优化
Reproducibly turn a compatible Swift-Qwen3.8-27B W4A16 checkpoint into a syv-optimised single-GPU fast variant: target-calibrated draft vocabulary, GPTQ int4 lm_head/MTP, structural verification, RTX 3090 benchmarks.
To associate your repository with the rtx3090 topic, visit your repo's landing page and select "manage topics."