token-visualization

🐼 Steer your LLMs and agents at the token level: A web tool for token visualization & control, model inspection, installation-free harness, and LLM & agent trajectory annotation.

Shows which words of an AI answer to double-check. Rust CLI.

Explore how Hugging Face tokenizers work across models like LLaMA, PHI-3, and StarCoder2. Includes examples for encoding, decoding, chat formatting, and token visualization. Ideal for understanding text preprocessing in LLMs.

Token Pie runs locally on each developer's machine, reads telemetry Copilot already writes, Nothing leaves the machine. No server, no account, no telemetry of our own.

Native Windows app for chatting with local GGUF models and visualizing token-driven neural activity through a live connectome.