LLM Attention Visualization
A visualization of the attention mechanism in LLMs.
An interactive visualization of the attention mechanism in a small large language model. Hover or tap any generated token to see which earlier tokens influenced it, with opacity scaled by attention weight across all heads and layers. Example prompts show how LLMs copy exact details like addresses and dates from their input. First shared by its maker on Show HN: https://news.ycombinator.com/item?id=49613068
✨ The overview, FAQ and details below were drafted by AI from the product's own website and haven't been reviewed by the maker yet. Are you the maker? Claim this listing to review and edit them.
Overview
Who is it for?
Developers, students and anyone curious about how transformer-based language models work.
Problem
How LLMs attend to earlier tokens is hard to picture from papers and equations alone.
Solution
A browser demo that highlights the past tokens each generated token drew from, using example prompts like an office-move summary.
What makes it unique
It reduces attention to one value per past token so the patterns stay readable, and explains the simplifications it makes.
LLM Attention Visualization FAQ
What does the visualization show?
For each generated token, how strongly earlier tokens influenced it, computed from attention weights scaled by value-vector magnitude and summed across heads and layers.
Is it exact?
No. The author notes it is highly simplified, so 'affected' is an approximation.
Do I need to install anything?
No. It runs in the browser with JavaScript enabled.
Details
- Pricing
- Free
- Categories
- Research & DataLLMs & Infrastructure
- Built for
- DevelopersStudentsResearchers
- Platforms
- Web
Hunted by AI Launch Team
Discussion (0)
Sign in to join the discussion.