Skip to content
Sign inLaunch

LLM Attention Visualization

A visualization of the attention mechanism in LLMs.

An interactive visualization of the attention mechanism in a small large language model. Hover or tap any generated token to see which earlier tokens influenced it, with opacity scaled by attention weight across all heads and layers. Example prompts show how LLMs copy exact details like addresses and dates from their input. First shared by its maker on Show HN: https://news.ycombinator.com/item?id=49613068

✨ The overview, FAQ and details below were drafted by AI from the product's own website and haven't been reviewed by the maker yet. Are you the maker? Claim this listing to review and edit them.

Overview

Who is it for?

Developers, students and anyone curious about how transformer-based language models work.

Problem

How LLMs attend to earlier tokens is hard to picture from papers and equations alone.

Solution

A browser demo that highlights the past tokens each generated token drew from, using example prompts like an office-move summary.

What makes it unique

It reduces attention to one value per past token so the patterns stay readable, and explains the simplifications it makes.

LLM Attention Visualization FAQ

What does the visualization show?

For each generated token, how strongly earlier tokens influenced it, computed from attention weights scaled by value-vector magnitude and summed across heads and layers.

Is it exact?

No. The author notes it is highly simplified, so 'affected' is an approximation.

Do I need to install anything?

No. It runs in the browser with JavaScript enabled.

Details

Pricing
Free
Built for
DevelopersStudentsResearchers
Platforms
Web

Hunted by AI Launch Team

Discussion (0)

Sign in to join the discussion.