Need to ask Martin for more details (so maybe work on this later), but here are some things it might involve:

• input box where user can type in a sentence
  • this may not be necessary...
• add user's sentence to the query/key plots (right side)
• show attention plot for user's sentence (left side)
  • right now, code for these plots is here: att.html
  • original bertviz repo: https://github.com/jessevig/bertviz
• allow user to click on lines in the attention plot, two types of interaction:
  • emphasize query/key pair (aka. want to see where this pair has high attention)
  • de-emphasize query/key pair (aka. we don't want this pair to have high attention)
• then, update matrix view to show plots with the corresponding features selected by the user (right side)

example attn plot for reference:

another potential option besides clicking in matrix view or using the dropdown menus

Include table with all sentences in visualization that users can interact with. E.g.,

• allow clicking on tokens from table to easily visualize sentences (could be especially helpful in matrix view)
• show distribution of words in sentence (could highlight ones w/ repeated words for induction heads)

potentially a helpful resource: https://2x.antdv.com/components/table

Left sidebar:

• when user clicks on a point (right side), show attention plot for sentence on left sidebar:
  • right now, code for these plots is here: att.html
  • original bertviz repo: https://github.com/jessevig/bertviz
• highlight token corresponding to point in attention plot (see below)
• allow user to toggle lines off and on in attention plot to more clearly observe attn patterns
  • currently, i do this by letting people click on the token text in the attention plot
• renormalize line weights after toggling
• label q + k columns
• potentially also add more complex features (e.g., input box) like in matrix view? will ask martin...

current attn view for reference:

Main panel:

• will clarify with martin, but may also still want to highlight points in sentence on main plot when the user clicks on a token (see above)
  • update: yes we should still do this but don't make points yellow, keep query/key color schemes just outline in different color maybe? if we use highlight token method like with search, seems to be a bit slow at the moment...

E.g., filter by

• norm
• position
• token length?
• type (e.g., just queries or keys)

(connected to idea of #11)

• create method to read in attention data on backend (from another json file)
• test out how long computing on fly would take
• ultimately, connect to attention plots in single(#7) /matrix view(#4)

Currently, the point cloud data is computed upon two listeners, which is very inefficient. I did not implement memory management so sometimes the browser crashes after several runs. Ideally, the point cloud data should be computed once upon loading the data.

• large query/key scatterplot view corresponding to single layer + head
• colors:
  • one main color scheme for queries (e.g., greens) and another for keys (e.g., pinks)
  • have different options for user to color points by within query/key groups, e.g.,
    • normalized position (the default right now)
    • vector norm
    • martin suggested something like coloring by every kth token (e.g., for n = 5, all tokens with pos % 5 == 0 are one color, all tokens with post % 5 == 1 are another, etc.)
    • maybe others? so we want it to be easy to add coloring options later on.
  • show legend for each plot coloring too
• hover view (probably will be similar to what i have now-- see below):
  
  show:
  • token value/string
  • token type (query/key)
  • position in sentence (also requires total sentence length)
  • norm
  • the full sentence it came from (and highlight current token)

Create a "projection of projections" (similar to fig. 6 in What Does Bert Look At?) to help users discover attention heads that may have similar attention patterns. Could be an alternate global view of attention (more clusters than matrix).

• add 3D coords
• have option to switch b/t 2D and 3D plots

Could help highlight some of the outlier points in scatterplot

The goal is to visualize the scatter matrix (12 * 12 * 10k points).

The code is in front/src/components/MatrixView.

Tried two packages:

1. d3.js, results below
   

Cons: slow; take about 15s to load all visualizations on my machine;

2. https://github.com/flekschas/regl-scatterplot
   Failed. Did not debug yet. I think it has something to do with the package. The package seems not to be good for creating 12*12 canvases simultaneously. I can imagine that the package would be good for showing one scatterplot.

To-dos:

• [] explore other packages (using canvas/WebGL preferred).

• add destroy data method or something?

Let users search for tokens in single view & highlight results: e.g.,

E.g.,

• unnormalized position + sentence length (e.g., 5 out of 33)
• query/vector norm
• full sentence
• dot product?

• rerun code to generate umap/tsne coordinates for GPT
  • 2d
  • 3d
• recompute other info (e.g., attention)
• add to attention-viz

Show layer/head info on hover (to help with knowing where you're clicking, especially in 3D view)

• When matrix/single view is loading, show message/indicator so user knows
• May want to stylize better later :)

• Have a button/option to show or hide token labels on plot, e.g.:
  
• make sure labels update when user zooms in and out (e.g., at every zoom level, show only enough so there's no overlap)
  • check out existing libraries like this d3 option

Not sure if we still need a clustering option too (that just shows labels inside cluster or something?)... I'll ask martin too haha

• this is lower priority (get the show/hide feature working first)

Another possibility if we want to include another type of transformer in the visualization tool

E.g., norm determines dot size

E.g., something like this dropdown menu:

check to make sure color palettes are color blind friendly (esp. categorical position view)

• lower priority, but make sure text scales properly when user zooms in on matrix :) (i think points look fine though?)

E.g., show correlation b/t q-k distance + dot product in single view (and as part of tooltip in matrix view?)

Alternatively (or in addition): show cosine similarity scores, difference between norms, etc.?

We will migrate the program to a web app for on-the-fly interactivity (#1).

The src for the web app is in /web, which consists of:

• /back: in python3, w. Flask for communicating between the front end

  • [] Dump data into a local file; in dataService.py, add functions for reading raw data and sending data to the front end

• /front: in typescript, w. Vue (framework) + Vuex (state management).

  • dataService.ts: for getting/posting data from/to the backend API
  • /store/index.ts: for state management, you might want to check Vuex
  • /components/*: modulized Vue components, you might want to check Vue
  • /assets: for static assets such as logos
  • /utils: helper function and types definition (like a heading file in C++).

Not sure if it's just my computer, but sometimes zooming in and out seems a bit lagging... maybe because I have too many watcher methods?

• fix position of camera (right now, default view a little awkward for both matrix/single mode)
• currently, a little hard to click precisely on plots in matrix view (sometimes 1 attention head off)
• change placement of layer/head labels too?

• add some sort of slider/dropdown menu/etc. in case people want to change the size of the dots in the scatterplots?

If things are working well and not too slow, we can try reloading the original larger dataset into the web app. Martin wanted to be able to get more results when searching keywords :).

Probably would be good to add some sort of progress bar/loading icon when interface is updating (sometimes it takes a few seconds...)

Highlight clicked on token, then show remaining queries + keys (instead of currently, it just shows opposite of clicked on token)

Overlaps w/ #6 but martin wants us to experiment with different color schemes:

• have different options for user to color points by within query/key groups, e.g.,
  • normalized position (the default right now)
  • vector norm
  • just plain green vs. pink (for example) for query and key
  • martin suggested something like categorical coloring by every kth token (e.g., for n = 5, all tokens with pos % 5 == 0 are one color, all tokens with post % 5 == 1 are another, etc.), could use colors like "paired" palette here
  • maybe others? so we want it to be easy to add coloring options later on.
• show legend for each plot coloring too
• also experiment with other query/key colors (e.g., instead of green for query and pink for key, blue for query and orange for key? let user pick from some options?)

As in #7, clicking a point will "freeze" the view for a few seconds, which is not friendly.

Add aggregate attention visualizations for each head (e.g., try heatmap / Jesse Vig bipartite graphs)

• one resource: d3 heatmap

• compute aggregate matrices

• load data into tool

• create heatmaps

Note: right now, probably mostly for single view, but potentially could add to matrix view too maybe?

• have button/option to switch visualization type on both views
• changes should propagate between matrix and single view

Have option for users to hide points with low attention (e.g., hide tokens w/ attention < 0.5) in single view.

• could make this value adjustable by user?
• maybe could add to matrix view too?

sometimes transition is very choppy -- not sure if there is a way around it though?
also:

• loading symbol when going from single --> matrix? (seems to be somewhat slow in this direction for some reason)
• @wowjyu: on zoom, naturally go from matrix to single & vice versa (w/ smooth transition)

If it's possible to just zoom into a specific plot in the matrix (and do all operations there), maybe we don't need single view?

• and then we could also have a "lock" feature if users don't want to accidentally move around while they're looking at a specific plot
• maybe just select to zoom area

Might help to focus on a more specific NLP task/domain and reveal more semantic patterns :).

• q & a
• entailment
• math
• fill in the blank
• translation (would we need finetuned model for this?)

• When in matrix view, user should be able to click on a single plot (or label/overlay corresponding to plot) to switch to single view (if possible, make other points fade away for cool transition)
• When in single view, user should be able to click a button (or something similar) to be taken back to matrix view

• 2D/3D BERT
• 2D/3D GPT

• add reset button to matrix & single view that resets zoom/pan settings
• also reset other actions (e.g., user added sentences)? or have separate reset buttons for those?

Could plot value vectors too for each attention head... would definitely add to computational load though.

E.g., show position when coloring by position and norm when coloring by norm

Find and add a 2D colormap for representing the positional embeddings of image patches in VIT.

Alternatively add two separate coloring schemes: one represents the position of image tokens on the y-axis of the images (rows), and one represents the position of image tokens on the x-axis (columns).

Make text size more responsive -- right now a little hard to read when zoomed out:

Allow users to toggle between normal (light) mode and dark mode: e.g.,

Building off #7, would be cool to connect tokens with lines (e.g., weighted by attention values) to show their intersections and contribute novel visualization of attention.

Maybe only draw top 2-3 attention weights for each token

E.g., something like this:

just across the top/left side of matrix maybe?