The melody, how surprised a music generation model (MusicGen large) is by each note, and the average EEG of 14 to 21 listeners. All real data. The panels on the left share one time axis: the thin strip on top is the whole song (click it to jump), below are the zoomed score, the model curve and the EEG.
Click any note on the score to see it explained below. Keys: Space play/pause, ←→ step 0.05 s (Shift: 0.5 s), N/P next/previous surprising note. Rhythm and key are transcribed automatically from the recording, so no bar lines are drawn.
Average scalp potential 0.1 to 0.35 s after the note. Red rings = the 3 most positive electrodes, blue rings = the 3 most negative.
Positive and negative are relative. Each electrode is measured against the average of all 32 electrodes (average reference), so at any moment some places are positive and others negative, summing to zero. Red (positive) means higher than the scalp average, blue (negative) means lower.
Red and blue are usually the two ends of one source. The auditory cortex lies deep on both sides of the brain, above the ears. When a population of neurons there fires together it acts like a small battery: one end points to the top of the head (positive at fronto-central sites, red), the other to behind the ears and the back of the head (negative at TP9/TP10 and occipital sites, blue). So red does not mean "excited here" and blue does not mean "inhibited there". What matters is how strong the pair is: the stronger, the larger the brain's response to the note.
Why the delay. Sound reaches the auditory cortex within tens of milliseconds, but the large cortical responses come later: a first negative wave at about 0.1 s (N1) and a positive wave at 0.15 to 0.3 s (P2). The statistical "surprise effect" sits at 0.1 to 0.35 s, peaking near 0.16 s: the more surprising the note, the larger this positive wave. That is why the light red band is drawn after the note, not on it.
Units. Each listener's EEG is first converted to standard deviations per electrode and then averaged across listeners, so the unit is standard deviations, not microvolts. 0.3 means 0.3 standard deviations above that listener's usual fluctuation at that electrode.
Every note has a peak, so how does this show "surprise"? Any note evokes a response of the same shape (positive fronto-central, negative at the back), so the presence of a peak says nothing about surprise. What matters is how high the peak is: across hundreds of notes, are peaks systematically higher for more surprising notes (see the scatter plot below)? Beware one trap: the model's surprise curve also jumps at each note onset, and the EEG also moves after each note, so the two curves look "in rhythm" just because both follow the notes, not because of surprise. In the scatter plot each dot is one note, so this rhythm factor is removed.
Why single notes often do not look like the rule. A melody has a new note every 0.2 to 0.4 s, so responses to neighbouring notes overlap, and only 14 to 21 listeners are averaged, so single-note pictures are noisy. The real evidence is "all notes, all listeners" on the right: 29 listeners and all 107 songs, with a regression that separates the overlap between neighbouring notes.
Each dot is one note (songs on this page, first 1 s excluded). x: the model's surprise for the note (standardized within the song). y: average EEG at the white-ring electrodes 0.1 to 0.35 s after the note. Large colored dots are the current song, the ringed dot is the note being explained. The red line is the mean in 5 surprise bins (bars: standard error). Click any dot to jump to that note. Tick "Control: random model" to put the random model's "surprise" on x.
Nose up, left is the listener's left. White rings = the 11 electrodes (fronto-central) that respond to surprise statistically.
Top row: after this one note in this song (average of 14 to 21 listeners). Bottom row: across 29 listeners and all songs, the extra EEG per one standard deviation of surprise (regression estimate).
Average response at the white-ring electrodes, 29 listeners, all songs. Red: notes one standard deviation more surprising. Blue: one standard deviation less surprising. Shaded band: standard error of the red line.
Data: OpenNeuro ds005876 (song familiarity experiment, 32-channel EEG, 29 listeners), listening periods only. EEG 1 to 30 Hz, average reference, low-passed to 8 Hz for display. Notes transcribed automatically from the recordings. Surprise comes from MusicGen-large's frame-by-frame prediction of the real audio. This public page shows public domain songs only.