Simulation · v0.2 · the closed loop

Can a dish of neurons learn Pong?

Two copies of the same culture play Pong through their electrodes and face the same serves. The only difference is what they are told after each hit or miss. The first gets the feedback from the DishBrain experiment: a predictable pulse after a hit, four seconds of noise after a miss. The control gets the same pulses at random. If the feedback teaches the culture anything, its rallies should get longer than the control's.

DishBrain feedback

Predictable pulse after a hit, noise after a miss.

Playing
Rallies
0
Hits per rally
–
Last 5 min
–
Aces
–
Mean synapse
–

Control

Playing
Rallies
0
Hits per rally
–
Last 5 min
–
Aces
–
Mean synapse
–

Hits per rally

mean over the last 2 minutes of each culture's play

DishBrain feedback Control, random feedback Simulated time 0:00

Seed – · both cultures grow from it, so they start identical.

sensory motor up motor down

How the loop works

  • Sensing. The two middle columns of electrodes are sensory. The row that matches the ball's height is pulsed, from 4 Hz when the ball is far away to 40 Hz when it reaches the paddle.
  • Acting. The outer columns are motor regions. More spikes in the top half move the paddle up, more in the bottom half move it down.
  • Feedback. After a hit, all sensory electrodes pulse at 100 Hz for 100 ms. After a miss, random electrodes fire at random times for 4 s while the ball waits to be served.
  • Plasticity. Synapses change with spike timing (STDP): input that arrives just before a neuron fires gets stronger, input that arrives just after gets weaker.

What we found so far

We ran 16 culture pairs of 2 048 neurons for 20 simulated minutes per condition and compared the first five minutes with the last five. With DishBrain feedback, rallies grew by 0.05 ± 0.05 hits; with random feedback they changed by −0.02 ± 0.04, and without feedback by 0.00 ± 0.05. That is no reliable learning. With 4 096 neurons, or with stronger plasticity, the picture was the same.

One likely reason: STDP links spikes about 20 ms apart, but the feedback arrives seconds after the motor activity that earned it. Something has to bridge that gap, and finding out what is the next experiment. Run your own pair here and see whether your seed behaves differently.

Background

DishBrain wired real cortical neurons to Pong and reported that rallies improved within minutes of feedback. The interpretation is debated. Hotaru is a simplified model of that protocol, not of real neurons, and makes no claim about either.

Kagan et al. (2022), In vitro neurons learn and exhibit sentience when embodied in a simulated game-world, Neuron. Izhikevich (2003). Tsodyks & Markram (1997). Sinapayen, Masumori & Ikegami (2017), Learning by stimulation avoidance. Reproduce the numbers with cargo run --release --example experiment.