Show HN: Watch a neural net learn to play Snake

ppo.gradexp.xyz

84 points by c1b a day ago

In browser PPO training demo, made possible by tinygrad: TinyJit -> WebGPU kernels.

Requires WebGPU.

mavdol04 - 7 minutes ago

That's cool, i did exactly the same few years ago

simedw - 4 hours ago

Cool project!

I noticed that if you go from training to watch and then back, the training temporarily drop significantly in score.

bguberfain - 2 hours ago

It seems to be something related the moving average calculation. So it is just a glitch on the chart.

snats - an hour ago

did a pretty similar thing last month for the text rendering library last month.

trained and made a viz for the model and then made it displace text.

should probably do a proper write-up:https://x.com/i/status/2038367016969724259

ldoughty - 3 hours ago

Really cool! But right as it was nearing 4,000, it seems to have corrupted itself and no longer got any scores above 0. Not sure if that's a code bug or a neural net issue.

avg500 -4.6 last 500 episodes

peak 3959.3 best window

roll/s 20.68 20-step avg

progress 4388 562749 episodes

c1b - an hour ago

Yes it just collapses eventually — never stabilizes. The training process is flawed, I suspect it has to do with the fact that some weights blow up over time, you can see in “weights” tab.
But at around 4K avg score you should see it solve the env almost every time.
Just a demo :) optimized for speed over stability.
Reward structure: Step: -1 Dot: +100 Win: +1000 so ~4k is max theoretical score on 6x6; the model will eventually break if you keep training long enough.
- an hour ago

[deleted]
- an hour ago

[deleted]
r3trohack3r - 2 hours ago

I think I noticed it reach “end game.” The snake reaches a point where, if it gets any longer, it is out of squares and hits its own tail. So it finds the route through the squares that it can infinitely loop, never eats the ball, and score starts dropping and goes negative.

- 4 hours ago

[deleted]

beardsciences - 4 hours ago

My average eventually made it to about 3900, and then stagnated between 3600-3900. I'm curious if this is universal behavior or not. I'm up to about 5k steps.

neduma - 4 hours ago

More details and implementation notes please?

cshimmin - 12 minutes ago

[dead]

LowLevelKernel - 2 hours ago

Link to repo?

bozhark - an hour ago

Crashed

jmclnx - 3 hours ago

> WebGPU not available in this browser

Looks like this is for Linux and Windows, on NetBSD I get this issue :(

NoboruWataya - 2 hours ago

I got this in Firefox on Linux, just had to enable WebGPU in about:config (`dom.webgpu.enabled` = true).
- jmclnx - an hour ago
  
  Did not know that existed, I enabled it but no luck. Must be a NetBSD thing based upon this new message:
  > WebGPU is not yet available in Release or late Beta builds.
redshiftza - 2 hours ago

If you are using brave (which i assume also applies to chrome) , there is a menu at brave://flags , you can enable unsafe web GPU from there

th1nhng0 - 3 hours ago

cool project

austinthetaco - 3 hours ago

my training on a 10x10 just randomly broke. i got to like 3600 then the graph went flat, the viewer on the left just showed it constantly restarting the game, and the scores in the negative. my average is now -10.