Jun 18
•
Articles
• 1 min read
Hey everyone. I’m an 18yo indie dev, and I’ve been experimenting with Spiking Neural Networks SNNs for language modeling. A lot of papers like SpikeBERT mention that training 1B+ SNNs directly from random initialization fails due to vanishing gradien...