用C/C#实现的、受SNN（脉冲神经网络）启发的大脑式生成式AI，并且不依赖外部AI库，这有前景吗？
1 分•作者: adinhitlore•4 个月前
在过去的几个月里，我尝试了许多潜在的 Transformer 替代方案，其中之一就是所谓的脉冲神经网络（SNN）。对于不熟悉的人来说，这是一种旨在使人工智能尽可能接近人脑的方法，换句话说，虽然 Transformer 或 RNN、LSTM 是结果导向的（不寻求 1:1 模拟人类）——SNN 实际上试图精确地模仿人类的学习方式。使用它们进行 NLP 的想法并非全新（谷歌的 SpikeGPT），但据我所知，还没有人尝试过以下几点：
* 在 SNN 中每隔 n 个 token 之后加入情景式长期记忆（我自己的想法...）；
* 在 C 和 C# 中实现，无需 torch/tensorflow（SpikeGPT 使用 python 和 torch）；
* 几种类型的“注意力”和训练模式以及记忆模式；
* 无需反向传播的训练/学习；
* 对 CPU 友好，因为虽然它仍然有点慢（不幸的是），但至少不需要 GPU
以下是 C# Windows Forms 实现和 C/cygwin 端口的截图...以及关于该程序的 Claude Sonnet 4.6 和 Gemini Pro 3.1 的 2 张随机截图：
[https://imgur.com/a/SAQqKmm](https://imgur.com/a/SAQqKmm)
为什么从种子生成的文本仍然远未完美？两个原因：语料库非常小，并且 C# 的准确率 <100%。
然而，一个令人惊喜的好消息是：它似乎同时学习了语法和语义，再加上我提出的包含一种长期情景记忆的方式，一个长上下文可以扩展到微小的“ctx”窗口之外的数千个 token，而不会降低速度——这可能会使它成为一个实用的程序。生成速度也很快。
未来工作：
* BPE，现在它只是单词分词器...对代码不好；
* 我说“代码”了吗？它实际上可能对编码来说是一个彻底的失败...或者也许不是：完全未经测试；
* 该程序实际上有两个版本，另一个版本与此版本明显不同，并且有 C 甚至 F# 端口，但是 F# 根本无法工作...它总是产生完全的胡言乱语...主要错误。
* 从未在实际的神经形态 CPU 上测试过，只是普通的英特尔通用笔记本电脑；
* 应该可以进行 python 移植；
* 最后的大测试：大型文本语料库（兆字节）和超过 95% 的准确率 <- 终极测试。
查看原文
So over the past months I've experimented with many potential alternatives to transformers, one of which is the so called Spiking Neural Networks
which, to the ones unfamiliar with them, is an approach to AI which seeks to make AI as similar to the human brain as possible, in other words while transformers or RNN, LSTMs are result-driven (not seeking to simulate human 1:1) - SNNs actually try to mimic the way humans learn exactly. The idea to use them for NLP isn't entirely new (google SpikeGPT), however to the best of my knowledge no one yet has tried this:<p>* inclusion of episodic long term memory in SNN every-n-number-of-tokens behind (my own idea...);
* implementations in C and C# ports without torch/tensorflow (SpikeGPT is in python with torch);
* several types of 'attention' and training modes and memory modes;
* training/learning without backpropagation;
* CPU-friendly in the sense that while it's still kind of slow (unfortunately) at least GPU isn't mandatory<p>Here are the screenshots of both the c# windows forms implementation and the C/cygwin port...and 2 random screenshots of claude sonnet 4.6 and gemini pro 3.1 about the program:<p>https://imgur.com/a/SAQqKmm<p>why is the text generated from seed still far from perfect? 2 reasons: very small corpus and the c# has <100% accuracy.<p>However the big nice surprise: It seems like grammar and semantics are both learned, this coupled with my idea to include a way for long term episodic memory a long context outside the tiny 'ctx' window can be extended easily to thousands of tokens behind without decrease in speed - could make it a practical program. Generation is also very fast.<p>future work:<p>* BPE, right now it's just words tokenizer...not good for code;
* did i say "code"? It may actually be a total failure for coding...or maybe not: completely untested;
* The program actually has 2 versions, the other one noticeably deviates from this one and it has c and even f# port, however the f# just doesn't work...it always produces complete
gibberish...major bug.
* never tested on actual neuromorphic CPU, just goood ol' intel universal laptop ones;
* python port should be possible;
* finally the big test: large text corpus (megabytes) and accuracy over 95% <- ultimate test.