深層ニューラルネットワーク不要のLLM：新アーキテクチャの提案と事例研究 LLMs Without Deep Neural Networks: New Architecture, Benefits and Case Study

Large language models have, almost without exception, been built on deep neural networks. From the original transformer to the flood of variants that followed, DNN-based architectures have defined what it means to build a capable LLM. A new preprint, arXiv:2605.30385, challenges that assumption directly, proposing an alternative architecture that dispenses with deep neural networks altogether and offering early validation of the approach in an LLM context.

The author appears to have been developing a DNN alternative for some time and has now turned to LLMs as a proving ground. While the abstract reveals only a high-level framing, the paper claims to demonstrate that this alternative architecture can deliver meaningful results at the language modeling scale. A case study is included to ground the argument in concrete evidence, though as with any preprint, independent replication will be essential before strong conclusions can be drawn.

The idea of replacing DNNs in machine learning is not new. Kernel methods, random feature models, tensor networks, and symbolic or probabilistic approaches have all been explored as alternatives, each motivated in part by well-known criticisms of deep learning: opacity, enormous parameter counts, and steep computational costs. The challenge has always been whether these alternatives can scale. LLMs represent perhaps the hardest scaling test imaginable, which makes this paper's ambition notable regardless of how the results ultimately hold up.

The broader research landscape offers useful context. Architectures like Mamba (based on state space models) and RWKV (a recurrent approach) have already demonstrated that transformers are not the only viable path for sequence modeling at scale. These still rely on learned, layered nonlinear transformations — they are DNNs in spirit even if not in the canonical sense. A proposal that removes DNNs entirely occupies a more radical position on the spectrum and will likely attract both skepticism and curiosity from the research community.

There are practical motivations for this line of inquiry beyond academic novelty. The energy and infrastructure costs of training and serving large neural models have become genuine concerns for industry and policymakers alike. If an alternative architecture could match DNN performance with lower compute requirements or better interpretability, the implications would extend well beyond academic interest.

For now, this work sits at the preprint stage, and the claims should be read accordingly. The machine learning community has seen bold architectural proposals come and go, and the path from promising paper to widely adopted method is long. Still, as a prompt for rethinking foundational assumptions about how language models must be built, the paper is a worthwhile contribution to an increasingly important conversation.