view post Post 3,968 ๐ง Zero-FLOP MemoryCan a small transformer do more by remembering differently?39.5M parameters + discrete count tables + document cache.1.017 BPC on enwik8.No gradient updates needed to build the memory.Exploring a different way to scale language modeling: not just more parameters, but better memory.Paper & code follows ... if someones interested See translation 4 replies ยท ๐ 7 7 ๐ 4 4 ๐ง 3 3 + Reply
view post Post 313 BORIS-2 IS OUT! opencerebral/Boris-2It tops the AxiomicLabs Intelligence Index with a score of 34.15! See translation 12 replies ยท ๐ฅ 8 8 โค๏ธ 2 2 + Reply