Backup of BitNet/1.58 (No. 1)

https://arxiv.org/abs/2402.17764

• https://arxiv.org/pdf/2402.17764.pdf

https://github.com/frodo821/BitNet-Transformers

https://twitter.com/BoufrawFrodo2/status/1763435835935047789 オリジナルのBitNetを1.58bの論文に従って3値にするように修正しました

• https://gigazine.net/news/20240229-microsoft-1bit-llm/

• https://topics.smt.docomo.ne.jp/article/wirelesswire/business/wirelesswire-20240286094?

https://twitter.com/andrew_n_carr/status/1770487200234213758?s=12

1.58 bit code is out (in an appendix) https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf

https://twitter.com/teortaxesTex/status/1773861506674741570

It seems that results of that Microsoft paper about ternary LLMs can be replicated after all – for 3B@100B at least. https://huggingface.co/1bitLLM/bitnet_b1_58-3B