Mixture of Depth is Vibe
Thought and code about mixture of depth
Bitnet
Experiments with Bitnet 1.5 (~ngmi~)
The blog explores implementing Bitnet 1.5, highlighting its efficiency gains and challenges in quantized neural network training.
Thought and code about mixture of depth
The blog explores implementing Bitnet 1.5, highlighting its efficiency gains and challenges in quantized neural network training.