Nils Durner's Blog Ahas, Breadcrumbs, Coding Epiphanies

Entropy-based "Shrek Sampling"

The community is porting the “Shrek Sampler” to different hardware (MLX) and Transformer architectures, just days after Entropix was first released. Nice visualization from the MLX port: 9.9 vs. 9.11. Pierre-Carl Langlais, Co-founder of Pleias.fr, posted his Colab notebook that runs entropix with Smollm-360M (the original release uses Llama 3.2 1B). He kindly answered my question if the entropix hype is real:

Definitely needs more testing but better stabilization of tiny models output while not sacrificing their creativity is certainly the way. Long term objective is efficient reasoner able to parse and reformulate external information with minimum compute cost.