Kaido Orav and Byron Knoll's fx2-cmix Wins 7950€ Hutter Prize Award!

[James Bowery]

Kaido Orav and Byron Knoll have improved 1.59% on the Hutter Prize for Lossless Compression of Human Knowledge with their "fx2-cmix" entry.

Because the Hutter Prize restricts contestants to a single general purpose processor and it uses the most general loss function, the required algorithmic advances are generally applicable regardless of the industry's "Hardware Lottery" or loss function compromises.  In this respect it provides a unique and low risk incentive for scientific advancement in machine learning.

Some of fx2-cmix's algorithmic advances over the prior Hutter Prize winning algorithm:

Mixer and Predictor: Mixers now skip weight updates when errors are below a certain threshold, which enhances processing speed.

Single Pass Wikipedia Transform: This update reduces the time and disk usage needed for processing large datasets like Wikipedia by simplifying the transformation process from a previous multi-step approach to a single pass, thereby significantly speeding up preprocessing stages.

New Stemming and Context Methods: Utilizing Natural Language Processing techniques like new word types in stemming processes to create more compact and relevant word streams. This not only improves the quality of the training data but also enhances compression, reducing the storage requirements.

Efficient Article Ordering: By embedding entire articles in a large vector and using t-SNE to reduce them to a single dimension, the entire corpus can be rapidly reordered to further speed up training.

Detailed descriptions of the advances along with Jupyter notebooks and additional documents are available in the fx2-cmix README.