Nimbus + AI Post Mortem

[Jared Lockhart]

Hi all,

In 2025, a series of experiments was run to measure the effect of using local AI models in Firefox Desktop.  These experiments uncovered a subtle bug in the Remote Settings and Nimbus clients that prevented the experiments from completing successfully.  After investigating, a root cause was identified and a fix applied, which has allowed these AI experiments to complete successfully.  This resolves not only these specific experiments, but improves Nimbus' ability to support experimentation with local AI in Firefox Desktop more generally.  I have attached a more detailed writeup of the detection, analysis, and resolution of this bug.  Please feel free to contact me with any questions or followup.  

Thank you!

Jared Lockhart

Engineering Lead, Nimbus

 Nimbus + AI Post Mortem

[Bored Videos]

Can you give everyone read access?