Thread by @CauseMean

Excited to share the details of our work at @DeepMind on using reinforcement learning to help large-scale commercial cooling systems save energy and run more efficiently: arxiv.org/abs/2211.07357.

Here’s what we found 🧵

First, #RL can substantially outperform industry standard controllers.

📉 We reduced energy use by 9% and 13% at two separate sites, while satisfying all of the constraints at a level comparable with the baseline policy.

🔧 We built on the existing RL controller used for cooling Google’s data centers and extended it to a more challenging setup.

There’s a higher dimensional action space (jointly controlling multiple chillers), more complex constraints, and less data standardization.

🔴 We also had to tackle issues that tend to arise when using RL on real physical systems.

They included learning from a relatively small dataset with limited exploration, non-stationary dynamics and observations, and multiple time scales for the dynamics.

🤔 So how did we deal with these?

The solutions required a strong core RL algorithm together with domain-specific additions such as sensitivity analysis, model unit tests, feature engineering, or heuristic action pruning (lots more detail in the paper).

🤝 This project has been a joint effort with great people from @DeepMind, @Google and @TraneCommercial, and I am grateful for the chance to work with them.

We will present this research at the @rl4reallife workshop at #NeurIPS2022 on December 3rd - looking forward to it!