What is Horovod?
Horovod is an open-source framework for distributed training of deep learning models across many GPUs or machines. Originally developed at Uber, it supports TensorFlow, Keras, PyTorch and MXNet and synchronises the gradients computed by the participating machines using ring-allreduce, without a central parameter server. The project is hosted by the LF AI & Data Foundation; the latest stable release, 0.28.1, dates from 2023 (as of October 2026).