Distributed Load Balancing: A New Framework and Improved Guarantees

Authors Sara Ahmadian, Allen Liu, Binghui Peng, Morteza Zadimoghaddam

Thumbnail PDF


  • Filesize: 0.54 MB
  • 20 pages

Document Identifiers

Author Details

Sara Ahmadian
  • Google Research, New York, NY, USA
Allen Liu
  • MIT, Cambridge, MA, USA
Binghui Peng
  • Columbia University, New York, NY, USA
Morteza Zadimoghaddam
  • Google Research, Cambridge, MA, USA

Cite AsGet BibTex

Sara Ahmadian, Allen Liu, Binghui Peng, and Morteza Zadimoghaddam. Distributed Load Balancing: A New Framework and Improved Guarantees. In 12th Innovations in Theoretical Computer Science Conference (ITCS 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 185, pp. 79:1-79:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)


Inspired by applications on search engines and web servers, we consider a load balancing problem with a general convex objective function. In this problem, we are given a bipartite graph on a set of sources S and a set of workers W and the goal is to distribute the load from each source among its neighboring workers such that the total load of workers are as balanced as possible. We present a new distributed algorithm that works with any symmetric non-decreasing convex function for evaluating the balancedness of the workers' load. Our algorithm computes a nearly optimal allocation of loads in O(log n log² d/ε³) rounds where n is the number of nodes, d is the maximum degree, and ε is the desired precision. If the objective is to minimize the maximum load, we modify the algorithm to obtain a nearly optimal solution in O(log n log d/ε²) rounds. This improves a line of algorithms that require a polynomial number of rounds in n and d and appear to encounter a fundamental barrier that prevents them from obtaining poly-logarithmic runtime [Berenbrink et al., 2005; Berenbrink et al., 2009; Subramanian and Scherson, 1994; Rabani et al., 1998]. In our paper, we introduce a novel primal-dual approach with multiplicative weight updates that allows us to circumvent this barrier. Our algorithm is inspired by [Agrawal et al., 2018] and other distributed algorithms for optimizing linear objectives but introduces several new twists to deal with general convex objectives.

Subject Classification

ACM Subject Classification
  • Theory of computation → Distributed algorithms
  • Load balancing
  • Distributed algorithms


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Shipra Agrawal, Vahab Mirrokni, and Morteza Zadimoghaddam. Proportional allocation: Simple, distributed, and diverse matching with high entropy. In International Conference on Machine Learning, pages 99-108, 2018. Google Scholar
  2. Zeyuan Allen-Zhu and Lorenzo Orecchia. Using optimization to break the epsilon barrier: A faster and simpler width-independent algorithm for solving positive linear programs in parallel. In Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms, pages 1439-1456. SIAM, 2014. Google Scholar
  3. Zeyuan Allen-Zhu and Lorenzo Orecchia. Nearly linear-time packing and covering lp solvers. Mathematical Programming, 175(1-2):307-353, 2019. Google Scholar
  4. Sepehr Assadi, Aaron Bernstein, and Zachary Langley. Improved bounds for distributed load balancing. arXiv preprint, 2020. URL: http://arxiv.org/abs/2008.04148.
  5. Baruch Awerbuch and Rohit Khandekar. Stateless distributed gradient descent for positive linear programs. SIAM Journal on Computing, 38(6):2468-2486, 2009. Google Scholar
  6. Petra Berenbrink, Tom Friedetzky, and Zengjian Hu. A new analytical method for parallel, diffusion-type load balancing. Journal of Parallel and Distributed Computing, 69(1):54-61, 2009. Google Scholar
  7. Petra Berenbrink, Tom Friedetzky, and Russell Martin. Dynamic diffusion load balancing. In International Colloquium on Automata, Languages, and Programming, pages 1386-1398. Springer, 2005. Google Scholar
  8. Andrzej Czygrinow, Michal Hanćkowiak, Edyta Szymańska, and Wojciech Wawrzyniak. Distributed 2-approximation algorithm for the semi-matching problem. In International Symposium on Distributed Computing, pages 210-222. Springer, 2012. Google Scholar
  9. Magnús M Halldórsson, Sven Köhler, Boaz Patt-Shamir, and Dror Rawitz. Distributed backup placement in networks. Distributed Computing, 31(2):83-98, 2018. Google Scholar
  10. Michael Luby and Noam Nisan. A parallel approximation algorithm for positive linear programming. In Proceedings of the twenty-fifth annual ACM symposium on Theory of computing, pages 448-457, 1993. Google Scholar
  11. Michael W Mahoney, Satish Rao, Di Wang, and Peng Zhang. Approximating the solution to mixed packing and covering lps in parallel o(epsilon^-3) time. In 43rd International Colloquium on Automata, Languages, and Programming (ICALP 2016). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016. Google Scholar
  12. Albert W Marshall and Frank Proschan. An inequality for convex functions involving majorization. Technical report, BOEING SCIENTIFIC RESEARCH LABS SEATTLE WASH, 1964. Google Scholar
  13. Yuval Rabani, Alistair Sinclair, and Rolf Wanka. Local divergence of markov chains and the analysis of iterative load-balancing schemes. In Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No. 98CB36280), pages 694-703. IEEE, 1998. Google Scholar
  14. Google Search Statistics. Internet Live Stats, 2020. URL: https://www.internetlivestats.com/google-search-statistics/.
  15. Raghu Subramanian and Isaac D Scherson. An analysis of diffusive load-balancing. In Proceedings of the sixth annual ACM symposium on Parallel algorithms and architectures, pages 220-225, 1994. Google Scholar
  16. Di Wang. Fast Approximation Algorithms for Positive Linear Programs. PhD thesis, UC Berkeley, 2017. Google Scholar
  17. Di Wang, Satish Rao, and Michael W Mahoney. Unified acceleration method for packing and covering problems via diameter reduction. In 43rd International Colloquium on Automata, Languages, and Programming (ICALP 2016). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016. Google Scholar
  18. Neal E Young. Sequential and parallel algorithms for mixed packing and covering. In Proceedings 42nd IEEE symposium on foundations of computer science, pages 538-546. IEEE, 2001. Google Scholar