Document Open Access Logo

In-Kernel Aggregation and Broadcast Acceleration for Distributed Communication

Authors Jianchang Su , Yifan Zhang , Wei Zhang

PDF

File

Thumbnail PDF
PDF
OASIcs.NINeS.2026.13.pdf
  • Filesize: 1.26 MB
  • 23 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Networks → Cloud computing
  • Networks → In-network processing
  • Computer systems organization → Distributed architectures
Keywords
  • eBPF
  • distributed communication
  • broadcast
  • aggregation
  • in-kernel processing
  • XDP

Metrics

Abstract

Broadcasting and aggregation dominate the communication overhead in distributed systems, from machine learning training to data analytics. Current acceleration approaches require specialized hardware (RDMA) or dedicated resources (DPDK), limiting their deployment in commodity clouds. However, we present a counter-intuitive alternative: rather than bypassing the kernel, we move operations into it using eBPF. While this imposes severe constraints including no floating-point, limited memory, and stateless execution, we show these restrictions paradoxically drive innovative protocol designs that yield unexpected benefits. 
We introduce AggBox, which implements broadcast and aggregation operations entirely within eBPF’s constrained environment. Our key innovations include stateless group acknowledgments for reliability, edge quantization for floating-point aggregation using only integer arithmetic, and tail-call chains that create virtual memory beyond eBPF’s 512-byte stack limit. These designs emerge from and exploit the constraints rather than fighting them. AggBox achieves remarkable performance on commodity hardware: 84.5% reduction in broadcast latency, 43× speedup for MapReduce workloads, and 56.1% faster ML gradient aggregation, all without specialized NICs or dedicated cores. Beyond performance, our work demonstrates that constrained environments can drive fundamental innovation in protocol design, offering insights for future resource-limited and verified systems.

Cite As Get BibTex

Jianchang Su, Yifan Zhang, and Wei Zhang. In-Kernel Aggregation and Broadcast Acceleration for Distributed Communication. In 1st New Ideas in Networked Systems (NINeS 2026). Open Access Series in Informatics (OASIcs), Volume 139, pp. 13:1-13:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/OASIcs.NINeS.2026.13

Author Details

Jianchang Su
  • University of Connecticut, Storrs, CT, USA
Yifan Zhang
  • University of Connecticut, Storrs, CT, USA
Wei Zhang
  • University of Connecticut, Storrs, CT, USA

Funding

This work is supported in part by the National Science Foundation under Award #2530909.

Acknowledgements

We thank our shepherd Nate Foster and the anonymous reviewers for their helpful comments. We thank Chameleon Cloud [Kate Keahey et al., 2020] for the development and evaluation infrastructure.

Supplementary Materials

References

  1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org. URL: https://www.tensorflow.org/.
  2. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations, 2020. URL: https://arxiv.org/abs/2006.11477.
  3. K. A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for federated learning on user-held data. In NIPS Workshop on Private Multi-Party Machine Learning, 2016. URL: https://arxiv.org/abs/1611.04482.
  4. Xiang Chen, Hongyan Liu, Qun Huang, Peiqiao Wang, Dong Zhang, Haifeng Zhou, and Chunming Wu. Speed: Resource-efficient and high-performance deployment for data plane programs. In 2020 IEEE 28th International Conference on Network Protocols (ICNP), pages 1-12, 2020. URL: https://doi.org/10.1109/ICNP49622.2020.9259384.
  5. Xiang Chen, Hongyan Liu, Qingjiang Xiao, Kaiwei Guo, Tingxin Sun, Xiang Ling, Xuan Liu, Qun Huang, Dong Zhang, Haifeng Zhou, Fan Zhang, and Chunming Wu. Toward low-overhead inter-switch coordination in network-wide data plane program deployment. In 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS), pages 370-380, 2022. URL: https://doi.org/10.1109/ICDCS54860.2022.00043.
  6. Hong-Ning Dai, Raymond Chi-Wing Wong, Hao Wang, Zibin Zheng, and Athanasios V Vasilakos. Big data analytics for large-scale wireless networks: Challenges and opportunities. ACM Computing Surveys (CSUR), 52(5):1-36, 2019. URL: https://doi.org/10.1145/3337065.
  7. Nadeen Gebara, Manya Ghobadi, and Paolo Costa. In-network aggregation for shared machine learning clusters. In A. Smola, A. Dimakis, and I. Stoica, editors, Proceedings of Machine Learning and Systems, volume 3, pages 829-844, 2021. Google Scholar
  8. Yoann Ghigoff, Julien Sopena, Kahina Lazri, Antoine Blin, and Gilles Muller. BMC: Accelerating memcached using safe in-kernel caching and pre-stack processing. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21), pages 487-501. USENIX Association, April 2021. URL: https://www.usenix.org/conference/nsdi21/presentation/ghigoff.
  9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. URL: https://arxiv.org/abs/1512.03385.
  10. Yongchao He, Wenfei Wu, Yanfang Le, Ming Liu, and ChonLam Lao. A generic service to provide in-network aggregation for key-value streams. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, ASPLOS 2023, pages 33-47, New York, NY, USA, 2023. Association for Computing Machinery. URL: https://doi.org/10.1145/3575693.3575708.
  11. Yimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi, Yong Cui, and Chuanxiong Guo. A unified architecture for accelerating distributed DNN training in heterogeneous GPU/CPU clusters. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pages 463-479, 2020. URL: https://www.usenix.org/conference/osdi20/presentation/jiang.
  12. Kostis Kaffes, Jack Tigar Humphries, David Mazières, and Christos Kozyrakis. Syrup: User-defined scheduling across the stack. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, SOSP '21, pages 605-620, New York, NY, USA, 2021. Association for Computing Machinery. URL: https://doi.org/10.1145/3477132.3483548.
  13. Kate Keahey, Jason Anderson, Zhuo Zhen, Pierre Riteau, Paul Ruth, Dan Stanzione, Mert Cevik, Jacob Colleran, Haryadi S. Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha, and Joe Stubbs. Lessons learned from the chameleon testbed. In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20). USENIX Association, July 2020. URL: https://www.usenix.org/conference/atc20/presentation/keahey.
  14. Guangchen Lan, Dong-Jun Han, Abolfazl Hashemi, Vaneet Aggarwal, and Christopher Brinton. Asynchronous federated reinforcement learning with policy gradient updates: Algorithm design and convergence analysis. In The Thirteenth International Conference on Learning Representations (ICLR), 2025. Google Scholar
  15. Guangchen Lan, Huseyin A Inan, Sahar Abdelnabi, Janardhan Kulkarni, Lukas Wutschitz, Reza Shokri, Christopher G Brinton, and Robert Sim. Contextual integrity in LLMs via reasoning and reinforcement learning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2025. Google Scholar
  16. Guangchen Lan, Sipeng Zhang, Tianle Wang, Yuwei Zhang, Daoan Zhang, Xinpeng Wei, Xiaoman Pan, Hongming Zhang, Dong-Jun Han, and Christopher G Brinton. Mappo: Maximum a posteriori preference optimization with prior knowledge. arXiv preprint arXiv:2507.21183, 2025. URL: https://doi.org/10.48550/arXiv.2507.21183.
  17. ChonLam Lao, Yanfang Le, Kshiteej Mahajan, Yixi Chen, Wenfei Wu, Aditya Akella, and Michael Swift. ATP: In-network aggregation for multi-tenant learning. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21), pages 741-761. USENIX Association, April 2021. URL: https://www.usenix.org/conference/nsdi21/presentation/lao.
  18. Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. A review of applications in federated learning. Computers & Industrial Engineering, 149:106854, 2020. URL: https://doi.org/10.1016/J.CIE.2020.106854.
  19. Yiran Li, Kevin Gao, Xin Jin, and Wei Xu. Concerto: cooperative network-wide telemetry with controllable error rate. In Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys '20, pages 114-121, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3409963.3410499.
  20. Priyanka Mary Mammen. Federated learning: Opportunities and challenges. arXiv preprint arXiv:2101.05428, 2021. URL: https://arxiv.org/abs/2101.05428.
  21. Navneet Kumar Pandey, Kaiwen Zhang, Stéphane Weiss, Hans-Arno Jacobsen, and Roman Vitenberg. Distributed event aggregation for content-based publish/subscribe systems. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, DEBS '14, pages 95-106, New York, NY, USA, 2014. Association for Computing Machinery. URL: https://doi.org/10.1145/2611286.2611302.
  22. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library, 2019. URL: https://arxiv.org/abs/1912.01703.
  23. pcap(3PCAP). pcap(3PCAP) man page | TCPDUMP & LIBPCAP - tcpdump.org. https://www.tcpdump.org/manpages/pcap.3pcap.html. [Accessed 15-06-2024].
  24. Shixiong Qi, Leslie Monis, Ziteng Zeng, Ian-chin Wang, and K. K. Ramakrishnan. Spright: extracting the server from serverless computing! high-performance ebpf-based event-driven, shared-memory processing. In Proceedings of the ACM SIGCOMM 2022 Conference, SIGCOMM '22, pages 780-794, New York, NY, USA, 2022. Association for Computing Machinery. URL: https://doi.org/10.1145/3544216.3544259.
  25. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners, 2019. Google Scholar
  26. Amedeo Sapio, Marco Canini, Chen-Yu Ho, Jacob Nelson, Panos Kalnis, Changhoon Kim, Arvind Krishnamurthy, Masoud Moshref, Dan Ports, and Peter Richtarik. Scaling distributed machine learning with In-Network aggregation. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21), pages 785-808. USENIX Association, April 2021. URL: https://www.usenix.org/conference/nsdi21/presentation/sapio.
  27. Jianchang Su, Yifan Zhang, Linpu Huang, and Wei Zhang. Boad: Optimizing distributed communication with in-kernel broadcast and aggregation. In Proceedings of the ACM SIGCOMM 2024 Workshop on eBPF and Kernel Extensions, pages 51-57, 2024. Google Scholar
  28. Nik Sultana, John Sonchack, Hans Giesen, Isaac Pedisich, Zhaoyang Han, Nishanth Shyamkumar, Shivani Burad, André DeHon, and Boon Thau Loo. Flightplan: Dataplane disaggregation and placement for p4 programs. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21), pages 571-592. USENIX Association, April 2021. URL: https://www.usenix.org/conference/nsdi21/presentation/sultana.
  29. Zeyi Tao and Qun Li. eSGD: Communication efficient distributed deep learning on the edge. In USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18), Boston, MA, July 2018. USENIX Association. URL: https://www.usenix.org/conference/hotedge18/presentation/tao.
  30. Maarten Van Steen and Andrew S Tanenbaum. Distributed systems. Maarten van Steen Leiden, The Netherlands, 2017. Google Scholar
  31. Joost Verbraeken, Matthijs Wolting, Jonathan Katzy, Jeroen Kloppenburg, Tim Verbelen, and Jan S. Rellermeyer. A survey on distributed machine learning. ACM Comput. Surv., 53(2), March 2020. URL: https://doi.org/10.1145/3377454.
  32. Marcos AM Vieira, Matheus S Castanho, Racyus DG Pacífico, Elerson RS Santos, Eduardo PM Câmara Júnior, and Luiz FM Vieira. Fast packet processing with ebpf and xdp: Concepts, code, challenges, and applications. ACM Computing Surveys (CSUR), 53(1):1-36, 2020. URL: https://doi.org/10.1145/3371038.
  33. Zhi Wang, Tao Gong, Shu Hui Huang, and Bo Yi. Graph neural network with soft actor-critic and attention based large model for intelligent edge routing in consumer internet of things. IEEE Transactions on Consumer Electronics, 2025. Google Scholar
  34. Wenquan Xu, Zijian Zhang, Yong Feng, Haoyu Song, Zhikang Chen, Wenfei Wu, Guyue Liu, Yinchao Zhang, Shuxin Liu, Zerui Tian, and Bin Liu. Clickinc: In-network computing as a service in heterogeneous programmable data-center networks. In Proceedings of the ACM SIGCOMM 2023 Conference, ACM SIGCOMM '23, pages 798-815, New York, NY, USA, 2023. Association for Computing Machinery. URL: https://doi.org/10.1145/3603269.3604835.
  35. Sepehr Abbasi Zadeh, Ali Munir, Mahmoud Mohamed Bahnasy, Shiva Ketabi, and Yashar Ganjali. On augmenting tcp/ip stack via ebpf. In Proceedings of the 1st Workshop on eBPF and Kernel Extensions, pages 15-20, 2023. URL: https://doi.org/10.1145/3609021.3609300.
  36. Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. Apache spark: a unified engine for big data processing. Commun. ACM, 59(11):56-65, October 2016. URL: https://doi.org/10.1145/2934664.
  37. Qianyu Zhang, Gongming Zhao, Hongli Xu, and Peng Yang. Xagg: Accelerating heterogeneous distributed training through xdp-based gradient aggregation. IEEE/ACM Transactions on Networking, pages 1-15, 2023. URL: https://doi.org/10.1109/TNET.2023.3339524.
  38. Yuhong Zhong, Haoyu Li, Yu Jian Wu, Ioannis Zarkadas, Jeffrey Tao, Evan Mesterhazy, Michael Makris, Junfeng Yang, Amy Tai, Ryan Stutsman, and Asaf Cidon. XRP: In-Kernel storage functions with eBPF. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 375-393, Carlsbad, CA, July 2022. USENIX Association. URL: https://www.usenix.org/conference/osdi22/presentation/zhong.
  39. Yang Zhou, Zezhou Wang, Sowmya Dharanipragada, and Minlan Yu. Electrode: Accelerating distributed protocols with eBPF. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 1391-1407, Boston, MA, April 2023. USENIX Association. URL: https://www.usenix.org/conference/nsdi23/presentation/zhou.
  40. Yang Zhou, Xingyu Xiang, Matthew Kiley, Sowmya Dharanipragada, and Minlan Yu. DINT: Fast In-Kernel distributed transactions with eBPF. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 401-417, Santa Clara, CA, April 2024. USENIX Association. URL: https://www.usenix.org/conference/nsdi24/presentation/zhou-yang.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact