Document Open Access Logo

GSOHC: Global Synchronization Optimization in Heterogeneous Computing

Authors Soumik Kumar Basu , Jyothi Vedurada

PDF

File

Thumbnail PDF
PDF
LIPIcs.ECOOP.2025.21.pdf
  • Filesize: 1.4 MB
  • 30 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → Compilers
  • Software and its engineering → Frameworks
  • Software and its engineering → Automated static analysis
Keywords
  • Static Analysis
  • Synchronization
  • CPU-GPU
  • Heterogeneous Computing
  • Parallelization

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

The use of heterogeneous systems has become widespread and popular in the past decade with more than one type of processor, such as CPUs, GPUs (Graphics Processing Units), and FPGAs (Field Programmable Gate Arrays) etc. A wide range of applications use both CPU and GPU to leverage the benefits of their unique features and strengths. Therefore, collaborative computation between CPU and GPU is essential to achieve high program performance. However, poorly placed global synchronization barriers and synchronous memory transfers are the main bottlenecks to enhanced program performance, preventing CPU and GPU computations from overlapping. 
Based on this observation, we propose a new optimization technique called hetero-sync motion that can relocate such barrier instructions to new locations, resulting in improved performance in CPU-GPU heterogeneous programs. Further, we propose GSOHC, a compiler analysis and optimization framework that automatically finds opportunities for hetero-sync motion in the input program and then performs code transformation to apply the optimization. Our static analysis is a context-sensitive, flow-sensitive inter-procedural data-flow analysis with three phases to identify the optimization opportunities precisely. We have implemented GSOHC using LLVM/Clang infrastructure. On A4000, P100 and A100 GPUs, our optimization achieves speedups of up to 1.8x, 1.9x and 1.9x over the baseline, respectively.

Cite As Get BibTex

Soumik Kumar Basu and Jyothi Vedurada. GSOHC: Global Synchronization Optimization in Heterogeneous Computing. In 39th European Conference on Object-Oriented Programming (ECOOP 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 333, pp. 21:1-21:30, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ECOOP.2025.21

Author Details

Soumik Kumar Basu
  • Department of Computer Science and Engineering, IIT Hyderabad, India
Jyothi Vedurada
  • Department of Computer Science and Engineering, IIT Hyderabad, India

Funding

  • Kumar Basu, Soumik: Funded by SERB SRG.

Acknowledgements

We thank Anilava Kundu for his insightful comments during implementation. Further, we thank the anonymous reviewers of ECOOP for their constructive feedback.

Supplementary Materials

References

  1. Rajeev Alur, Joseph Devietti, Omar S. Navarro Leija, and Nimit Singhania. Gpudrano: Detecting uncoalesced accesses in gpu programs. In Rupak Majumdar and Viktor Kunčak, editors, Computer Aided Verification, pages 507-525, Cham, 2017. Springer International Publishing. URL: https://doi.org/10.1007/978-3-319-63387-9_25.
  2. Rajeev Alur, Joseph Devietti, and Nimit Singhania. Block-size independence for gpu programs. In International Static Analysis Symposium, pages 107-126. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-99725-4_9.
  3. Andmax. Github - gpufilter: Gpu recursive filtering. URL: https://github.com/andmax/gpufilter.git.
  4. Matthew B Ashcraft, Alexander Lemon, David A Penry, and Quinn Snell. Compiler optimization of accelerator data transfers. International Journal of Parallel Programming, 47(1):39-58, 2019. URL: https://doi.org/10.1007/S10766-017-0549-3.
  5. Michal Babej and Pekka Jääskeläinen. Hipcl: Tool for porting cuda applications to advanced opencl platforms through hip. In Proceedings of the International Workshop on OpenCL, IWOCL '20, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3388333.3388641.
  6. David F. Bacon, Susan L. Graham, and Oliver J. Sharp. Compiler transformations for high-performance computing. ACM Comput. Surv., 26(4):345-420, December 1994. URL: https://doi.org/10.1145/197405.197406.
  7. Dip Sankar Banerjee and Kishore Kothapalli. Hybrid algorithms for list ranking and graph connected components. In 2011 18th International Conference on High Performance Computing, pages 1-10, 2011. URL: https://doi.org/10.1109/HiPC.2011.6152655.
  8. Gautam Chakrabarti, Vinod Grover, Bastiaan Aarts, Xiangyun Kong, Manjunath Kudlur, Yuan Lin, Jaydeep Marathe, Mike Murphy, and Jian-Zhong Wang. Cuda: Compiling and optimizing for a gpu platform. Procedia Computer Science, 9:1910-1919, 2012. Proceedings of the International Conference on Computational Science, ICCS 2012. URL: https://doi.org/10.1016/j.procs.2012.04.209.
  9. Gautam Chakrabarti, Vinod Grover, Bastiaan Aarts, Xiangyun Kong, Manjunath Kudlur, Yuan Lin, Jaydeep Marathe, Mike Murphy, and Jian-Zhong Wang. Cuda: Compiling and optimizing for a gpu platform. Procedia Computer Science, 9:1910-1919, 2012. URL: https://doi.org/10.1016/J.PROCS.2012.04.209.
  10. Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W. Sheaffer, Sang-Ha Lee, and Kevin Skadron. Rodinia: A benchmark suite for heterogeneous computing. In 2009 IEEE International Symposium on Workload Characterization (IISWC), pages 44-54, 2009. URL: https://doi.org/10.1109/IISWC.2009.5306797.
  11. Zhe Chen, Andrew Howe, Hugh T. Blair, and Jason Cong. Clink: Compact lstm inference kernel for energy efficient neurofeedback devices. In Proceedings of the International Symposium on Low Power Electronics and Design, ISLPED '18, New York, NY, USA, 2018. Association for Computing Machinery. URL: https://doi.org/10.1145/3218603.3218637.
  12. Bruno Coutinho, Diogo Sampaio, Fernando Magno Quintao Pereira, and Wagner Meira Jr. Divergence analysis and optimizations. In 2011 International Conference on Parallel Architectures and Compilation Techniques, pages 320-329, 2011. URL: https://doi.org/10.1109/PACT.2011.63.
  13. Csl-Iisc. Github - csl-iisc/scor: A (sco)ped (r)acey benchmark suite, containing cuda code for many different gpu benchmarks. URL: https://github.com/csl-iisc/ScoR.git.
  14. L. Dagum and R. Menon. Openmp: an industry standard api for shared-memory programming. IEEE Computational Science and Engineering, 5(1):46-55, 1998. URL: https://doi.org/10.1109/99.660313.
  15. A. Danalis, Ki-Yong Kim, L. Pollock, and M. Swany. Transformations to parallel codes for communication-computation overlap. In SC '05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, pages 58-58, 2005. URL: https://doi.org/10.1109/SC.2005.75.
  16. Anthony Danalis, Lori Pollock, Martin Swany, and John Cavazos. Mpi-aware compiler optimizations for improving communication-computation overlap. In Proceedings of the 23rd International Conference on Supercomputing, ICS '09, pages 316-325, New York, NY, USA, 2009. Association for Computing Machinery. URL: https://doi.org/10.1145/1542275.1542321.
  17. Dibyendu Das, Manish Gupta, Rajan Ravindran, W. Shivani, P. Sivakeshava, and Rishabh Uppal. Compiler-controlled extraction of computation-communication overlap in mpi applications. In 2008 IEEE International Symposium on Parallel and Distributed Processing, pages 1-8, 2008. URL: https://doi.org/10.1109/IPDPS.2008.4536193.
  18. Bjorn De Sutter, Ludo Van Put, and Koen De Bosschere. A practical interprocedural dominance algorithm. ACM Trans. Program. Lang. Syst., 29(4):19-es, August 2007. URL: https://doi.org/10.1145/1255450.1255452.
  19. Pedro Diniz and Martin Rinard. Synchronization transformations for parallel computing. In Proceedings of the 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '97, pages 187-200, New York, NY, USA, 1997. Association for Computing Machinery. URL: https://doi.org/10.1145/263699.263718.
  20. M.J. Flynn. Very high-speed computing systems. Proceedings of the IEEE, 54(12):1901-1909, 1966. URL: https://doi.org/10.1109/PROC.1966.5273.
  21. Scott Grauer-Gray, Lifan Xu, Robert Searles, Sudhee Ayalasomayajula, and John Cavazos. Auto-tuning a high-level language targeted to gpu codes. In 2012 Innovative Parallel Computing (InPar), pages 1-10, 2012. URL: https://doi.org/10.1109/InPar.2012.6339595.
  22. HyTruongSon. Github - graphflow: Deep learning framework in c++/cuda that supports symbolic/automatic differentiation, dynamic computation graphs, tensor or matrix operations accelerated by gpu and implementations of various state-of-the-art graph neural networks and other machine learning models including covariant compositional networks for learning graphs [risi et al]. URL: https://github.com/HyTruongSon/GraphFlow.git.
  23. Thomas B. Jablin, Prakash Prabhu, James A. Jablin, Nick P. Johnson, Stephen R. Beard, and David I. August. Automatic cpu-gpu communication management and optimization. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '11, pages 142-151, New York, NY, USA, 2011. Association for Computing Machinery. URL: https://doi.org/10.1145/1993498.1993516.
  24. Abhinav Jangda, Saeed Maleki, Maryam Mehri Dehnavi, Madan Musuvathi, and Olli Saarikivi. A framework for fine-grained synchronization of dependent gpu kernels. In Proceedings of the 2024 IEEE/ACM International Symposium on Code Generation and Optimization, CGO '24, pages 93-105. IEEE Press, 2024. URL: https://doi.org/10.1109/CGO57630.2024.10444873.
  25. Zheming Jin. HeCBench. https://github.com/zjin-lcf/HeCBench.git, 2022. Retrieved: 2022-11-10.
  26. Jaehoon Jung, Daeyoung Park, Youngdong Do, Jungho Park, and Jaejin Lee. Overlapping host-to-device copy and computation using hidden unified memory. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '20, pages 321-335, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3332466.3374531.
  27. Christoph Kessler. Global optimization of operand transfer fusion in heterogeneous computing. In Proceedings of the 22nd International Workshop on Software and Compilers for Embedded Systems, pages 49-58, 2019. URL: https://doi.org/10.1145/3323439.3323981.
  28. Yasir Noman Khalid, Muhammad Aleem, Usman Ahmed, Muhammad Arshad Islam, and Muhammad Azhar Iqbal. Troodon: A machine-learning based load-balancing application scheduler for cpu-gpu system. Journal of Parallel and Distributed Computing, 132:79-94, 2019. URL: https://doi.org/10.1016/J.JPDC.2019.05.015.
  29. Khronos, 2020. URL: https://registry.khronos.org/SYCL/specs/.
  30. Gwangsun Kim, Jiyun Jeong, John Kim, and Mark Stephenson. Automatically exploiting implicit pipeline parallelism from multiple dependent kernels for gpus. In Proceedings of the 2016 International Conference on Parallel Architectures and Compilation, pages 341-352, 2016. URL: https://doi.org/10.1145/2967938.2967952.
  31. Chris Lattner and Vikram Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO'04), Palo Alto, California, March 2004. Google Scholar
  32. Changmin Lee, Won Woo Ro, and Jean-Luc Gaudiot. Boosting cuda applications with cpu-gpu hybrid computing. International Journal of Parallel Programming, 42(2):384-404, 2014. URL: https://doi.org/10.1007/S10766-013-0252-Y.
  33. Yunsup Lee, Ronny Krashinsky, Vinod Grover, Stephen W. Keckler, and Krste Asanović. Convergence and scalarization for data-parallel architectures. In Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pages 1-11, 2013. URL: https://doi.org/10.1109/CGO.2013.6494995.
  34. LLVM. LLVM Dominator Tree Wrapper Pass. URL: https://llvm.org/doxygen/classllvm_1_1DominatorTreeWrapperPass.html.
  35. LLVM. LLVM LoopInfoWrapperPass Class Reference. URL: https://llvm.org/doxygen/classllvm_1_1LoopInfoWrapperPass.html.
  36. LLVM. LLVM Clang. https://clang.llvm.org/, 2024. Retrieved: 2024-03-19.
  37. Sparsh Mittal and Jeffrey S. Vetter. A survey of cpu-gpu heterogeneous computing techniques. ACM Comput. Surv., 47(4), July 2015. URL: https://doi.org/10.1145/2788396.
  38. Steven Muchnick et al. Advanced compiler design implementation. Morgan kaufmann, 1997. Google Scholar
  39. V. Krishna Nandivada, Jun Shirako, Jisheng Zhao, and Vivek Sarkar. A transformation framework for optimizing task-parallel programs. ACM Trans. Program. Lang. Syst., 35(1), April 2013. URL: https://doi.org/10.1145/2450136.2450138.
  40. Ajay Nayak and Arkaprava Basu. Over-synchronization in gpu programs. MICRO, 2024. Google Scholar
  41. Van Man Nguyen, Emmanuelle Saillard, Julien Jaeger, Denis Barthou, and Patrick Carribault. Automatic code motion to extend mpi nonblocking overlap window. In High Performance Computing: ISC High Performance 2020 International Workshops, Frankfurt, Germany, June 21-25, 2020, Revised Selected Papers 35, pages 43-54. Springer, 2020. URL: https://doi.org/10.1007/978-3-030-59851-8_4.
  42. Alexandru Nicolau, Guangqiang Li, and Arun Kejariwal. Techniques for efficient placement of synchronization primitives. In Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '09, pages 199-208, New York, NY, USA, 2009. Association for Computing Machinery. URL: https://doi.org/10.1145/1504176.1504207.
  43. Alexandru Nicolau, Guangqiang Li, Alexander V. Veidenbaum, and Arun Kejariwal. Synchronization optimizations for efficient execution on multi-cores. In Proceedings of the 23rd International Conference on Supercomputing, ICS '09, pages 169-180, New York, NY, USA, 2009. Association for Computing Machinery. URL: https://doi.org/10.1145/1542275.1542303.
  44. NVIDIA. CUDA Sample SDK. https://github.com/NVIDIA/cuda-samples.git, 2022. Retrieved: 2024-03-19.
  45. David A. Padua and Michael J. Wolfe. Advanced compiler optimizations for supercomputers. Commun. ACM, 29(12):1184-1201, December 1986. URL: https://doi.org/10.1145/7902.7904.
  46. Anthony Pajot, Loic Barthe, Mathias Paulin, and Pierre Poulin. Combinatorial Bidirectional Path-Tracing for Efficient Hybrid CPU/GPU Rendering. Computer Graphics Forum, 30(2):315-324, April 2011. URL: https://doi.org/10.1111/j.1467-8659.2011.01863.x.
  47. Song Jun Park, James Ross, Dale Shires, David Richie, Brian Henz, and Lam Nguyen. Hybrid core acceleration of uwb sire radar signal processing. IEEE Transactions on Parallel and Distributed Systems, 22(1):46-57, 2011. URL: https://doi.org/10.1109/TPDS.2010.117.
  48. Daniel P Playne, Mitchell Johnson, and Kenneth A Hawick. Benchmarking gpu devices with n-body simulations. CDES, 9:132, 2009. Google Scholar
  49. Martin Rinard and Pedro Diniz. Eliminating synchronization bottlenecks in object-based programs using adaptive replication. In Proceedings of the 13th International Conference on Supercomputing, ICS '99, pages 83-92, New York, NY, USA, 1999. Association for Computing Machinery. URL: https://doi.org/10.1145/305138.305167.
  50. V. Sarkar. Instruction reordering for fork-join parallelism. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation, PLDI '90, pages 322-336, New York, NY, USA, 1990. Association for Computing Machinery. URL: https://doi.org/10.1145/93542.93590.
  51. Charitha Saumya, Kirshanthan Sundararajah, and Milind Kulkarni. Darm: Control-flow melding for simt thread divergence reduction. In Proceedings of the 20th IEEE/ACM International Symposium on Code Generation and Optimization, CGO '22, pages 28-40. IEEE Press, 2022. URL: https://doi.org/10.1109/CGO53902.2022.9741285.
  52. Yifan Sun, Xiang Gong, Amir Kavyan Ziabari, Leiming Yu, Xiangyu Li, Saoni Mukherjee, Carter McCardwell, Alejandro Villegas, and David Kaeli. Hetero-mark, a benchmark suite for cpu-gpu collaborative computing. In 2016 IEEE International Symposium on Workload Characterization (IISWC), pages 1-10. IEEE, 2016. Google Scholar
  53. J.B. White III and J.J. Dongarra. Overlapping computation and communication for advection on hybrid parallel computers. In 2011 IEEE International Parallel & Distributed Processing Symposium, pages 59-67, 2011. URL: https://doi.org/10.1109/IPDPS.2011.16.
  54. Jingyue Wu, Artem Belevich, Eli Bendersky, Mark Heffernan, Chris Leary, Jacques Pienaar, Bjarke Roune, Rob Springer, Xuetian Weng, and Robert Hundt. Gpucc: An open-source gpgpu compiler. In Proceedings of the 2016 International Symposium on Code Generation and Optimization, CGO '16, pages 105-116, New York, NY, USA, 2016. Association for Computing Machinery. URL: https://doi.org/10.1145/2854038.2854041.
  55. Ming Xu, Feiguo Chen, Xinhua Liu, Wei Ge, and Jinghai Li. Discrete particle simulation of gas–solid two-phase flows with multi-scale cpu–gpu hybrid computation. Chemical Engineering Journal, 207-208:746-757, 2012. 22nd International Symposium on Chemical Reaction Engineering (ISCRE 22). URL: https://doi.org/10.1016/j.cej.2012.07.049.
  56. Yingchun Zhu and Laurie J. Hendren. Communication optimizations for parallel c programs. SIGPLAN Not., 33(5):199-211, May 1998. URL: https://doi.org/10.1145/277652.277723.
  57. Zjin-Lcf. Github - hecbench/src/entropy-cuda. URL: https://github.com/zjin-lcf/HeCBench/.
  58. zjin lcf. Github - hecbench/src/xlqc-cuda, 2020. URL: https://github.com/zjin-lcf/HeCBench/.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact