Document Open Access Logo

When to Give up on a Parallel Implementation

Authors Nathan S. Sheffield , Alek Westover

PDF
Thumbnail PDF

File

LIPIcs.ITCS.2025.87.pdf
  • Filesize: 0.79 MB
  • 18 pages

Document Identifiers

Author Details

Nathan S. Sheffield
  • MIT, Cambridge, MA, USA
Alek Westover
  • MIT, Cambridge, MA, USA

Cite As Get BibTex

Nathan S. Sheffield and Alek Westover. When to Give up on a Parallel Implementation. In 16th Innovations in Theoretical Computer Science Conference (ITCS 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 325, pp. 87:1-87:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ITCS.2025.87

Abstract

In the Serial Parallel Decision Problem (SPDP), introduced by Kuszmaul and Westover [SPAA'24], an algorithm receives a series of tasks online, and must choose for each between a serial implementation and a parallelizable (but less efficient) implementation. Kuszmaul and Westover describe three decision models: (1) Instantly-committing schedulers must decide on arrival, irrevocably, which implementation of the task to run. (2) Eventually-committing schedulers can delay their decision beyond a task’s arrival time, but cannot revoke their decision once made. (3) Never-committing schedulers are always free to abandon their progress on the task and start over using a different implementation. Kuszmaul and Westover gave a simple instantly-committing scheduler whose total completion time is 3-competitive with the offline optimal schedule, and proved two lower bounds: no eventually-committing scheduler can have competitive ratio better than ϕ ≈ 1.618 in general, and no instantly-committing scheduler can have competitive ratio better than 2 in general. They conjectured that the three decision models should admit different competitive ratios, but left upper bounds below 3 in any model as an open problem.
In this paper, we show that the powers of instantly, eventually, and never committing schedulers are distinct, at least in the "massively parallel regime". The massively parallel regime of the SPDP is the special case where the number of available processors is asymptotically larger than the number of tasks to process, meaning that the work associated with running a task in serial is negligible compared to its runtime. In this regime, we show (1) The optimal competitive ratio for instantly-committing schedulers is 2, (2) The optimal competitive ratio for eventually-committing schedulers lies in [1.618, 1.678], (3) The optimal competitive ratio for never-committing schedulers lies in [1.366, 1.500]. We additionally show that our instantly-committing scheduler is also 2-competitive outside of the massively parallel regime, giving proof-of-concept that results in the massively parallel regime can be translated to hold with fewer processors.

Subject Classification

ACM Subject Classification
  • Theory of computation → Online algorithms
Keywords
  • Scheduling
  • Multi-Processor
  • Online-Algorithms

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. S Anand, Naveen Garg, and Nicole Megow. Meeting deadlines: How much speed suffices? In Automata, Languages and Programming: 38th International Colloquium, ICALP 2011, Zurich, Switzerland, July 4-8, 2011, Proceedings, Part I 38, pages 232-243. Springer, 2011. URL: https://doi.org/10.1007/978-3-642-22006-7_20.
  2. James Aspnes, Yossi Azar, Amos Fiat, Serge Plotkin, and Orli Waarts. On-line routing of virtual circuits with applications to load balancing and machine scheduling. Journal of the ACM (JACM), 44(3):486-504, 1997. URL: https://doi.org/10.1145/258128.258201.
  3. Baruch Awerbuch, Yossi Azar, Edward F Grove, Ming-Yang Kao, P Krishnan, and Jeffrey Scott Vitter. Load balancing in the l_p norm. In Proceedings of IEEE 36th Annual Foundations of Computer Science, pages 383-391. IEEE, 1995. URL: https://doi.org/10.1109/SFCS.1995.492494.
  4. Brenda S Baker and Jerald S Schwarz. Shelf algorithms for two-dimensional packing problems. SIAM Journal on Computing, 12(3):508-525, 1983. URL: https://doi.org/10.1137/0212033.
  5. Ioannis Caragiannis. Better bounds for online load balancing on unrelated machines. In Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms, pages 972-981, 2008. URL: http://dl.acm.org/citation.cfm?id=1347082.1347188.
  6. Shichuan Deng, Jian Li, and Yuval Rabani. Generalized unrelated machine scheduling problem. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 2898-2916. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977554.CH110.
  7. Richard A. Dutton and Weizhen Mao. Online scheduling of malleable parallel jobs. In Proceedings of the 19th IASTED International Conference on Parallel and Distributed Computing and Systems, PDCS '07, pages 136-141, USA, 2007. ACTA Press. Google Scholar
  8. R. L. Graham. Bounds on multiprocessing timing anomalies. SIAM Journal on Applied Mathematics, 17(2):416-429, 1969. URL: https://doi.org/10.1137/0117039.
  9. Shouwei Guo and Liying Kang. Online scheduling of malleable parallel jobs with setup times on two identical machines. European Journal of Operational Research, 206(3):555-561, November 2010. URL: https://doi.org/10.1016/j.ejor.2010.03.005.
  10. Anupam Gupta, Amit Kumar, Viswanath Nagarajan, and Xiangkun Shen. Stochastic load balancing on unrelated machines. Mathematics of Operations Research, 46(1):115-133, 2021. URL: https://doi.org/10.1287/MOOR.2019.1049.
  11. Varun Gupta, Benjamin Moseley, Marc Uetz, and Qiaomin Xie. Stochastic online scheduling on unrelated machines. In Integer Programming and Combinatorial Optimization: 19th International Conference, IPCO 2017, Waterloo, ON, Canada, June 26-28, 2017, Proceedings 19, pages 228-240. Springer, 2017. URL: https://doi.org/10.1007/978-3-319-59250-3_19.
  12. Varun Gupta, Benjamin Moseley, Marc Uetz, and Qiaomin Xie. Greed works—online algorithms for unrelated machine stochastic scheduling. Mathematics of operations research, 45(2):497-516, 2020. URL: https://doi.org/10.1287/MOOR.2019.0999.
  13. Ellis Horowitz and Sartaj Sahni. Exact and approximate algorithms for scheduling nonidentical processors. J. ACM, 23(2):317-327, April 1976. URL: https://doi.org/10.1145/321941.321951.
  14. Johann L Hurink and Jacob Jan Paulus. Online algorithm for parallel job scheduling and strip packing. In Approximation and Online Algorithms: 5th International Workshop, WAOA 2007, Eilat, Israel, October 11-12, 2007. Revised Papers 5, pages 67-74. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-77918-6_6.
  15. Sungjin Im and Shi Li. Improved approximations for unrelated machine scheduling. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 2917-2946. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977554.CH111.
  16. William Kuszmaul and Alek Westover. Scheduling jobs with work-inefficient parallel solutions. In Proceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures, pages 101-111, 2024. URL: https://doi.org/10.1145/3626183.3659960.
  17. Jan Karel Lenstra, David B Shmoys, and Éva Tardos. Approximation algorithms for scheduling unrelated parallel machines. Mathematical programming, 46:259-271, 1990. URL: https://doi.org/10.1007/BF01585745.
  18. Walter Ludwig and Prasoon Tiwari. Scheduling malleable and nonmalleable parallel tasks. In Proceedings of the fifth annual ACM-SIAM symposium on Discrete algorithms, pages 167-176, 1994. URL: http://dl.acm.org/citation.cfm?id=314464.314491.
  19. Marco Molinaro. Stochastic 𝓁_p Load Balancing and Moment Problems via the L-Function Method, pages 343-354. SIAM, 2019. URL: https://doi.org/10.1137/1.9781611975482.22.
  20. Gregory Mounie, Christophe Rapine, and Dennis Trystram. Efficient approximation algorithms for scheduling malleable tasks. In Proceedings of the eleventh annual ACM symposium on Parallel algorithms and architectures, SPAA '99, pages 23-32, New York, NY, USA, 1999. Association for Computing Machinery. URL: https://doi.org/10.1145/305619.305622.
  21. Daniel R. Page, Roberto Solis-Oba, and Marten Maack. Makespan minimization on unrelated parallel machines with simple job-intersection structure and bounded job assignments. Theoretical Computer Science, 809:204-217, 2020. URL: https://doi.org/10.1016/j.tcs.2019.12.009.
  22. Andreas S. Schulz and Martin Skutella. Scheduling unrelated machines by randomized rounding. SIAM Journal on Discrete Mathematics, 15(4):450-469, 2002. URL: https://doi.org/10.1137/S0895480199357078.
  23. John Turek, Walter Ludwig, Joel L. Wolf, Lisa Fleischer, Prasoon Tiwari, Jason Glasgow, Uwe Schwiegelshohn, and Philip S. Yu. Scheduling parallelizable tasks to minimize average response time. In Proceedings of the sixth annual ACM symposium on Parallel algorithms and architectures, SPAA '94, pages 200-209, New York, NY, USA, 1994. Association for Computing Machinery. URL: https://doi.org/10.1145/181014.181331.
  24. John Turek, Uwe Schwiegelshohn, Joel L Wolf, and Philip S Yu. Scheduling parallel tasks to minimize average response time. In Proceedings of the fifth annual ACM-SIAM symposium on Discrete algorithms, pages 112-121, 1994. URL: http://dl.acm.org/citation.cfm?id=314464.314485.
  25. John Turek, Joel L Wolf, and Philip S Yu. Approximate algorithms scheduling parallelizable tasks. In Proceedings of the fourth annual ACM symposium on Parallel algorithms and architectures, pages 323-332, 1992. URL: https://doi.org/10.1145/140901.141909.
  26. Nodari Vakhania, Jose Hernandez, and Frank Werner. Scheduling unrelated machines with two types of jobs. International Journal of Production Research, 52:1-9, February 2014. URL: https://doi.org/10.1080/00207543.2014.888789.
  27. Andrew Chi-Chin Yao. Probabilistic computations: Toward a unified measure of complexity. In 18th Annual Symposium on Foundations of Computer Science (sfcs 1977), pages 222-227. IEEE Computer Society, 1977. Google Scholar
  28. Deshi Ye, Danny Z. Chen, and Guochuan Zhang. Online scheduling of moldable parallel tasks. Journal of Scheduling, 21(6):647-654, 2018. Publisher: Springer. URL: https://ideas.repec.org//a/spr/jsched/v21y2018i6d10.1007_s10951-018-0556-2.html, URL: https://doi.org/10.1007/S10951-018-0556-2.
  29. Deshi Ye, Xin Han, and Guochuan Zhang. A note on online strip packing. Journal of Combinatorial Optimization, 17(4):417-423, 2009. URL: https://doi.org/10.1007/S10878-007-9125-X.
  30. Xiaoyan Zhang, Ran Ma, Jian Sun, and Zan-Bo Zhang. Randomized selection algorithm for online stochastic unrelated machines scheduling. Journal of Combinatorial Optimization, pages 1-16, 2022. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact