Document Open Access Logo

Maniposynth: Bimodal Tangible Functional Programming

Authors Brian Hempel, Ravi Chugh

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2022.16.pdf
  • Filesize: 3.78 MB
  • 29 pages

Document Identifiers

Author Details

Brian Hempel
  • University of Chicago, IL, USA
Ravi Chugh
  • University of Chicago, IL, USA

Funding

This work was supported by U.S. NSF grant Direct Manipulation Programming Systems (CCF-1651794).

Acknowledgements

We extend our gratitude to Justin Lubin for advice (so far ignored) about caching during synthesis, Byron Zhang for feedback on the presentation, Matt Teichman for providing hardware, Kartik Singhal for technical support, and the user study participants for their great patience with bugs and for the invaluable feedback they provided.

Cite AsGet BibTex

Brian Hempel and Ravi Chugh. Maniposynth: Bimodal Tangible Functional Programming. In 36th European Conference on Object-Oriented Programming (ECOOP 2022). Leibniz International Proceedings in Informatics (LIPIcs), Volume 222, pp. 16:1-16:29, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)
https://doi.org/10.4230/LIPIcs.ECOOP.2022.16

Abstract

Traditionally, writing code is a non-graphical, abstract, and linear process. Not everyone is comfortable with this way of thinking at all times. Can programming be transformed into a graphical, concrete, non-linear activity? While nodes-and-wires [Sutherland, 1966] and blocks-based [Begel, 1996] programming environments do leverage graphical direct manipulation, users perform their manipulations on abstract syntax tree elements, which are still abstract. Is it possible to be more concrete - could users instead directly manipulate live program values to create their program? We present a system, Maniposynth, that reimagines functional programming as a non-linear workflow where program expressions are spread on a 2D canvas. The live results of those expressions are continuously displayed and available for direct manipulation. The non-linear canvas liberates users to work out-of-order, and the live values can be interacted with via drag-and-drop. Incomplete programs are gracefully handled via hole expressions, which allow Maniposynth to offer program synthesis. Throughout the workflow, the program is valid OCaml code which the user may inspect and edit in their preferred text editor at any time. With Maniposynth’s direct manipulation features, we created 38 programs drawn from a functional data structures course. We additionally hired two professional OCaml developers to implement a subset of these programs. We report on these experiences and discuss to what degree Maniposynth meets its goals of providing a non-linear, concrete, graphical programming workflow.

Subject Classification

ACM Subject Classification
  • Software and its engineering → Integrated and visual development environments
  • Software and its engineering → Visual languages
  • Software and its engineering → Programming by example
  • Human-centered computing → Human computer interaction (HCI)
Keywords
  • direct manipulation
  • tangible programming
  • programming user interfaces

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Michel Adam, Moncef Daoud, and Patrice Frison. Direct Manipulation versus Text-based Programming: An Experiment Report. In Conference on Innovation and Technology in Computer Science Education (ITiCS), 2019. Google Scholar
  2. Haytham Amairah. Wrong Pattern for Filling Month Names. https://techcommunity.microsoft.com/t5/excel/flash-fill-wrong-pattern-for-filling-month-names/m-p/355213, 2019.
  3. Andrew Begel. LogoBlocks: A Graphical Programming Language for Interacting with the World, 1996. Advanced Undergraduate Project, MIT Media Lab. Google Scholar
  4. Josh Berdine, Guillaume Petiot, hhugo, and contributors. Ocamlformat. https://github.com/ocaml-ppx/ocamlformat, 2021.
  5. Bob Boothe. Efficient Algorithms for Bidirectional Debugging. In Conference on Programming Language Design and Implementation (PLDI), 2000. URL: https://doi.org/10.1145/349299.349339.
  6. Frédéric Bour, Thomas Refis, Gemma Gordon, Simon Castellan, and contributors. Merlin. https://github.com/ocaml/merlin, 2021.
  7. Andrew Bragdon, Steven P. Reiss, Robert Zeleznik, Suman Karumuri, William Cheung, Joshua Kaplan, Christopher Coleman, Ferdi Adeputra, and Joseph J. LaViola, Jr. Code Bubbles: Rethinking the User Interface Paradigm of Integrated Development Environments. In International Conference on Software Engineering (ICSE), 2010. Google Scholar
  8. Andrew Bragdon, Robert Zeleznik, Steven P. Reiss, Suman Karumuri, William Cheung, Joshua Kaplan, Christopher Coleman, Ferdi Adeputra, and Joseph J. LaViola, Jr. Code Bubbles: A Working Set-based Interface for Code Understanding and Maintenance. In Conference on Human Factors in Computing Systems (CHI), 2010. Google Scholar
  9. Alexander Breckel and Matthias Tichy. Embedding Programming Context into Source Code. In International Conference on Program Comprehension (ICPC), 2016. URL: https://doi.org/10.1109/ICPC.2016.7503732.
  10. Glen Chiacchieri. F-F-F-Flowsheets v2 demo. https://www.youtube.com/watch?v=y1Ca5czOY7Q, 2017.
  11. Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. Programmatic and Direct Manipulation, Together at Last. In Conference on Programming Language Design and Implementation (PLDI), 2016. Google Scholar
  12. Naëla Courant, Julien Lepiller, and Gabriel Scherer. camlboot. https://github.com/Ekdohibs/camlboot/, 2020.
  13. Allen Cypher, Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, and Alan Turransky, editors. Watch What I Do: Programming by Demonstration. MIT Press, 1993. Google Scholar
  14. Jacob Diamond-Reivich. Mito: Edit a Spreadsheet. Generate Production Ready Python. In LIVE Workshop, 2020. Google Scholar
  15. Andrea A. diSessa and Harold Abelson. Boxer: A Reconstructible Computational Medium. Communications of the ACM (CACM), 1986. Google Scholar
  16. Ian Drosos, Titus Barik, Philip J Guo, Robert DeLine, and Sumit Gulwani. Wrex: A Unifed Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. Conference on Human Factors in Computing Systems (CHI), 2020. Google Scholar
  17. Jonathan Edwards. Example Centric Programming. In Conference on Object-Oriented Programming Languages, Systems, and Applications (OOPSLA), 2004. URL: https://doi.org/10.1145/1028664.1028713.
  18. Jonathan Edwards. Direct Programming. https://vimeo.com/274771188, 2018.
  19. Conal Elliott. Tangible Functional Programming. In International Conference on Functional Programming (ICFP), 2007. URL: http://conal.net/papers/Eros/.
  20. Enso. Enso. URL: https://enso.org/.
  21. Kasra Ferdowsifard, Allen Ordookhanians, Hila Peleg, Sorin Lerner, and Nadia Polikarpova. Small-Step Live Programming by Example. In Symposium on User Interface Software and Technology (UIST), 2020. Google Scholar
  22. Patrice Frison. A Teaching Assistant for Algorithm Construction. In Conference on Innovation and Technology in Computer Science Education (ITiCS), 2015. Google Scholar
  23. Koumei Fukahori, Daisuke Sakamoto, Jun Kato, and Takeo Igarashi. CapStudio: An Interactive Screencast for Visual Application Development. In Conference on Human Factors in Computing Systems (CHI), Extended Abstracts, 2014. Google Scholar
  24. Joel Galenson, Philip Reames, Rastislav Bodík, Björn Hartmann, and Koushik Sen. CodeHint: Dynamic and Interactive Synthesis of Code Snippets. In International Conference on Software Engineering (ICSE), 2014. Google Scholar
  25. Thomas R. G. Green and Marian Petre. Usability Analysis of Visual Programming Environments: A Cognitive Dimensions Framework. J. Vis. Lang. Comput., 1996. URL: https://doi.org/10.1006/jvlc.1996.0009.
  26. Rudi Grinberg, Andrey Popp, Rusty Key, Louis Roché, Oleksiy Golovko, Sacha Ayoun, cannorin, Ulugbek Abdullaev, Thibaut Mattio, and Max Lantas. OCaml-LSP. https://github.com/ocaml/ocaml-lsp, 2021.
  27. Sumit Gulwani. Automating String Processing in Spreadsheets Using Input-Output Examples. In Symposium on Principles of Programming Languages (POPL), 2011. Google Scholar
  28. Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. Program Synthesis. Foundations and Trends in Programming Languages, 2017. URL: https://doi.org/10.1561/2500000010.
  29. Philip J. Guo. Online Python Tutor: Embeddable Web-based Program Visualization for Cs Education. In Technical Symposium on Computer Science Education (SIGCSE), 2013. Google Scholar
  30. Keith Hanna. Interactive Visual Functional Programming. In International Conference on Functional Programming (ICFP), 2002. Google Scholar
  31. Keith Hanna. A Document-Centered Environment for Haskell. In International Workshop on Implementation and Application of Functional Languages (IFL), 2005. Google Scholar
  32. Brian Hempel. The Magnificent Maniposynth. http://maniposynth.org, 2022.
  33. Brian Hempel, Justin Lubin, and Ravi Chugh. Sketch-n-Sketch: Output-Directed Programming for SVG. In Symposium on User Interface Software and Technology (UIST), 2019. Google Scholar
  34. Joshua Horowitz. PANE: Programming with Visible Data. In LIVE Workshop, 2018. URL: http://joshuahhh.com/projects/pane/.
  35. Yen-Teh Hsia and Allen L. Ambler. Programming Through Pictorial Transformations. In International Conference on Computer Languages, 1988. Google Scholar
  36. Qinheping Hu, Roopsha Samanta, Rishabh Singh, and Loris D'Antoni. Direct Manipulation for Imperative Programs. In Static Analysis Symposium (SAS), 2019. Google Scholar
  37. Christopher D. Hundhausen and Jonathan Lee Brown. What You See Is What You Code: A live Algorithm Development and Visualization Environment for Novice Learners. Journal of Visual Languages and Computing, 2007. URL: https://doi.org/10.1016/j.jvlc.2006.03.002.
  38. Christopher D. Hundhausen, Sean Farley, and Jonathan Lee Brown. Can Direct Manipulation Lower the Barriers To Computer Programming and Promote Transfer of Training? An Experimental Study. ACM Trans. Comput. Hum. Interact., 2009. URL: https://doi.org/10.1145/1592440.1592442.
  39. Ruyi Ji, Yican Sun, Yingfei Xiong, and Zhenjiang Hu. Guiding Dynamic Programing Via Structural Probability for Accelerating Programming by Example. Proc. ACM Program. Lang., (OOPSLA), 2020. URL: https://doi.org/10.1145/3428292.
  40. Saketh Kasibatla and Alex Warth. Seymour: Live Programming for the Classroom. In LIVE Workshop, 2017. Google Scholar
  41. Mary Beth Kery, Donghao Ren, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, and Kayur Patel. mage: Fluid Moves Between Code and Graphical Work In Computational Notebooks. In Symposium on User Interface Software and Technology (UIST), 2020. Google Scholar
  42. Andrew J. Ko and Brad A. Myers. Barista: An Implementation Framework for Enabling New Tools, Interaction Techniques and Views in Code Editors. In Human Factors in Computing Systems (CHI), 2006. Google Scholar
  43. Kevin Kwok and Guillermo Webster. Carbide Alpha, 2016. URL: https://alpha.trycarbide.com/.
  44. A. H. Land and A. G. Doig. An automatic method of solving discrete programming problems. Econometrica, 1960. Google Scholar
  45. Woosuk Lee, Kihong Heo, Rajeev Alur, and Mayur Naik. Accelerating Search-based Program Synthesis using Learned Probabilistic Models. In Conference on Programming Language Design and Implementation (PLDI), 2018. Google Scholar
  46. Sorin Lerner. Projection Boxes: On-the-fly Reconfigurable Visualization for Live Programming. Conference on Human Factors in Computing Systems (CHI), 2020. Google Scholar
  47. H. Lieberman, editor. Your Wish is My Command: Programming by Example. Morgan Kaufmann Publishers Inc., 2001. Google Scholar
  48. Justin Lubin, Nick Collins, Cyrus Omar, and Ravi Chugh. Program Sketching with Live Bidirectional Evaluation. Proc. ACM Program. Lang., (ICFP), 2020. URL: https://doi.org/10.1145/3408991.
  49. John H. Maloney and Randall B. Smith. Directness and Liveness In the Morphic User Interface Construction Environment. In Symposium on User Interface Software and Technology (UIST), 1995. Google Scholar
  50. Mikaël Mayer, Viktor Kunčak, and Ravi Chugh. Bidirectional Evaluation with Direct Manipulation. Proceedings of the ACM on Programming Languages (PACMPL), Issue OOPSLA, 2018. Google Scholar
  51. Sean McDirmid. A Live Programming Experience. In Future Programming Workshop, Strange Loop, 2015. https://www.youtube.com/watch?v=YLrdhFEAiqo. URL: https://onedrive.live.com/download?cid=51C4267D41507773&resid=51C4267D41507773%2111492&authkey=AMwcxdryTyPiuW8.
  52. Lambert Meertens. Designing Constraint Maintainers for User Interaction. https://www.kestrel.edu/people/meertens/pub/dcm.pdf, 1998.
  53. Microsoft. Visual studio code. https://code.visualstudio.com, 2022.
  54. Francesmary Modugno, Albert T. Corbett, and Brad A. Myers. Graphical Representation of Programs In a Demonstrational Visual Shell - An Empirical Evaluation. ACM Trans. Comput. Hum. Interact., 1997. URL: https://doi.org/10.1145/264645.264659.
  55. National Instruments. Labview. URL: https://www.ni.com/en-us/shop/labview.html.
  56. Tobias Nipkow and Mohammad Abdulaziz. Functional Data Structures (in2347). https://github.com/nipkow/fds_ss20/tree/daae0f92277b0df86f34ec747c7b3f1c5f0a725c, 2020. Technische Universität München.
  57. Robert O'Callahan, Chris Jones, Nathan Froyd, Kyle Huey, Albert Noll, and Nimrod Partush. Engineering Record and Replay for Deployability. In USENIX Annual Technical Conference, 2017. URL: https://www.usenix.org/conference/atc17/technical-sessions/presentation/ocallahan.
  58. Cyrus Omar, Ian Voysey, Ravi Chugh, and Matthew A. Hammer. Live Functional Programming with Typed Holes. In ACM SIGPLAN Symposium on Principles of Programming Languages (POPL), 2019. Google Scholar
  59. Peter-Michael Osera and Steve Zdancewic. Type-and-Example-Directed Program Synthesis. In Conference on Programming Language Design and Implementation (PLDI), 2015. Google Scholar
  60. Roly Perera, Minh Nguyen, Tomas Petricek, and Meng Wang. Linked Visualisations Via Galois Dependencies. Proc. ACM Program. Lang., (POPL), 2022. URL: https://doi.org/10.1145/3498668.
  61. David Rauch, Patrick Rein, Stefan Ramson, Jens Lincke, and Robert Hirschfeld. Babylonian-style Programming - Design and Implementation of An Integration of Live Examples Into General-purpose Source Code. Art Sci. Eng. Program., 2019. URL: https://doi.org/10.22152/programming-journal.org/2019/3/9.
  62. Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, and Yasmin Kafai. Scratch: Programming for All. Communications of the ACM (CACM), 2009. Google Scholar
  63. Robin Schreiber, Robert Krahn, Daniel H. H. Ingalls, and Robert Hirschfeld. Transmorphic: Mapping Direct Manipulation to Source Code Transformations. HPI/Potsdam University, 2017. Google Scholar
  64. Christopher Schuster and Cormac Flanagan. Live Programming by Example: Using Direct Manipulation for Live Program Synthesis. In LIVE Workshop, 2016. Google Scholar
  65. Paul Shen. natto.dev. URL: https://natto.dev/.
  66. Ben Shneiderman. Direct Manipulation: A Step Beyond Programming Languages. Computer, August 1983. Google Scholar
  67. David Canfield Smith. Pygmalion: A Creative Programming Environment. PhD thesis, Stanford University, 1975. Google Scholar
  68. Armando Solar-Lezama. Program Synthesis by Sketching. PhD thesis, UC Berkeley, 2008. Google Scholar
  69. Fabio Spampinato. Highlight VS Code Extension. https://marketplace.visualstudio.com/items?itemName=fabiospampinato.vscode-highlight, 2021.
  70. Sun Microsystems. Self: The movie;. http://www.smalltalk.org.br/movies/self.html, 1995.
  71. William Robert Sutherland. The On-line Graphical Specification of Computer Procedures. PhD thesis, Massachusetts Institute of Technology, 1966. Google Scholar
  72. David M. Ungar and Randall B. Smith. Self: The Power of Simplicity. In Conference on Object-Oriented Programming Languages, Systems, and Applications (OOPSLA), 1987. Google Scholar
  73. Victor, Bret. Inventing on Principle, 2012. URL: https://vimeo.com/36579366.
  74. Xiaoyin Wang, Lu Zhang, Tao Xie, Yingfei Xiong, and Hong Mei. Automating Presentation Changes in Dynamic Web Applications via Collaborative Hybrid Analysis. In International Symposium on the Foundations of Software Engineering (FSE), 2012. Google Scholar
  75. Yifan Wu, Joseph M. Hellerstein, and Arvind Satyanarayan. B2: Bridging Code and Interactive Visualization In Computational Notebooks. In Symposium on User Interface Software and Technology (UIST), 2020. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact