Educational Programming Languages and Systems

OCaml Blockly is a block-based environment for OCaml, built on top of Google Blockly. It has the following features:

• $\mathrm{■}$

  It produces no syntax error, once all the holes are filled with blocks.

• $\mathrm{■}$

  It produces no unbound variable error, since the scoping rules of OCaml are built in. Variable blocks can exist only at the legal places.

• $\mathrm{■}$

  It produces no type error, since the typing rules of OCaml are built in.

In a nutshell, OCaml Blockly behaves decently as the language designer expects. It has been used seriously since 2019 in a functional language course for CS-major students as well as an introductory game programming course for non-CS-major students and one-day seminars for high school students.

OCaml Stepper is an algebraic stepper for OCaml that supports the basic constructs of OCaml as well as exception handling, outputs, and modules. Since it produces each step incrementally, it can step execute non-terminating programs. It is used in a functional language course for CS-major students to help the students understand the behavior of programs, in particular, how recursion works, where an infinite loop occurs, and where programs exhibit unintended behaviors.

Snap! has broad support for making customizations to the programming environment that aid in teaching students. Most of these tools are designed for instructors to build Snap! Projects that they then give to students, rather than something students use themselves. Snap! has always had the ability to make custom control structures which aid in creating “Domain Specific Languages” that may be easier for students to learn. Recently, we have expanded the suite of tools that allow instructors to create “microworlds” which direct the student’s attention to a subset of blocks [1]. Finally, I’ll show how we can use metaprogramming in Snap! to write code that dynamically changes the environment, such as adding or hiding blocks. This allows instructors to add “levels” or different modes to projects.

Data processing is an important use of computers. Spreadsheets have issues with usability and flexibility, while programming languages like R can be confusing for novices and can obscure the data which should be the central focus. Columnal is a tool that on the surface appears like a spreadsheet, but underneath has a pure functional programming model that manipulates tables in a processing pipeline. The transformations are made visible and fully reproducible in a visual system.

Program source code can often be separated into statements and expressions. Statements are well-suited to being represented as draggable items in block-based programming. However, expressions are less well suited to mouse manipulation as blocks. In this talk I showed an alternative approach, from our “Stride” tool, which structures expressions more at the lexing level than the parsing level, and supports keyboard entry while maintaining the structured nature that eliminates many syntax errors.

The program design recipe is a sequence of steps for defining functions. It serves as an excellent guidance for beginning students, but students tend to skip intermediate steps and go straight to coding, either because they think those steps are not important, or because they do not know what to write at each step. We propose Mio, an environment with built-in support for design-recipe-based programming. Mio provides blocks for steps before coding, and generates feedback on those steps. A preliminary experiment shows that the blocks and feedback can be effective in encouraging students to follow the recipe.

Ask-Elle is a tutor for learning the higher-order, strongly-typed functional programming language Haskell. It supports the stepwise development of Haskell programs by verifying the correctness of incomplete programs, and by providing hints. Programming exercises are added to Ask-Elle by providing a task description for the exercise, one or more model solutions, and properties that a solution should satisfy. The properties and model solutions can be annotated with feedback messages, and the amount of flexibility that is allowed in student solutions can be adjusted. The main contribution of our work is the design of a tutor that combines (1) the incremental development of different solutions in various forms to a programming exercise with (2) automated feedback and (3) teacher-specified programming exercises, solutions, and properties. The main functionality is obtained by means of strategy-based model tracing and property-based testing. We have tested the feasibility of our approach in several experiments, in which we analyse both intermediate and final student solutions to programming exercises. Similar techniques have been used in tutoring systems for code refactoring and for proofs with structural induction.

Every year, millions of students learn how to write programs. Learning activities for beginners almost always include programming tasks that require a student to write a program to solve a particular problem. When learning how to solve such a task, many students need feedback on their previous actions, and hints on how to proceed. For tasks such as programming, which are most often solved step by step, the feedback should take the steps a student has taken towards implementing a solution into account, and the hints should help a student to complete or improve a possibly partial solution. Research on feedback addresses questions about when feedback should be given, why, and how. This talk discusses how this research is translated to when, why and how to give feedback on, or a hint at, a particular step a student takes when solving a programming task. We select datasets consisting of sequences of steps students take when working towards a solution to a programming problem, and annotate these datasets at those places at which we think an expert should intervene, why the expert intervenes, and how the expert wants to intervene. We use these datasets to compare feedback and hints given by learning environments for programming to the hints and feedback in the annotated datasets.

Development environments mediate between the programmer and the machine. Aimed at novice programmers, TigerJython is a Python environment that seeks to improve, in particular, the programmer’s interaction with the executing interpreter. It separates I/O more clearly from program code, displays enhanced error messages and provides a debugger for program visualisation, but also deliberately refrains from more complex features such as project management. A key insight for the design was to base the various features on a consistent and novice-friendly mental model of the machine.

What materials and media can we reach for to practice and learn programming? Many present themselves to our attention. Over multiple talks, we sampled several of these:

• $\mathrm{■}$

  Program planning using a mix of code and text (thanks, Snap!)

• $\mathrm{■}$

  Multiple notional machines based on stacks and trees

• $\mathrm{■}$

  New models of reactive programming, especially ones that attend to a diversity of educational and intellectual needs beyond just the production of output

• $\mathrm{■}$

  The exploration of variation in semantics through bottom-up discovery followed by top-down theory formation

There have been past (Liao & Brigth, 1991) and more recent (Scherer et al., 2019) attempts to conduct meta-analyses on the transfer effects of learning to program on the acquisition of near (e.g., programming, programming conceptions) and far cognitive skills (e.g., mathematics, reasoning, creativity). These meta-analyses have reported overall transfer effect sizes of 0.41 and 0.49 respectively. However, when taking a closer look at the results, it becomes clear that it is premature to draw such conclusions as many studies on which the meta-analyses are based suffer from methodological problems, such as small sample size, the absence of an alternative treatment control group, and the absence of a pretest before the intervention. In addition, a couple of unexpected findings are reported, such as very high effect sizes (>2.0 or even >3.0), no effect of treatment duration and the fact that the largest effect sizes were found for transfer to creative thinking. It is concluded that future meta-analyses should first evaluate the quality of the studies on which the analyses are based and consider if there are enough appropriate studies to conduct the meta-analysis on.

Combining eye tracking and fMRI measures to disentangle the effects of single pieces of code on comprehension showed that descriptive identifiers activate natural-language areas more than nonsense identifiers. Kids in elementary school can already learn basic structures of programming with an offline Scratch Junior version. Mental health in academia is a huge problem and needs more attention. Can we improve on that to a Star-Trek like future?

The research investigates how students transfer conceptual knowledge between programming languages (Python and Java) during code comprehension. A Model of Programming Language Transfer for relative novice programmers that is based on code comprehension was developed. Through validating the model, the research concludes that similarities between programming languages play a significant role in semantic and conceptual transfer between programming languages. This research also shows how the model was used to shape the design of a transfer pedagogy in the classroom. It revealed how the pedagogy can lead to improved conceptual transfer and understanding.

Interactive ebooks are increasingly replacing traditional textbooks. There is evidence that they improve student satisfaction and learning and can be used to identify struggling students. There are commercial ebook platforms like Zybooks and several free and open-source ebook platforms including Runestone, Open DSA, and CS Circles. Runestone has over 30 ebooks for computing and math. These ebooks can contain text, images, videos, editable and runnable code, and many other types of practice problems including: Parsons problems, clickable code, and fill-in-the-blank problems. Runestone also includes a spaced practice tool. Instructors can create a custom course, create assignments, create timed exams, automatically grade assignments, and visualize student progress.

Peer Instruction is a pedagogical technique to improve learning in lecture through active and collaborative learning. In Peer Instruction the instructor may assign pre-reading with an assessment before or at the beginning of lecture and then during lecture stop every 10-15 minutes to display a hard question. Students answer individually, discuss their answer with peers, and then answer again. Instructors lead a discussion about the question. Peer Instruction has led to twice the learning gains over traditional lecture in Physics and has increased student engagement and understanding in many fields including computer science. We have been adding support for Peer Instruction to the Runestone ebook platform to make it easier to run Peer Instruction sessions, find peer instruction questions, test research questions, and improve peer instruction questions over time. The tool supports synchronous in-person and synchronous remote users (via a chat interface) and asynchronous users via a saved chat.

Programming problems generally admit multiple solutions, which can differ in choices of data structures, control structures, or the orders in which lower-level steps are performed. Research in other settings suggests that people develop more flexible design skills from seeing multiple solutions to the same problem. We suggest that subtasks can be a useful mechanism for helping students contrast different solutions to the same problem. We demonstrate this with problems that involve compositions of multiple list-manipulation tasks. We show how different mappings from each task to either being computed in a separate function or being tracked in a variable summarizes high-level differences in solution structures. These mappings highlight how different implementation choices lead to different solutions within the same task structure.

A higher-order function can take a function as an input parameter to another function. Common higher-order functions include filter, map, and sort (taking the sort criterion as an input). How do we teach students what HOFs are and how they come about? One approach is to show students two functions that perform the same core action (such as filtering), then abstract over the common code to define the filter function. We have been exploring a different approach based on teaching students the features of different higher-order function. We present students with a set of input/output pairs and ask them to classify or cluster the pairs into ones that could be produced by the same function. This is designed to help students recognize the features of higher-order list functions (such as relative lengths of input/output elements, relative types of inputs/outputs, whether all input elements are preserved). We hypothesize that learning these features might help students better understand what different higher-order functions do and when they might apply.

What’s the first data structure that we show students? Many courses use arrays, lists, or classes (as tuples of fields). We propose that 2D tables, akin to spreadsheets or CSV files, are a compelling first data structure for teaching computer science (whether to majors or non-majors). Tables are authentic and pervasive. Tables lend themselves to data from a wide range of domains, some of which raise challenges around socially-responsible computing. Programming languages that support tables provide operations such as sorting, filtering, and computing; these can be taught before showing manual iteration or recursion. This choice thus lets us focus on practical tasks and issues from early in a course. We hypothesize that this will help reach populations of students who care less about programming for programming’s sake, but without sacrificing depth of computing content.

When languages and tools are developed, there is an assumption of the process that learners go through to complete tasks. However, there is great variation in this process, and there are more and less effective processes. Therefore, it is imperative that we identify what those processes / steps are and develop strategies and scaffolds for those strategies so that more students can be successful in CT. For example, our strategy, TIPP&SEE, is useful for learning from example code. We saw statistically significantly stronger performance in students at academic risk in the treatment group, and their performance was comparable to students not at academic risk in the control group. The variance in TIPP&SEE classrooms was significantly smaller than the variance in control classrooms. There are many areas still ripe for benefiting from strategies, such as planning, decomposition, and debugging.

Learning trajectories are possible intermediate learning goals, dependencies between them, and activities that can build knowledge from one goal to another. The creation of the learning goals themselves and their dependencies is influenced by pedagogical approaches and education theory, such as constructivism, pieces of knowledge, and spiral conceptual ordering. Our learning trajectories start with everyday understandings of CS concepts, proceed to computational thinking ideas that apply to computation, and end with coding-specific goals.

Programmers often rely on online resources – such as code examples, documentations, blogs, and Q&A forums – to compare similar libraries and select the one most suitable for their own tasks and contexts. However, this comparison task is often done in an ad-hoc manner, which may result in suboptimal choices. Inspired by Analogical Learning and Variation Theory, we hypothesize that rendering many concept-annotated code examples from different libraries side-by-side can help programmers (1) develop a more comprehensive understanding of the libraries’ similarities and distinctions and (2) make more robust, appropriate library selections. We designed a novel interactive interface, ParaLib, and used it as a technical probe to explore to what extent many side-by-side concepted-annotated examples can facilitate the library comparison and selection process. A within-subjects user study with 20 programmers shows that when using ParaLib, participants made more consistent, suitable library selections and provided more comprehensive summaries of libraries’ similarities and differences.

Participatory design is a process of asking those using (or participating in) a program or piece of software to help with the design task. We are challenged to figure out how to set the stage so that participants and leaders have a shared understanding of what is to be designed, and then to have participation from the range of possible participants. I describe three examples of where I have used participatory design: In re-designing an on-line MS in CS program to involve more women, to design a data visualization tool for social studies teachers, and to design new computing education courses.

Apple introduced HyperCard with every Macintosh sold in 1987. At one point, it was likely the most used end-user programming environment in the world. It is unusual for its wordy, phrase-based grammar, and a tight integration with a set of GUI-building tools.

Teaching block-based languages is popular at the elementary school age, but from the middle school age (10 to 14) kids tend to want to learn textual languages, but Python is still quite tricky for them because its syntax can cause high cognitive load. Hedy aims to make the path to using Python easier, with a gradual approach, using different language levels, so learners do not have to learn all syntax rules at once. In level 1, there is hardly any syntax at all, for example, printing is done with: print Hello Dagstuhl!

In every level, new syntax and concepts are added, until kids are doing a subset of Python in level 18 with conditions, loops, variables and lists. In addition to the cognitive load of syntax, in this talk we also discuss other ways in which Hedy aims to reduce cognitive load, such as built-in lesson plans aiming at broad applications of programming.

Hedy was launched in 2020 and since its creation 2.5 million Hedy programs have been created by children worldwide. Try Hedy at www.hedy.org.

Most programming languages are built with English keywords, and often leaning on the assumption that users will only use latin characters. However, for non-English speakers, especially for non-Latin or right to left languages, using English programming languages can be a large barrier. Hedy (www.hedy.org) allows users to program in any natural language, for which a number of technical challenges needed to be addressed, including (but not limited to) allowing non-Latin variable names, calculating using non-Latin numerals, right-to-left parsing and rendering and using multilingual grammars. Open challenges remain, such as production rules that deviate from the traditional English structure, f.e. In Turkish, one would write the condition before a keyword (i==0 iken) rather than before it (while i==0).

Snap! is a programming language which supports the use of the Higher Order Functions MAP, KEEP (filter) and COMBINE (fold) and allows users to build their own HOFs.

Since Snap! is a blocks-based programming language, Higher Order Functions as well as lambdas are represented visually. They are ovally shaped like any other function but all contain a grey ring – the lambda – as one of their input slots.

The plain rings are part of Snap!’s Operators category and turn functions into data. To get to the return value again, they can be used as an input to the CALL block.

Functions in the rings have implicit parameters meaning all the input slots are left empty. When running the function, all empty input slots in the lambda function will be filled with the current data. In this example, the numbers from 1 to 10 are filled into both empty input slots in the + function.

map ( _ + _ ) over (numbers from 1 to 10)

In this talk, we showed how we represent lambda with the grey rings, how to recursively build MAP and then apply it to different forms of data like a table of Titanic passengers to find out the percentage of survivors per class or samples of a recording to create an Echo effect.

Learning programming languages is hard, partly because the semantic rules that govern how programs execute in a language are often invisible. We present PLTutor, an approach to providing granular visualizations of these rules, helping learners infer them through causal inference through a series of examples through forward and backward stepping through a program visualization, coupled with direct instruction about the rules. We evaluated learning gains among self-selected CS1 students using a block randomized lab study comparing PLTutor with Codecademy, a writing tutorial. In our small study, we find some evidence of improved learning gains on the SCS1, with average learning gains of PLTutor 60% higher than Codecademy (gain of 3.89 vs. 2.42 out of 27 questions). These gains strongly predicted midterms (R2=.64) only for PLTutor participants, whose grades showed less variation and no failures.

Teaching the world to code at scale is hard, in part due to the lack of access to learning materials and teachers. This talk describes Gidget – a free, online puzzle game designed to teach programming concepts by debugging faulty code – and major results from a selection of studies published over the past decade. In Gidget, learners help the eponymous robotic character complete its missions by modifying existing code that is nearly correct. The game also allows learners to run their code step-by-step to highlight how things change within the system at different levels of granularity. This encourages learners to inspect code structure and syntax at their own pace in a friendly environment. Additional features – such as frustration detection hints, embedded formative assessments, avoiding terms of violence (e.g., “remove” instead of “destroy”), choosing pro-social game motives (e.g., cleaning up a chemical spill), and automatically generated levels – have shown to be effective in greatly increasing novice programmers’ self-efficacy and attitudes towards coding, and that it leads to measurable learning gains of introductory programming (CS1) concepts.

Following an unwritten rule of sorts, that if something is important for a programming language it becomes (or should turn into) a first-class citizen *of* the language, programming language paradigms are often named after the “first-class citizens” they support, i.e. concepts that can be accessed as data. As “Object-Oriented” has first-class bundles of data-structured and behavior, “Functional” has reified and anonymous functions, an as of yet to be discovered “Block-Oriented” programming paradigm would feature blocks (syntax elements) as first-class citizens of the language. Snap! Now experimentally supports blocks that programmatically manipulate other blocks and can modify stacks of blocks at runtime.

Summary of questions we’re trying to discuss / answer:

• $\mathrm{■}$

  How do we support the planning process? You get a complex problem statement. How can you structure that thinking process before starting to code?

• $\mathrm{■}$

  How to teach the importance of planning? Let them peer-review each other’s designs/ programs/ documentation?

• $\mathrm{■}$

  What parts of the verbal question gives you cues as to what data type, what control structure, etc?

• $\mathrm{■}$

  How do we generate subgoal labels that people would agree on? Would it be easier to agree on functional programming labels?

• $\mathrm{■}$

  What are the scaffolds for the different parts of the process? Problem understanding phase. Algorithmic thinking to solve the problems. Implementation.

• $\mathrm{■}$

  What are the phases of meta-cognitive processes: pre-, during, post (evaluating)? Can these processes be measured?

• $\mathrm{■}$

  How can we relate the theories we’re talking about in this space to theories in HCI?

• $\mathrm{■}$

  How do we relate planning for program design / function design to planning for experience design or UI design or design thinking?

• $\mathrm{■}$

  How do we make planning a more valuable part of the planning process? Bidirectional with coding. Planning becomes much more valuable in group projects because there, you need to identify the tasks and split up the group equitably. If you teach planning processes as part of the debugging process, then they hit the wall and then do it.

• $\mathrm{■}$

  How can we create educational experiences so that students see the value of planning / documentation? A long-term course in which students need to keep reusing their code. At some point in the course, you inherit someone else’s code and documentation.

• $\mathrm{■}$

  Can we extract a plan from examples that are close and modify them? What can you extract out of your plan to find code that does pieces or something close. Kelleher has worked on creating examples, help students figure out what example is relevant, and help them modify it.

• $\mathrm{■}$

  What tool features should we have to support these educational interventions and make it so much easier to complete if you do the planning features we’ve provided?

  Potential features:

  • –

    Define input and output pairs, provide feedback on their accuracy

  • –

    Collaboration support – use planning as a communication mechanism for groups.

    Input / output types (and comments)

    Input / output pairs (for tasks and subtasks)

    Three categories: Complete, Incomplete, Incorrect

  • –

    Scaffolding:

    • *

      If they get it wrong how do we scaffold them?

    • *

      highlight text description, model how to translate from text description to input/output and types, what are the common mistakes in that process (given the input give them the output)

    • *

      Give them simpler inputs and have them give outputs

    • *

      Understanding what is/ went wrong (and led them self-correct) vs. leading them to the solution.

    • *

      Monitoring-scaffolds asking if what they are doing still aligns with their goals

    • *

      Interface that allows them to go back and forth between planning (design), execution (code) and evaluation (testing & debugging?) phases so that they can adapt the planning if need be.

    • *

      Detect plans from code and tests and give them a hint (e.g., that there is a misfit?) or can we detect that parts haven’t been debugged/ tested?

    • *

      Specify solutions (subtasks) and tests upfront and check if they appear in the student environment?

    • *

      Challenge of matching tasks and solutions – how to cope with different solutions – would ML help here?

    • *

      Monitoring if they got stuck: Finding/Distinguishing sink states (i.e., where did they get stuck) -

    • *

      How to scaffold the design of test cases – especially the boundary conditions, error conditions,

    • *

      Properties of valid / invalid inputs

    • *

      Properties of valid / invalid outputs

    Perhaps other criteria – like complexity or run time – compare with optimal solution
    Task breakdown and division of labor (enter student names, then they divide tasks here)
    Input / output pairs for the subtasks
    Define function names and comments that all can see
    If you have a bug, could we have a debugger that steps through only a subset of the code – only the code that is relevant to a specific subset of the code. Can we automatically identify which is the relevant or the irrelevant code? Then we could grey out working code and highlight code that has not been verified correct yet.
    Can we ask them what the intermediate values are?
    When are they wrong about the values?
    When is the code wrong about the values?
    “Stepper 2.0”, that skips the stuff that does not need to be checked :)

• $\mathrm{■}$

  Strategies
  What parts of the verbal question gives you cues as to what data type, what control structure, etc? What are the common mistakes?

Participants: Mike Lee, Kenichi Asai, Felienne Hermanns, Jadga Huegle, Neil Brown, Jens Mönig, Mark Guzdial

Participants: Kathi, Ethel, Michael, Tobias, Amy, Janet

Understanding the question

Planning

Program Specification / Clarification

Program decomposition

Choosing overall model / data structures

Translation – Data Model/Data Structure selection

Implementation

Testing

Debugging

Reflection

What would a data structure-inspired notational machine look like?

Highlight changes in data, not the control structure.

What variables should you print out when debugging?

Have a data-driven stepper – watch a subset of variables, and have the stepper highlight when each variable value changes. Updated watch variables done in a very friendly way.

Input-Output-Error does not mean the error is in the code, but maybe also on the conceptual level; can students identify relevant parts of source code? So move into a Parson’s problem like setting

How to make reliable subgoal labels/plans that actually help students?

Give an example, and then let students work on a similar example to support their understanding

Strategies of students have limits; and if one strategy leads to a dead end, what to do? Allow them to continue that strategy? Highlight a different strategy?

When students have bugs, they often do not know what the intermediate steps are

Have a debugger that works with a student: Ask after each step of execution the state (e.g., fill in numbers in a linked list).

Students do not break down the problem (the debugging problem, not the coding problem), just focusing on the too big a problem, e.g., failing a test case; they do not do the detective work during debugging

Could we generate a sequence of questions to ask them that are the questions they should ask themselves when they debug?

Barb’s grant:

Scaffold the TA-student interaction, so that TA can figure out what the misconceptions are;

Nested for loops vs iterating over two lists at the same time; many misconceptions, e.g., schema retrieval, workings of nested loops, incorrect model of data-program state; break down to first steps of loop?

How to detect student’s misconceptions in their plans? Break down of iteration step by step, then go one step back

Soloway said that they understand control structures individually but do not understand how to compose them.

There are different kinds of misconceptions at different programming stages (and you would need different supports)

How do we scaffold going from algorithm to code?

Parson’s problems of planning steps with vocabulary, e.g., order, clean, filter, aggregate

What kind of language would work for this process? Diana: data-driven approach to let students implement, then analyze and develop vocabulary that works for students

Experiment/study: Audience: starting students or upper level students? Does that make a difference? Also upper-level students have troubles planning

Ask students before and after implementation about their plans

Paper: Scaffolding design problems using Parson’s problems: https://dl.acm.org/doi/10.1145/3279720.3279746

Should students implement standard data structures? What is the take away for students, what should they learn? Rather using data structures?

• $\mathrm{■}$

  Whatever the source of funding, it’s key to align a project’s fundraising strategy with the incentives, constraints, and expectations of the funding climate. For some, that might mean particular kinds of scholarship to justify sustain funding, for others that might mean adoption numbers, and for others still that might demonstrate impact on education systems. Thinking carefully about these incentives is important, as they can often have novelty biases, rigor biases, and technology biases. Accounting for these biases, and ensuring they don’t end up warping the scholarship, requires explicit planning.

• $\mathrm{■}$

  Another consideration is how much funding is restricted toward particular expenses; obviously, having unrestricted funding is the most valuable, as it allows for a project to meet whatever needs come, rather than being constrained. It’s also the least common and hardest to obtain.

• $\mathrm{■}$

  There are almost always politics that influence how long money can be held and what it is spent on, whether it’s a corporate politics game or an academic or foundation politics game. It’s key to have someone who can manage those politics and relationships and ensure that they do not interfere with project goals in problematic ways. That could be a project lead, or a principal investigator, or even a funder or corporate partner who provides cover.

• $\mathrm{■}$

  There was a tangible sense of a continuum from deception to omission when it came to reporting and persuading funders. Everyone agreed that deception is unacceptable, but everyone also agreed that sometimes omission was necessary to amplify the outcomes that a funder might care most about, while hiding other details that might be in alignment with academic or innovation goals, but in tension with funder goals.

• $\mathrm{■}$

  It was quite common for successful projects to have one or two people with semi-permanent positions as the backbone of a project, from a maintenance perspective and from a resource perspective. This might be a corporate job or academic position. But it also means that projects have a single point of failure.

• $\mathrm{■}$

  Backend infrastructure can be a key risk to project sustainability. It requires regular maintenance, cloud costs, and staffing. Committing to back end infrastructure is a significant decision with long term consequences. Using university bandwidth and hosting is one way to avoid these costs, but is often restricted to static hosting, or has costs for university IT. But the quality of university IT service can be highly variable, and can also require some political negotiations to navigate policy restrictions.

• $\mathrm{■}$

  It’s important to consider other sources of revenue. Communities can be a source of revenue. User events can have registration fees and that can generate unrestricted funding. But where to store that money can be complicated and impose accounting challenges. Donations can also be a source of revenue. Sometimes people will just give and this can also be a source of funding, especially when requests are targeted towards those with philanthropic capacity. Some talked about offering nearly meaningless premium services that offer almost nothing extra, but allow corporations that are often reluctant to donate to pay for a service. This might not sustain the project, but it can help sustain it. All of this requires a place to keep money, which may or may not be the backbone organization.

• $\mathrm{■}$

  We also talked about other sources of staffing, such as ways of trying to onboard, supervise, and engage students to contribute, for credit or for modest pay. There are all kinds of challenges of doing this, including ensuring they have sufficient expertise, that there is onboarding for them, and that they get feedback through code reviews or other mechanisms. We also talked about open source contributions and the limited value of “drive by” contributors that don’t have enough context for the project to make meaningful contributions. Some talked about ways of engaging contributors socially first, to get context, and then have them contribute later after they have it.

Participants: Jens, Elena, Michael Lee, Kenichi, Jadga, Mark

Mark Proposed Study: Test transitions between our languages and tools. Pairs of us work on transitions from (for example) from Hedy to Pyret, or Snap to Hedy. (Requesting Ethel to be part of this!)-I love this Mark-Ethel.

This is Felienne: I love this too! Many of our Hedy classrooms do Scratch first! We even already have data of users of whether or not they have used Scratch before! We can totally do data analysis on behavioral differences between Hedy users with and without Scratch experience!

Of course, Mark is interested in ANYBODY transitioning from Teaspoon to ANY PL! I’m so interested to see how we support that transition.

Ethel-i think this is the right time to explore transitions, i would encourage people to participate in this and understand transfer at a deeper level. Also mapping constructs across different programming languages is something I’m looking forward to. The mappings can be put in a public domain later on if possible ->Love this idea!!

Teacher Transitions: Examine the transition/process of teachers’ learning and choice of second language. Can I suggest you read Ethel’s paper about this :) ->https://dial.uclouvain.be/pr/boreal/object/boreal:251251

• $\mathrm{■}$

  Ethel’s paper is great <3 In our breakout, we also talked about the challenge of moving curriculum, learning content. Do teachers focus on making their content better, or transitioning to new languages and versions?

• $\mathrm{■}$

  Thanks Mark! I was trying to remember how we phrased it in our breakout. – Mike

Kathi wants to explore program representations/notional machines that center around data transformations, partly to help students create and execute debugging plans. What would such a representation look like? What differences might arise based on the semantics of the programming language? Would a data-centered approach help connect data to plans to programs? Johan: Don’t you want more than debugging plans: also the kind of planning you discussed in your talk. Kathi: yes, plans as well. I’m wondering whether data-centered plans would also give debugging guidance, which would again tie to notional machines.

Johan proposed study: (Should probably be combined with others.) Different approaches to how we can support planning. Using Mio, Shriram’s, and Kathi’s approach to planning, add a testing based approach to it (as discussed in the break-out session this morning), and study how it works. Ben likes this idea. See related idea below. Youyou also likes this idea. Kathi is interested. Diana

Diana Proposed study: Cog Sci & strategies people pair up to better understand the processes involved in different steps of programming (program clarification, planning / decomposition, algorithm development, coding, debugging) and propose new strategies. Eva is keen to join :) Kathi is interested.

Ben: Program annotation, planning, and debugging – If students are writing interleaved (i.e. not very decomposed) programs, can they annotate pieces of their code to indicate which plans/goals pieces of their code are meant to accomplish? A visual representation of this could also be contrasted with a visualization of data flow within their programs (e.g. to indicate that some information (e.g. input from a test case) is not making it into a vital part of computing a correct result). This could be used to build more focused steppers or to help students to figure out where to focus when dealing with incorrect behavior. Johan: can we for example connect annotations/subgoal labels to test cases? Or ask for test cases at these annotations? Can we grey out parts of the program that are not part of a subgoal in a stepper/debugger? +Barb

Bastiaan: nice idea! Kathi is interested.

Diana: How can we provide real-time hints / scaffolding to help students design “good” test cases (black-box tests) for the purposes of both design and testing? +Barb

Kathi is also interested in this, noting that tests characterize a space of inputs/scenarios not just individual scenarios. I see this as a form of sensemaking.

Diana: Can we analyze problem statements and develop a process for identifying keywords or aspects of the problem statement that point towards specific aspects of the design and/or implementation? +Barb: This might also help students write good purpose statements.

Barb: Can we all study the same problem – like Kathi’s shopping cart problem or the rainfall problem in a variety of contexts with a variety of scaffolding? +Johan: I think taking a particular example to discuss our ideas will be very fruitful +Tobias +Diana

• $\mathrm{■}$

  MIMN? Multi-institutional, multi-national?

• $\mathrm{■}$

  ITiCSE working group: “Everybody teach X in any way you want. We all use the same measure of learning/understanding at the end.”

Kenichi: Programming language teachers in CS departments do not necessarily know how their ways of teaching are effective or not. It would be great if education experts can see the class (although it takes quite a long time) and make advice on how to improve the class and even how to conduct experiments. Conversely, CS people could hear what teachers want to teach and advise what is feasible technically.

Diana: Not quite a study, but it seems like a Computing Research Infrastructure NSF (USA) grant could fund the development of an open-source framework for testing many innovations (in one particular language) so that research could be pushed forward. This could help many people.

Amy proposal: I’d love to see us try 5-10 radically different approaches to educational programming languages than exist now, both to explore the design space more broadly through the lens of cultural responsiveness, but also to understand more deeply the role of syntax, semantics, and representation on learning, teaching, curriculum. (What would the world look like if we had 10 different mature Scratch/Snap etc platforms, but doing things very differently for very different audiences). Anyone want to write a $25 million 5 year grant? Kathi would be interested in brainstorming around or discussing this.

Ben would as well …depending upon how radical we can be :-)