One-Shot Learning in Hybrid System Identification: A New Modular Paradigm

Hybrid systems combine continuous dynamics with discrete transitions, representing complex behaviors that are common in many real-world systems.

The system under learning is observed through a finite collection of execution traces $𝒪=\{{𝒪}_1,{𝒪}_2,\mathrm{\dots },{𝒪}_m\}$. Each trace ${𝒪}_j$ consists of a time series of sampled variable vectors from $X$. Formally, ${𝒪}_j[i]=[x_1,x_2,\mathrm{\dots },x_n]$ denotes the $i$th sample in trace ${𝒪}_j$, where $j=1,\mathrm{\dots },m$ and $i=1,\mathrm{\dots },k_j$ with $k_j$ being the length of the $j$th trace.

The goal is to identify a hybrid system $\mathrm{\Gamma }$ that explains the observed traces, capturing both the continuous dynamics and the discrete mode transitions of the underlying system behavior.

The learning process for hybrid system identification starts with a set of observed traces. Additionally, a domain $\mathrm{\Phi }$ of candidate flow functions is given at the beginning of the learning process. The set of candidate flow functions $\mathrm{\Phi }$ contains all flow functions that can be used to model the continuous dynamics of the system and is, e.g., defined by the application. The goal is to identify a hybrid system $\mathrm{\Gamma }$ that explains the observed traces using a minimal number of flow functions. As illustrated in Figure 3, the task is framed as a multi-objective optimization problem: minimizing the model complexity (e.g., the number of modes or flow functions) while maximizing the accuracy of the learned system on the observed data. We define a required accuracy by a threshold $ϵ$ for the deviation between the observed traces and the identified hybrid model. This reduces the learning problem to a minimization of the number of flow functions under this accuracy constraint:

where $E$ denotes an error function that evaluates the deviation between the behavior found from the flow functions $ℱ$ on the segments $𝒮$ composing a segmentation from the set $𝒫(𝒪)$ of all possible segmentations of the observed traces $𝒪$. In this formalization, a minimal exact solution, i.e., $ϵ=0$ exists only if the observed traces are free of noise and the domain of candidate flow functions $\mathrm{\Phi }$ covers the original system dynamics.

The traditional approach to learning hybrid systems uses a sequential procedure as shown in Figure 2:

1. 1.

   Transition detection: Identify potential transition points in each trace.

2. 2.

   Segment grouping: Cluster segments between transitions into groups that are assumed to represent the same mode.

3. 3.

   Flow function identification: Learn one flow function per segment group.

4. 4.

   Model construction: Construct the hybrid system by combining the learned functions and define transitions.

In this approach, the individual steps are independent of each other and can be formulated as follows. First, the set $𝐭$ of transition points is identified as

where $\phi$ is a predicate for transition detection. Every transition point is represented by a tuple $(j,i)$ with $j$ identifying the trace $O_j$ and $i$ giving an index $i$ in this trace. Using the transition points, all traces are cut into segments $𝒮$:

where $𝐭$ is sorted such that . $𝒮$ is one possible segmentation of the traces, i.e., $S\in 𝒫(𝒪)$, based on the identified transition points. Segments form a group $g$, if they are sufficiently similar, i.e.,

where $\sigma$ is a similarity function used to compare segments and $\tau$ is a similarity threshold. Note that similar dynamics do not necessarily create similar traces, which is one issue that our approach addresses. Every segment is assigned to a single group only, i.e., $g_i\cap g_j=\mathrm{\varnothing }\forall i\ne j$. The groups form the set

such that every group contains the learning data for one flow function. The set of flow functions $ℱ$ is found as follows:

where $e(f,g)$ is the approximation error of function $f$ on group $g$, e.g., the mean squared error between the output of the flow function and the observed output on the learning data.

The process defined by Equation 2 to Equation 6 is problematic as the identification of flow functions is postponed to the last step. Segments and groups form intermediate learning results that are not informed by the final goal of identifying accurate and compact flow functions, which is not inline with the original learning problem in Equation 1. This separation of concerns introduces strong interdependencies: transition points are only meaningful in light of changes in dynamics and accurate flow functions can only be learned from meaningful segments.

To address these limitations, we propose a new learning framework that integrates flow function identification into the segmentation and grouping process. Rather than treating the components of the hybrid system as separate inference steps, our method, as shown in Figure 1, alternates between the identification of flow functions (A) and the detection of segments (B) using partial model information to guide the refinement of each component. This iterative approach directly targets the optimization problem in Equation 1 by prioritizing minimality: new modes are introduced only when the existing set of flow functions cannot explain the data with sufficient accuracy. This design aligns the learning process with the goal of minimality in model structure. Our approach does not guarantee finding an optimal solution for Equation 1. Instead, we optimize the length of individual segments under the constraint that a single flow function represents the complete segment. With this procedure, the number of segments needed to cover all traces is reduced. Finally, a smaller number of segments facilitates a smaller number of flow functions.

Currently, our framework focuses on the identification of flow functions and their associated modes. The inference of transition conditions is treated as a separate post-processing step, to be learned independently once the mode structure is established.

Our iterative learning approach is described as follows:

• A   Identification of Flow Function: Find a segment of maximal length starting from a first trace such that a flow function exists, that achieves a sufficient accuracy on this segment. Identify the flow function for the found segment.

• B   Detection of Accurate Segments: Assess the accuracy of the identified flow function on all observed traces.

  • If accuracy sufficient: Stop, and finalize the modes.

  • If accuracy insufficient:

    • C   Refinement of Flow Function: Refine the flow function on segments with sufficient accuracy and keep this flow function for the mode.

    • Back to A   Continue with the remaining subtraces, returning to Step A.

At each iteration on the set of traces, a single flow function is learned and then used to identify segments where the flow function is valid, i.e., achieves sufficient accuracy and subtraces where the accuracy of the flow function is insufficient. The data for all segments with sufficient accuracy is used to refine the flow function for this mode. The process repeats on the remaining data until all segments have been assigned to a mode.

In relation to Figure 1, we formulate one iteration of the approach. The identification of a flow function (A) is given as

where $e(f,{𝒪}_j[0,l^{\ast }])$ is the approximation error which is determined for the function $\tilde{f}$ on the segment ${𝒪}_j[0,l^{\ast }]$ of length $l^{\ast }$ of an arbitrary trace ${𝒪}_j\in 𝒪$. The length $l^{\ast }$ is maximized in an iterative manner together with the identification of the flow function $f$:

where given a flow function $f^{(k-1)}$ from the previous iteration, the length $l^{(k)}$ is determined as the maximum length of a segment where the flow function $f^{(k-1)}$ achieves sufficient accuracy. The flow function $f^{(k)}$ is then learned on this segment of length $l^{(k)}$. The parameter $l_{\text{init}}$ is the initial length of the segment, and $k_j$ is the length of the trace ${𝒪}_j$. The segment length $l^{\ast }$ is the fixed point of Equation 8 and Equation 9, i.e., where $l^{(k)}=l^{(k+1)}$. This is the maximum length of a segment that achieves a sufficient accuracy on the trace ${𝒪}_j$.

The detection of accurate segments (B) finds ${𝒮}_A$ as a set of segments from the traces in $𝒪$ where the function $\tilde{f}$ achieves sufficient accuracy:

Finally, we use ${𝒮}_A$ to refine a final flow function $f$ (C) which represents a new dynamic for the segments of ${𝒮}_A$:

The next iteration of the algorithm starts on an updated set of observations

In this set, all subtraces of the traces in $𝒪$ where the flow function $f$ does not achieve sufficient accuracy, form input traces for the next iteration. These are the subtraces that are not included in the set of segments ${𝒮}_A$.

The iterative process continues until all traces in $𝒪$ have been covered, i.e., $𝒪\equiv \mathrm{\varnothing }$.

Compared to the traditional approach (Equation 2–Equation 6), this formulation eliminates the explicit representation of transition points and segment groups. Instead, transitions emerge naturally from the explanatory limits of each flow function on the data. Mode identification and segmentation are no longer treated as independent pre-processing steps but are tightly coupled through accuracy-driven inference. The condition for the detection of a transition point (formulated as $\phi$ in the traditional approach) is based directly on the accuracy of the identified flow function. Consequently, this procedure addresses the original problem in Equation 1 more directly than the traditional approach.

We concretize the learning process in Algorithm 1, which describes the steps of the learning process.

The algorithm begins with an arbitrary trajectory from $𝒪$. For the sake of reproducibility, we choose the first trace ${𝒪}_0$ in Line 2. A belief flow function $\tilde{f}$ is learned over a growing time window starting at the beginning of ${𝒪}_0$ with an initial length of $l_{init}$ in the loop from Line 4 to Line 8. The window is expanded in increments of size $l_{step}$ until the approximation error exceeds the threshold $ϵ$. The resulting function $\tilde{f}$ is assumed to characterize a new dynamic. All trace segments where $\tilde{f}$ achieves an error below $ϵ$ are considered additional instances of this dynamic and are used to find a final $f$ for the found mode. This set ${𝒮}_A$ of segments is determined by the function getAccurateSegments in Line 9. The set ${𝒮}_A$ is then used to refine the flow function in Line 10, which identifies the new dynamic. Afterward, all identified segments are removed from the traces in Line 12 which reduces existing traces or splits traces into multiple new traces. Figure 4 visualizes the effect of the function removeAccurateSegments. Starting from the traces ${𝒪}_1$, ${𝒪}_2$, and ${𝒪}_3$, the segments where the error is smaller than $ϵ$ are removed. These segments are covered by the flow function $f$. The new set of traces is $𝒪=\{{𝒪}_1^{\prime },{𝒪}_1^{\prime \prime },{𝒪}_2^{\prime },{𝒪}_2^{\prime \prime },{𝒪}_3^{\prime }\}$. The process repeats until all observed data have been covered, i.e., $𝒪\equiv \mathrm{\varnothing }$.

The learning approach presented in this paper is modular and allows the integration of different regression methods for the identification of flow functions. We consider three methods during evaluation of our learning approach:

• $\mathrm{■}$

  Linear regression (LR) for learning simple, interpretable linear flow functions.

• $\mathrm{■}$

  Symbolic regression (SR) to identify more expressive symbolic representations.

• $\mathrm{■}$

  Linear matrix (LM) differential equations for systems with multidimensional state spaces.

For symbolic and linear regression, we restrict our evaluation to hybrid systems with a single output variable that corresponds directly to the state variable, i.e., $O=S$ and $|O|=1$. The third regression method, i.e., linear matrix differential equations, applies to a multidimensional state.

We illustrate our learning approach with two representative examples: a piecewise linear function and a bouncing ball system. The first example serves as a simple baseline that allows a straightforward identification of the flow functions. The second example, the bouncing ball, is a classical benchmark in system identification. Although the bouncing ball exhibits discontinuities in velocity, it can be modeled using a single flow function. In addition, we demonstrate the applicability of different learning methods for both examples and compare identified flow functions with the original system dynamics.

We apply our learning approach with linear regression to both the piecewise-linear function and the bouncing ball system. As shown in Table 2, three distinct flow functions are identified for the piecewise linear function, each corresponding to one mode. The learned functions match the true system except for a minor difference in the first flow function. The maximum length of the found segments is $l^{\ast }=33$, which is equivalent to the sampling step of the ground truth transition point.

Similar results are obtained for the bouncing ball. Here, the error is slightly larger, but also a larger error threshold $ϵ$ is used. The learned flow function differs in the slope and offset from the original function. The linear function based on the heights $x_1$ and $x_2$ of the last and second-last sampling step is learned, respectively. This is an approximation of the actual differential equation of the bouncing ball. Due to the high sampling rate, the approximation is very close to the actual differential equation. This is also shown in Figure 8. Figure 8(a) shows the learned flow function compared to the ground truth. The learned flow function is close to the ground truth. Figure 8(b) shows the absolute error of the learned flow function compared to the ground truth. The error is small, but varies over time due to the mismatch of the slope and offset of the learned flow function compared to the ground truth. Larger errors occur at the transition points, where the ball bounces on the ground and the simulation resolution is not sufficient to capture the dynamics of the bouncing ball.

Further, the variant of our approach with linear regression has a very low learning time.

Symbolic regression offers greater flexibility compared to linear regression, enabling the identification of a wider class of functions depending on the available operators and basis functions. For both the piecewise-linear function and the bouncing ball, we use addition, subtraction, and multiplication as primitives. In addition, symbolic regression identifies constants for the learned expressions.

The results of the model learning are shown in Table 2. For the piecewise-linear function, we learn three flow functions, which are the three modes of the piecewise-linear function. The first two flow functions are equivalent to the original function. However, instead of $1$ for the second flow function, $0.5+0.5$ is learned. Symbolic regression does not simplify expressions internally, as this could hinder exploration of the search space in the genetic algorithm. We simplify the expression after learning, which results in the following set of flow functions:

The slope and offset of the third flow function deviate only in the range ${10}^{-8}$ from the original function. Again, the maximum length of the found segments is $l^{\ast }=33$, which is equivalent to the ground truth transition point.

Similar results are obtained for the bouncing ball. The coefficients for the features $x_1$ and $x_2$ are learned correctly, although $x_1+x_1$ is learned instead of $2\cdot x_1$. Compared to the original function, the constant offset is not learned. However, the mean squared error is close to the error of the linear regression. Figure 8(a) shows the learned flow function compared to the ground truth. The learned flow function is close to the ground truth. Figure 8(b) shows the absolute error of the learned flow function compared to the ground truth. The error is small for both linear regression and symbolic regression, and the error of linear regression is on average smaller than the error of symbolic regression. The error with symbolic regression is constant because only the constant offset is not learned correctly. Larger errors occur at the transition points, where the ball bounces on the ground and the simulation resolution is not sufficient to capture the dynamics of the bouncing ball.

Overall, linear regression achieves lower errors and significantly faster learning time than symbolic regression. This is due to the larger search space $\mathrm{\Phi }$ of symbolic regression and its dependence on heuristic optimization (here, genetic programming), in contrast to the closed-form solution used in linear regression.

We perform our learning approach with linear matrix regression for the bouncing ball. The goal is to learn the two-dimensional state space of the ball as given in Equation 18. The results are shown in Table 2 and visualized in Figure 8(a) and Figure 8(b).

The first line of the matrix differential equation is learned correctly except for the constant offset. The second line of the matrix differential equation differs from the original function. Instead of a constant reduction of the velocity, a factor smaller than one is learned for the previous velocity.

The MSE of the learned matrix differential equation for the height of the ball is higher compared to the results of one-dimensional linear regression and symbolic regression. The learning problem here is more complex as a two-dimensional state is learned. This regression method enables hybrid system identification for a higher-dimensional state space. The learning time for linear matrix regression is small.

Finally, we compare our learning approach with traditional learning of hybrid automata. We use the bouncing ball example and learn a hybrid automaton with the FaMoS tool [14]. The results are shown in Table 3.

The learning time of FaMoS is low and the MSE is small. FaMoS achieves a slightly lower MSE than all variants of our approach, which is due to the fact that FaMoS normalizes the data before learning and, thus, the MSE is evaluated on normalized data. The prediction of the state space with the learned flow function and the ground truth is shown in Figure 8(a) as a dotted green line and is very close to the ground truth. Figure 8(b) shows the denormalized absolute error of the learned flow function compared to the ground truth. The error is very similar to the error of our learning approach with linear matrix regression. However, instead of a single flow function, three flow functions are learned with FaMoS. All three flow functions are similar and are close to the ground truth, but not exactly equivalent to the original functions. This example shows that our approach can achieve a smaller number of flow functions and, thus, a smaller model. The traditional approach executed by FaMoS detects transition points at the bouncing points of the ball, i.e., whenever $x\equiv 0$. For the given learning data, transition points are detected at the sampling steps 361 and 695. The flow functions are learned on the segments individually. The grouping step in FaMoS is not able to group the segments correctly, as they are not similar when comparing the curves of the segments. Thus, FaMoS fails to recognize identical dynamics in the segments. Our learning approach instead groups segments based on the flow functions and, thus, needs only a single flow function to represent the dynamics of the bouncing ball.