Automating Control System Design: Using Language Models for Expert Knowledge in Decentralized Controller Auto-Tuning

When discussing automated controller design, most works focus on controller auto-tuning, which is popularly framed as a derivative-free optimization problem [16][17]. While gradient-based methods are also used for controller auto-tuning [13], derivative-free formulations have the desired advantage of making little or no assumptions on objective or constraint functions.

Multiple authors have proposed metaheuristic approaches to controller auto-tuning, with [16] doing a detailed survey on evolutionary optimization algorithms for multiple-input-multiple-output processes. However, Bayesian optimization (BO) approaches have become quite popular in recent years, due to their high sample efficiency [2] and overall performance. For example [17] present a joint tuning methodology that uses BO to simultaneously tune an LQR controller, an unscented Kalman Filter (UKF), and a guidance and navigation system for an autonomous underwater vehicle (UUV), yielding satisfactory results across multiple objectives. Most of the available solutions focus strictly on the parameter search itself and how to solve it using some optimization tool, leaving the design of the optimization problem, or the control algorithm, as an expert knowledge requirement.

In their work, [2] also propose a Bayesian optimization solution for controller auto-tuning of PID controllers in an underwater vehicle, improving the computational complexity of the controller tuning task by decomposing the search space into smaller dimension subspaces. In order to do so, they propose a multistage framework that aims to formalize all the necessary steps in automated controller design. This formalization highlights a variety of existing challenges in automated controller design that are often ignored, or underexplored in recent works. Examples of these challenges are how to define the parameter search space, the constraints, or the cost function for the optimization problem, issues that are often not discussed, or attributed to expert knowledge. Similarly, how to define the stages in the multistage framework defined in [2] is still one of the open problems highlighted in the paper.

An attempt to address one of these expert knowledge requirements can be found in [12], where an optimal control formulation is proposed in order to simultaneously optimize the controller parameters and produce an I/O pairing for the system. I/O pairing, or control loop design, is often assumed to be already defined in most controller auto-tuning works [17], when, in reality, it is usually performed through expert knowledge.

Overall, state-of-the-art research on automated controller design is significantly challenged by expert knowledge requirements across different stages of the problem.

Several papers have used LLMs for some aspect of system design, but fundamentally most approaches still adopt significant manual modeling, employing LLMs after the design process.

[14] uses a five-step LLM-based prompting approach that requires as inputs a piping and instrumentation diagram (P&ID) and natural language prompts. They evaluate the approach on a one-tank, one three-tank, and one four-tank system. Our approach differs in that we focus on control tuning and not diagnostics, we use a formal language for LLM prompting that provides guarantees about optimality of controller design, and we focus on control-model optimization and not just model generation.

A closely related approach, SmartControl [19], automatically determines the most suitable PID parameters to achieve the specified performance targets using PSO and DE algorithms. This process includes selecting the optimal gains that will ensure the system’s fast and stable response. After optimization, the closed-loop step response is simulated and basic performance metrics (such as settling time, rise time, overshoot, etc.) are calculated and presented to the user via an interactive graphical interface. SmartControl emphasizes interactive design, allowing users to engage directly with the LLM agent to refine the controller design. The approach in the provided paper is more automated; the LLM generates the I/O pairing and stage definitions without direct user interaction during that phase.

ControlAgent [7] employs a range of techniques for integrating domain knowledge, e.g., a knowledge graph or other structured representation of control engineering principles. Our approach aims for full automation once the initial system description is provided; in contrast, ControlAgent involves more interactive elements, allowing users to refine the design process through feedback or iterative refinement with the LLM.

Several frameworks exist to define systems based on interconnected component models, e.g., bond graphs, Simulink, and Modelica. The proposed approach is a fundamentally different framework from such approaches, most of which rely on manually-defined component models.

The approach is most similar to bond graphs (which focuses on representing energy flow and causality), although the current usage is quite different. In a bond graph, nodes consist of a standardized set of elements (e.g., C for capacitance, I for inertia, etc.) and edges/junctions represent system components and their interactions. The resulting graph allows for deriving system equations and performing simulations. Our proposed graph model is simpler than a bond graph model, as it does not specify the inherent physical properties of the nodes.

The two approaches are not mutually exclusive. One could conceivably use bond graphs to model a multi-tank system and then use the resulting model (or a simplified representation of it) as input to the LLM-based system for automated control design. The bond graph would provide a structured representation of the system’s dynamics, while the LLM would handle the more complex task of selecting appropriate control strategies and parameters. The paper, however, focuses solely on the LLM-based approach without explicit integration of bond graphs. The paper’s graph representation is a simplified topological representation, not a full bond graph model.

Let’s define a controller by a tuple $C=(U,𝒴,𝒰)$ where $𝒴\subset {ℝ}^{n_y}$ is the space of system measurements/outputs (e.g.: level in a tank, altitude in a drone), $𝒰\subset {ℝ}^{n_u}$ is the space of system inputs (e.g. liquid flow from a pump, propeller commands in a quadcopter), and $U:{𝒴}^t\times {𝒰}^{t-1}\to 𝒰$ is a control algorithm that produces a system input at every timestamp $t$ considering system inputs and outputs from previous system interactions. We are interested in a family of control frameworks called decentralized parametric controllers, which are commonly used in industrial and robotic systems [2].

A parametric control algorithm is a parametric function $U:{𝒴}^t\times {𝒰}^{t-1}\times \mathrm{\Xi }\to 𝒰$ such that $𝐮(t)=U(𝐲(0),\mathrm{\dots },𝐲(t),𝐮(0),\mathrm{\dots },𝐮(t-1),𝝃)$, with $𝐲(t)\in 𝒴,𝐮(t)\in 𝒰,𝝃\in \mathrm{\Xi }$, where the control algorithm is defined by a set of parameters $𝝃$ that must be previously adjusted (tuned). Parametric controllers are very common, with the proportional-integral-derivative controller and its P, I, and D parameters being the industry standard [16].

A decentralized control approach is defined by a set of independent control algorithms $𝒞=\{U_1,U_2,\mathrm{\dots },U_{n_u}\}$ where every control algorithm $U_i$ matches a single system input $u_i$ with a single system output $y_i$, and the changes in that system input are only conditioned by the corresponding system output, i.e.: $u_i(t)=U(y_i(0),\mathrm{\dots },y_i(t),u_i(0),\mathrm{\dots },u_i(t-1),{𝝃}_i)$. This is a common simplification in control system design [2] with assumptions regarding system coupling. The implications of these assumptions are discussed further in this paper.

A graph is defined by a tuple $G=(V,E)$ where $v_i\in V$ is a set of elements called vertices $v_i\in \mathbb{N}$ and $e\in E$ is a set of pairs $e=(v_i,v_j);v_i,v_j\in V$ called edges [6]. An edge describes a relationship between two vertices, we can say that an edge joins two vertices. For the purpose of this work, we are interested in graphs with the following properties:

• $\mathrm{■}$

  Directed: In a directed graph, the edges are ordered pairs and have an implied direction, i.e.: $e_1=(v_1,v_2)\ne e_2=(v_2,v_1)$. An edge in a directed graph is called a directed edge.

• $\mathrm{■}$

  Simple: A simple graph is a graph without loops. A loop in a graph is defined as an edge that joins a vertex to itself, i.e.: $e=(v_i,v_j);v_i=v_j$.

• $\mathrm{■}$

  Edge-Labeled: In an edge-labeled graph $G=(V,E,L)$, a labeling function $l:E\to L$ is defined to assign a label from the label set $L$ to each edge in the edge set.

Let’s discuss some graph theory concepts that will be relevant in the rest of this work [4].

Strongly connected components in a graph can be computed in linear time by a variety of algorithms (e.g.: depth-first-search algorithms) [4].

Let us define the prompt space $𝒵$ [11]. The elements in $𝒵$ are a composition of tokens selected from a token vocabulary $t\in 𝒯$, where $z=\{t_1,t_2,\mathrm{\dots },t_m\}$ can be a sequence of tokens of any length $m\in \mathbb{N}$. An LLM is then defined as a transformation $LLM:𝒵\to 𝒵,z_o=LLM(z_i)$ where, for a given input prompt $z_i$, the model produces an output prompt $z_o$.

A common feature in LLM APIs is the system prompt. A system prompt $z_s\in 𝒵$ permeates every interaction [20] with the LLM, providing context, setting the role and the tone for the LLM responses, ensuring output prompt formatting, etc. Particular implementations of the system prompt vary depending on the LLM; in some cases it is provided as an initial prompt before the input prompt [11], while in other cases a dedicated system token is implemented. For the purpose of this work, we are interested in LLMs with system prompt ($z_s$) options, i.e.: $LLM:𝒵\times 𝒵\to 𝒵,z_o=LLM(z_s,z_i)$. In Section 4.4, we define bindings from a graph language describing a family of physical systems to the input and system prompt spaces.

Bayesian optimization is a technique for globally optimizing black-box functions that are expensive to evaluate [9]. We consider the global minimization problem: ${𝝃}^{\ast }=\underset{𝝃\in \mathrm{\Xi }}{\arg \min }{f}_{\xi }(𝝃),$ with input space $\mathrm{\Xi }$ and objective function $f_{\xi }:\mathrm{\Xi }\to ℝ$. We consider costly-to-evaluate functions $f_{\xi }$ for which we have a limited number of evaluations available before we propose an optimum ${𝝃}^{\ast }$ in at most $J_{end}$ iterations. We assume we have access only to noisy evaluations of the objective $l=f_{\xi }+\epsilon$, where $\epsilon \sim 𝒩(0,{\sigma }_n^2)$ is i.i.d. Gaussian noise with variance ${\sigma }_n^2$. Finally, we make no assumptions regarding gradients or convexity properties of $f_{\xi }$.

The main steps of a BO routine in iteration $j$ involve (1) response surface learning, (2) optimal input selection ${𝝃}_{j+1}$, and (3) evaluation of the objective function $f_{\xi }$ at ${𝝃}_{j+1}$.

The BO framework uses the predictive mean and variance of a Gaussian Process (GP) [15] model to formulate an acquisition function that trades off exploitation, testing promising controller parameters given the current knowledge, with exploration, sampling unexplored regions of the design space. The BO algorithm can be initiated without any past observations, but it is usually more efficient to calibrate the GP model hyper-parameters and calculate a prior by first collecting an initial set of observations.

We define the controller tuning task as a multi-objective optimization (MOO) problem:

where $𝝃\in {\mathrm{\Re }}^{n_{\xi }}$ are controller parameters and $𝐋(𝝃)$ are objectives. We consider an MOO problem for a more general definition; however, controller auto-tuning is commonly framed as a scalar objective optimization problem. Our proposed framework is fully compatible with both the multi-objective and scalar-objective definitions of the problem. We assume a system with decentralized parametric controllers; other than that, the proposed framework makes no assumptions on the parameter and objective spaces. This general tuning problem can be redefined as a multistage optimization problem where a subset of controllers is tuned at every stage. Each stage can be defined as an MOO sub-task. Fig. 1 illustrates the process.

At stage $i=1,2,\mathrm{\dots },n_g$ of the tuning process, a subset of control parameters (${\mathrm{\Xi }}^i\subseteq \mathrm{\Xi }$) are tuned to minimize some stage-specific objectives (${𝐋}^i$), given reference signal $R^i$. The rest of the control parameters are fixed at a previous stage, or before the tuning process begins, through the constraint parameters $({\overline{𝝃}}^i,P^i)$. At each stage, a subset (${𝝃}_{min}^i$) of the optimal control parameters (${𝝃}_{min}$) is produced as a result of an MOO sub-task described below.

The I/O pairing and the definition of the stages are task-specific steps, both of which are commonly solved using expert knowledge [2][16]. The main contribution of this work is a methodology to automate this expert knowledge requirement and produce an I/O pairing and stage definition using LLMs. Further details on these steps can be found in Section 4.3.

The controller tuning optimization task is decomposed into sub-tasks defined as:

where (2c) is a constraint that sets control parameters to specified values (${\overline{𝝃}}^i$) from a previous stage or from before the tuning process begins.

These optimization sub-tasks can be solved using any optimization algorithm, as long as the cost function and search space are properly defined. For the purpose of this work, we use Bayesian Optimization to solve the optimization sub-tasks; we refer the reader to [2] for a more detailed discussion on the optimization sub-tasks and the use of Bayesian Optimization.

We now define a language, in the form of an enhanced system graph, to describe physical systems in terms of reservoirs, actuators, and connections:

Let’s first discuss the tank subgraph $G^T$ which describes relations between the different tanks. Each vertex in the set $V^T=\{1,2,\mathrm{\dots }{N}_T\},v_i^T\in \mathbb{N}$, represents a tank in the system, where tank labels (vertices) are assigned arbitrarily, and $N_T$ is the number of tanks in the system. The edge set $E^T\subset V^T\times V^T,e_k^T=(v_i^T,v_j^T)$ represents a liquid flow connection between the tanks. The direction of the edges represents possible flow directions. A regular directed edge $(v_i^T,v_j^T)$ represents liquid flow that is only feasible from tank $v_i^T$ to tank $v_j^T$, for example, the drain of a gravity-drained tank that feeds a tank below it. A bidirectional edge $\{(v_i^T,v_j^T),(v_j^T,v_i^T)\}$ represents an interconnection between tanks where liquid flow can occur in both directions, for example, two horizontally aligned tanks connected by a pipe.

Based on the edge directionality and the semantics of edges, we can assign a labeling to edges for constructing LLM prompts. The edge-label set $L^T=\{above,below,interconnected\}$ provides the necessary information about the relationships between the tanks for prompt engineering. The labeling function $l^T(e)$ is defined for any edge $e\in E^T,e=(v_i,v_j)$:

where an edge is labeled $interconnected$ if it is bidirectional (i.e.: the opposite direction edge is also in the graph) and $above$ or $below$ if the edge is not bidirectional (i.e.: the opposite direction edge is not in the graph). An edge $e=(v_i,v_j)$ is labeled $above$ if vertex $v_i$ is of lower numerical value than vertex $v_j$ and $below$ if the opposite is true. Note that the value of the vertices, which effectively act as labels for the tanks, are assigned arbitrarily, and the purpose of this labeling strategy is to ensure a consistent distinction between types of edges, which becomes relevant when building natural language prompts for the LLM. Therefore, the distinction between $above$ and $below$ edges is done without loss of generality.

We then define the actuator subgraph $G^A=(V^A,E^A,L^A)$ where each vertex in $V^A=\{1,2,\mathrm{\dots }{N}_A\},v_i^A\in \mathbb{N}$ represents an actuator in the multi-tank system, e.g.: a pump. Every edge in $E^A\subseteq V^A\times V^T$ represents a connection between an actuator and a tank, where an edge $e=(v_i^A,v_j^T)\in E^A$ is always directed from an actuator vertex to a tank vertex. One actuator vertex can be connected to multiple tank vertices. The labeling function for the actuator subgraph $l^A:V_A\to L_A$ maps each actuator vertex to a corresponding vertex label that describes attributes of the fluid going through the actuator which may include: temperature of the fluid, composition or concentration of the fluid, etc.; e.g.: $l^A(v)=hotwater$.

Fig. 2(a) shows an example three-tank system and its corresponding graph representation (Fig. 2(b)). Elements from the tank subgraph are presented in continuous lines while the actuator subgraph is represented by discontinuous lines.

In this section we formalize the concepts of I/O pairing and stage definition, and how they relate to the controller auto-tuning framework defined in Section 4.1.

In this section we discuss how to design input and system prompts (Section 3.3) for an LLM in order to automate I/O pairing and stage definition, given limited structural knowledge of the system, for a family of multi-tank systems. We present an algorithm for input prompt design given the graph representation of the system defined in Section 4.2.

In order to evaluate the proposed I/O pairing and stage definition methodologies we use multiple performance metrics which evaluate output prompt formatting and accuracy, as well as auto-tuned controller performance.

In this section, we will describe the different multi-tank configurations used to empirically evaluate the proposed methodology. In order to highlight the details of each task, we use three distinct set of benchmarks to evaluate: I/O pairing LLM output, stage definition LLM output, and performance of tuned controllers for stage definition.

For I/O pairing evaluation we use 5 configurations of the multi-tank system which include: (1) three cascaded tanks (Fig. 8(a)) and (2) four cascaded tanks (Fig. 8(c)). We also evaluate I/O pairing on three system configurations taken from [12], where input-output pairing is framed as an optimization problem, and solved iteratively. These configurations are: (3) a well-known quadruple tank system (Fig. 9(a)), (4) a variation of the quadruple tank system (Fig. 9(b)), and (5) a single tank system where both level and temperature in the tank are controlled (Fig. 9(c)). Table 1 shows the list of configurations for I/O pairing evaluation.

For stage definition LLM output evaluation we use 7 configurations of the multi-tank system: (1) two cascaded tanks (Fig. 7(a)), (2) alternative two cascaded tanks (Fig. 7(b)), (3) two interconnected tanks (Fig. 7(c)), (4) three cascaded tanks (Fig. 8(a)) (5) alternative three cascaded tanks (Fig. 8(b)), (6) the three tank system illustrated in Fig. 2, and (7) four cascaded tanks (Fig. 8(c)). Table 2 shows the list of configurations for stage definition LLM output evaluation. The pairs of configurations (1)-(2) and (4)-(5) respectively describe a system with the same layout, but different tank labeling. We evaluate alternative labeling of the same configurations to test robustness of the prompt design methodology to small changes in the system description, which has shown to be a challenging task [3].

Finally, we evaluate the performance of tuned controllers following different stage definitions. We evaluate the performance for three benchmarks: (1) two cascaded tanks, (2) three cascaded tanks, (3) four cascaded tanks. Tank labeling is not relevant in the case of control performance evaluation.

Controller performance is evaluated for different levels of coupling. For this purpose, we define a coupling parameter $\phi$ that represents how coupled the control loops in the system are. For the case of the cascaded configurations used, we model the coupling parameter $\phi$ as a factor that linearly maps the diameters of the drain in the tanks. A high value of $\phi$ results in a larger drain diameter, which results in higher drain flows for each tank, which in turn results in higher input flows into the respective tanks below, translating into higher coupling between tanks. Conversely, a small drain diameter might result in practically non-existent coupling between the tanks, which would result in a set of isolated control loops, rendering the motivation for proper stage definition null. We evaluate the controller performance for three coupling levels: nominal coupling and 1.5 and 2 times nominal coupling.

Three stage definitions are evaluated, labeled: correct stage definition, incorrect stage definition, and independent stage definition. Correct stage definition for the three cascaded configurations refers to a solution that tunes the control loops following the hierarchy; i.e.: the control loop for the tank on top is tuned first, then the tank below, and so on until the bottom tank is reached, while at each stage, control loops tuned in previous stages are closed and controllers are fixed to their tuned parameters. Incorrect stage definition refers to a solution that tunes control loops in the opposite order to the correct stage definition while maintaining control loops from previous stages closed. Independent stage definition refers to the process of tuning a control loop independently at each stage, ie.: control tunings from previous stages do not carry over to the current stage and every loop, except the loop being tuned, is open at each stage. The tuning order in this case is not important. This is technically also an incorrect stage definition for the cascaded configurations, however, we include it as a separate case because it is a common approach to controller tuning in the related literature [2][12]. Fig. 10 shows example stage definitions for the two cascaded tanks system.

To evaluate the performance of the LLM-based methodology when proposing I/O pairings in a given system, we consider the benchmarks described in Section 5.2. Every input-system prompt pair was evaluated $N=100$ times and percentage values were reported. Overall, we can see in Table 1 that the performance was high across all metrics, with 100% consistency in terms of formatting, which is a challenging task. Accuracy was also high at over 98% for all cases. This empirically shows that an LLM-based solution can yield expert knowledge recommendations regarding I/O pairing for benchmarks within the family of multi-tank systems. Furthermore, for the case of the benchmarks taken from [12], the I/O pairing matches the one achieved by the optimization approach with over 98% accuracy, while requiring less resources in terms of optimization problem formulation, expert interactions, and, potentially, computational cost (assuming inference cost from an LLM interaction is lower than the cost from solving the optimization problem).

In order to evaluate the performance of the LLM-based methodology to define stages for a multistage control tuning framework, we consider the system configurations described in Section 5.2. Each input-system prompt pair was evaluated $N=100$ times and percentage values were reported. Table 2 shows a summary of the performance. While formatting was not as consistent as the I/O pairing case, we can see a very high accuracy/formatting metric with 100% accuracy in most benchmarks. As discussed in Section 5.1.1, accuracy/formatting should be the priority metric, since incorrect formatting can be automatically detected, repeating the interaction with the LLM until a correctly formatted output is produced.

We have evaluated the proposed methodology for a variety of multi-tank benchmarks (Figures 7, 8, and 9), including a benchmark where temperature and level in a tank are controlled simultaneously (Fig. 9(c)), and a set of cascading tank benchmarks with different levels of coupling (Fig. 11). We have empirically shown that, for these benchmarks, stage definition can have significant impact on the performance of the tuned controllers. Furthermore, we have shown that the proposed methodology for stage definition using LLMs can propose the correct stage definition and I/O pairing with a high degree of consistency and accuracy.

We acknowledge that, even though the actuator label space ($L^A(e)$) can integrate notions of temperature control, as well as other potential extensions (e.g.: liquid composition/concentration), the proposed methodology is mostly centered around the problem of level control in a family of systems consisting of reservoirs, pumps, and connections. In spite of this limitation, this is a first step towards a system-informed general methodology for prompt engineering with the purpose of expert knowledge automation. An extension of these results to a wider variety of benchmarks would require a redefinition of the mathematical language and the prompt engineering algorithm; however, the core elements of the methodology would persist: a graph representation of the system components and interactions, and a prompt engineering algorithm that binds this graph language to the input prompt space of an LLM. Future works should focus on proposing a generalized mathematical language and prompt engineering algorithm, or, if this is not possible, a set of general guidelines for developing these graph representations and prompt engineering algorithms for different families of benchmarks.

We have proposed a simple topology-based graph representation of the system since we are aiming for a solution that requires minimal expert interaction and knowledge of the system, which would be a limitation for more complex system representations such as bond graphs, process graphs, or structural analysis. However, a generalization of this methodology for systems other than the multi-tank benchmarks will most likely require more complex representations of the system, such as the ones mentioned above.

We have focused on systems with centralized control since I/O pairing and stage definition for sequential controller tuning would rarely be relevant in a centralized control approach. Furthermore, decentralized parametric controllers, or a hybrid between decentralized and supervisory control are industry standard solutions. However, future works should focus on extending the methodology to consider systems with centralized/supervisory control, as well as systems with integrated decoupling strategies.

This work is also limited by the lack of formal definition of what is a correct I/O pairing or stage definition apart from what is referenced as expert knowledge in the associated literature. Therefore, future works will provide a formal framework to measure validity of expert knowledge in terms of I/O pairing and stage definition.