Automating Invoice Validation with Knowledge Graphs: Optimizations and Practical Lessons

The EDIFACT ontology was developed following best practices, like the NeOn method [31] or the formulation of ontology requirements and competency questions (Table 1). These steps were carried out in collaboration with Electronic Data Interchange experts from E/D/E. The development process is detailed in our previous work [19].

Following ontology design best practices [31, 23], we reuse existing resources to develop the EDIFACT-Ontology. An overview of the reused resources and the number of reuses can be found in Table 2. We apply the agent role pattern from the Modular Ontology Design Library (MODL) [28] for modelling the participation of organizations in invoices. This is necessary as the same organization can have many roles (seller, supplier, etc.) in the same or different invoices, to which different property values can be associated depending on its role. The ontologies listed in Table 2 also provide adequate solutions for our purpose. Most of them are reused in the class AgentRole; other examples are the country code from the Funding, Research Administration, and Projects Ontology [29] or the address of the vCard Ontology [21]. Also, four of the seven different main classes we defined in our EDIFACT Ontology are linked to concepts of these vocabularies or ontologies. For example, the class FormalOrganization is linked to the Core Organization Ontology (org), the E-Invoice to the P2P-O module document, and Item to the P2P-O module item.

The EDIFACT Ontology is tailored to represent the concepts and fields of the EDIFACT standard. The proposed OWL ontology comprises 31 classes, 22 OWL object properties, 261 OWL data properties, and 6 annotation properties. The ontology was developed using WebProtégé [12]. Figure 1 illustrates the main concept of the EDIFACT ontologies by showing the connections between the different classes. The EDIFACT Ontology can be found under https://purl.org/edifact/ontology. Additionally, it has been integrated into the LOV catalogue, available at https://lov.linkeddata.es/dataset/lov/vocabs/edifact-o. The classes and some of their most relevant properties for the invoice KGs are discussed in detail in the following. We focus on properties that are essential for representing the structural and semantic aspects of invoices, while omitting less relevant or supplemental properties to maintain clarity and conciseness. The relevant competency questions for each class are provided in Table 1.

• $\mathrm{■}$

  E-Invoice This is the main class of the EDIFACT ontology, and its individuals or entities represent electronic invoices. This class is connected to other classes through object properties, including EDIFACT-Structure via the property followsStandard, Item via the property hasItem, InvoiceDetails via the property hasInvoiceDetails, and AgentRoles via the property isProvidedBy to capture the role of organizations in the invoice. The only data property of this class is belongsToProcess, denoting the business process of the invoice.

• $\mathrm{■}$

  EDIFACT-Structure This class contains all the information that ensures that the invoice file meets the requirements of the EDIFACT file format. This information appears at the beginning and end of a message and, therefore, does not belong to the content of the individual invoices. As a result, this class and the class InvoiceContent are disjoint, specified with the owl:disjointWith predicate. However, as this information is part of an invoice, this class is connected to the class E-Invoice via the object property followsStandard. The data properties of this class include creationDate, dataExchangeCounter, messageReferenceNumber, and senderIndicator. Among the datatype properties of the entities of this class, we have creationDate, dataExchangeCounter, messageReferenceNumber, and senderIndicator.

• $\mathrm{■}$

  Invoice Details This class contains all the information that can only occur in the header- and summary sections of the invoices. Exemplary datatype properties of this class are the document date (hasDocumentDate), the document number (hasDocumentNumber), the delivery condition (deliveryCondition), and the invoice amount (hasInvoiceAmount). The connection of the InvoiceDetails class to the E-Invoice class is done via the object property hasInvoiceDetails.

• $\mathrm{■}$

  Item This class allows for representing an item listed in the invoice, which is found in the detail section of the invoice. Examples of the datatype properties are the name of the item (p2p-o-item:Item), the net price of the item (hasNetPriceOfItem), the net weight of an item (hasNetWeight), the number assigned to product according to the International Article Numbering Association (internationalArticleNumber). Examples of the datatype properties for an item are the name of the item (p2p-o-item:Item), the net price of the item (hasNetPriceOfItem), the net weight of an item (hasNetWeight), the number assigned to a manufacturer’s product according to the International Article Numbering Association (internationalArticleNumber), etc. We model this relationship as a multi-valued property by defining the object property hasItem (and its inverse isItemOf) between the Item and the E-Invoice classes.

• $\mathrm{■}$

  InvoiceContent This class allows for modelling the information that can be found in all three sections of the invoices. As a result, the information of the InvoiceContent class is defined as the union of the classes Item and InvoiceDetails, i.e., InvoiceContent owl:unionOf (Item InvoiceDetails). Creating the InvoiceContent class ensures the consistency of the ontology by defining properties that apply to all parts of the invoice.

• $\mathrm{■}$

  AgentRole According to the guidelines, an organization can, or sometimes must, have many different roles. For example, in the warehousing business, one organization needs to have three roles: Buyer Role, Invoicee Role, and Delivery Party Role. We model these cases using the Role ontology pattern as defined by Shimizu et. al [28] and Grüninger & Fox [8]. In our ontology, the purpose of the class AgentRole is to represent the manifold roles an organization can have in an invoice. As a solution, several subclasses have been created, each representing one of these roles. The connection to the E-Invoice class exists through the object property isProvidedBy with the E-Invoice class in the range.

• $\mathrm{■}$

  FormalOrganization This class represents the organizations involved in the messages, addressing CQ 1. The properties assigned to the class FormalOrganization serve two purposes: assigning the role an organization plays in an invoice through performsAgentRole, and providing a globally unique identifier for organizations, i.e., globalLocationNumber. In the EDIFACT ontology, the only connection of the class FormalOrganization to another class is the class AgentRole via the object property performsAgentRole with the AgentRole class in the range.

edifact-val is implemented in Python 3 and available online.³³3https://github.com/DE-TUM/EDIFACT-VAL For SHACL validation, we use pySHACL [30], an open-source Python library. edifact-val is equipped with 394 SHACL constraints provided by the experts following the EDIFACT standard. All experiments have been conducted on a machine with an AMD EPYC 9224 24-Core CPU, 578 GB of RAM, and running Ubuntu 24.04.1 LTS.

We use 44 different real-world EDIFACT messages from 12 different suppliers and 6 different business cases to evaluate the performance of the edifact-val tool. The selection of messages has been made together with the EDI experts from E/D/E to have a range of messages representing the day-to-day business workflow. Since each supplier may include different segments and data elements in the invoices, the selected messages represent a wide range of segments and segment combinations. All 394 SHACL constraints matched at least one triple across the 44 EDIFACT messages used in the evaluation, while also all resulted in at least one validation error. This corresponds to a relevance coverage of 100% and a violation coverage of 100%. These constraints, however, do not cover all data represented in the invoice knowledge graphs. Rather, they focus on the most critical aspects of invoice validation for legal compliance with e-invoice regulations, like mandatory standardized organizational data and required financial calculations.

One issue arises from the syntax of EDIFACT messages, where certain characters, such as $+$ and ^′, are used to indicate the start of the next element or segment. In some of the evaluated messages, these characters were also used as content within data elements, leading to incorrect splitting of the data elements. In edifact-val v.1, the input was an XML version of the invoice, and the in-house translation by E/D/E was able to handle these errors. However, because the use of these symbols as content constitutes an error and should be communicated to the customers, we decided to treat them as errors. This issue frequently occurred in the NAD-Segment, which describes the involved organizations, as some companies include the character $+$ in their name. As a result, the syntax of the following data elements, such as postal code or country code, was often missing. This led to a reduction in the number of properties in the RDF graph created for Message 12 by edifact-val v.2, as summarized in Table 3.

As outlined in Section 3.2.1, edifact-val v.2 omits data elements that have no content in the RDF graph. In contrast, edifact-val v.1 did not apply this filtering, resulting in the presence of triples with empty objects ("") that are no longer generated in the RDF graph by edifact-val v.2. Consequently, this filtering step also reduces the number of properties represented in the RDF graphs created by edifact-val v.2, as summarized in Table 3. It is important to note that the magnitude of this reduction varies between messages, since the messages are sourced from different suppliers with differing invoice practices. For instance, some suppliers – such as the supplier of Message 1 – do not include empty data elements, while others – like the supplier of Message 15 – do include them.

The simpler creation of the XML file in edifact-val v.2 has led to a reduction in the number of nested XML elements. As described in Section 3.2.1, this simplification has reduced the number of attributes needed during the pre-processing step to correctly identify and allocate the segments. This is particularly relevant for segments that can occur in all three parts of the message, such as the RFF-Segment and the DTM-Segment. In edifact-val v.1, it was often difficult to correctly allocate these segments because of the complex nested XML structure. Since certain specific combinations of these segments (for example, the valuta date represented by DTM+209) only occur in particular parts of the message, precise allocation is crucial and determines whether they are represented in the RDF graph. This improvement in the XML creation process directly affects the completeness of the RDF graph and explains why, in some cases, edifact-val v.2 generates more triples and properties than edifact-val v.1. This is particularly evident in the results for Messages 3 and 4 in Table 3, where Message 3 has one additional property and Message 4 has two additional properties in the new version of the RDF graph, and therefore also more triples. These findings highlight that the simpler XML creation in edifact-val v.2 not only reduces unnecessary data but also ensures that critical segments are correctly represented, contributing to a more complete and accurate RDF graph.

Since the number of triples differs between the RDF graphs generated by edifact-val v.1 and edifact-val v.2, we cannot claim that the RDF graphs are identical. However, our investigations demonstrate that the RDF graph generated by edifact-val v.2 offers an improvement in data representation. Importantly, the number of nodes (entities) remains the same in both graphs, ensuring that all concepts and entities of the invoices are still displayed. At the same time, edifact-val v.2 allows us to: (i) omit empty data elements, (ii) identify and exclude errors that were previously undetectable, and (iii) represent information that was previously not included due to the nested structure of the input data. These improvements collectively contribute to a more reliable, accurate, and domain-expert-validated foundation for further processing and analysis of EDIFACT invoices.

The runtime comparison shown in Figure 3 reveals a significant decrease in the total runtime of edifact-val v.2 compared to edifact-val v.1. For smaller EDIFACT messages, the runtime with edifact-val v.1 was around 8 seconds and is now reduced to about 2 seconds with edifact-val v.2. For larger EDIFACT messages, the runtime drops substantially, from 50 seconds with edifact-val v.1 to under 4 seconds with edifact-val v.2. These observations show that the improvement is consistent across all tested messages and demonstrate that edifact-val v.2 effectively handles increasing input sizes. A closer examination of each runtime’s composition (Figure 3) shows that this improvement results from the KG construction phase, which is the only part where edifact-val v.1 and edifact-val v.2 differ.

As noted before, the time spent on KG Construction is where edifact-val v.2 is significantly more efficient than edifact-val v.1. Looking only at the times for KG Construction, the average percentage of time saved with edifact-val v.2 is 82.10$\%$. There are several reasons for this improvement: First, with the tailored pre-processing described in Section 3.2.1, the number of created XML elements is reduced by removing unnecessary elements and avoiding nested structures that do not contribute to the final RDF graph. Figure 5 provides an overview of the number of XML elements generated by edifact-val v.2 and edifact-val v.1 for each EDIFACT message. This figure shows that, on average, the number of XML elements is reduced by 35.39$\%$ across all EDIFACT messages. This reduction means that there are fewer XML elements that need to be checked and processed by the RDF mappings in the following step. As a result, the RDF mappings can be executed more quickly and efficiently, which directly speeds up the KG construction process, as shown in Figure 4. A second improvement of the pre-processing that significantly speeds up the KG construction time is that the created XML files no longer need to be parsed and saved multiple times. In edifact-val v.1, the XML files were repeatedly parsed, modified, and saved again, leading to redundant file handling operations. By contrast, edifact-val v.2 streamlines this process by parsing, modifying, and saving the XML content in a single step within the pre-processing phase. This eliminates unnecessary operations, reduces memory usage, and improves the overall computational efficiency. As a result, the KG construction process becomes more streamlined and significantly faster, contributing to the overall runtime improvement observed in Figure 4. Third, Section 3.2.2 describes the reduction of both YARRRML-Mapping and RML-Mapping in edifact-val v.2. The YARRRML-Mapping was reduced by 42.2$\%$ due to fewer needed data sources, which led to a reduction of the RML-Mapping by 97.3$\%$. This substantial reduction directly impacts the KG construction runtime by decreasing the parsing and processing overhead for the RML Mapper, simplifying the graph construction phase, and reducing the number of triple patterns to be executed. Overall, the leaner and more direct mapping instructions in edifact-val v.2 make the RDF graph generation faster and more efficient, as reflected in the runtime comparison shown in Figure 3.

Since the RDF graphs created by edifact-val v.1 and edifact-val v.2 are almost identical, as discussed in Section 4.2, the number of RDF triples is also nearly the same (see Table 3). Consequently, the runtime of the RDF validation with SHACL is also similar in both approaches. Across all tested messages, the validation time ranges from as little as 0.05 seconds for smaller EDIFACT messages to a maximum of 1.2 seconds for the largest EDIFACT messages. On average, the runtime reduction in the KG validation phase is 10.8$\%$, which can be attributed to the slight reduction in triples described in Section 4.2.

Overall, the analysis of the runtime shows that edifact-val v.2 is significantly more efficient than edifact-val v.1, thanks to its optimized pre-processing and RDF mapping steps. The runtime per EDIFACT element is reduced from 28.53 milliseconds to 9.54 milliseconds, resulting in a normalized runtime reduction of 66.56$\%$. This substantial improvement stems from the more efficient KG construction phase in edifact-val v.2, which eliminates redundant processing steps and accelerates the generation of the RDF graph.