Trustworthy Generative AI for Financial Services

Guardrails for foundation models and requests are measures to ensure their safe, ethical, and effective use. They help in mitigating risks, preventing harmful outcomes, and ensuring compliance with legal and ethical standards [15]. In general, a distinction can be made between input and output guardrails. Input guardrails verify that requests that are classified as risky will not enter the LLM model, for example, inquiries on a critical topic such as the construction of weapons. In contrast, output guardrails also check for hallucinations [7]. All foundation model providers implement safety measures [15], like alignment fine-tuning. While this provides a solid basis for ethical GenAI, they are not sufficient in practice as they can not be adapted to specific use cases and can be overcome by malicious actors [3].

Therefore, a client-based prompt analysis, called GFT AI Shield, is used to detect, for example, unethical requests or violations of internal policies. This analysis is performed purely inside the client and the prompt under inspection is never forwarded to a commercial GenAI provider to prevent data leakage. Only when no violations, for example, a prompt containing credit card information, were detected, the user prompt is forwarded to an LLM for answer generation. The GFT AI Shield uses different methods to detect inadequate content:

• $\mathrm{■}$

  Simple pattern matching is used to detect restricted words

• $\mathrm{■}$

  Named-entity recognition [13] using a local neural network checks if restricted categories are used in the prompt

• $\mathrm{■}$

  Using a plugin interface, customers can provide additional classifiers to detect problematic prompts

If a prompt is flagged by one of the filters, the user is informed why the request is not being processed. All violations against the guardrails are logged in a database. As it is always the case with artificial intelligence (AI) applications, the GFT AI Shield can not guarantee the perfect correctness of all predictions. To prevent users from simply switching to public GenAI services, which have a lower security standard, the GFT AI Shield is tuned to limit false positive errors.

In general, GenAI applications can either be targeted to an internal or public user base of a company. In the case of an internal application, the employees can receive tailored training for responsible GenAI usage. Furthermore, employers can, to some extent, enforce compliant behavior through internal guidelines and terms of usage agreements. However, just relying on compliant behavior is not sufficient as users may make errors or behave maliciously.

Privacy for GenAI using GFT EnterpriseGPT is safeguarded through several key measures. Selecting appropriate licensing is crucial, as it defines the terms under which user-generated prompts, model answers, or datasets for fine-tuning, can be used by the foundation model provider. Even though all model providers guarantee that data is not visible to other users, end-users can not verify this guarantee. Using dedicated hardware, such as PTUs [11] or dedicated servers to host open-source models, helps to prevent unauthorized access and data leaks, providing a controlled environment where sensitive information can be processed without exposure risks. Additionally, the GFT AI Shield can also be used for enforcing data privacy. As prompts are directly processed on the user’s device rather than on remote servers, the need to transmit personally identifiable information (PII) or confidential information across networks to service providers is eliminated, minimizing the risk of data interception or misuse. Finally, as GFT EnterpriseGPT also uses tools accessing the internet, for example using a web search, user prompts are rewritten using GenAI to prevent prompt leakage to another third-party service. Together, these measures create a robust framework for protecting user privacy in the deployment and use of GenAI.

Grounding for GenAI refers to the process of linking their outputs to real-world contexts, facts, or external data sources. This ensures the models’ responses are accurate, relevant, and contextually appropriate, enhancing their reliability and applicability in practical scenarios. Retrieval-augmented generation (RAG) [9] is a standard method for grounding [8]. It augments a user prompt with information retrieved from external sources before sending it to a foundation model. By implementing our own RAG approach instead of using a commercial provider, a fine-tuned solution per use-case can be developed without exposing all data sources to a third party. While this approach can be used to encode domain-specific knowledge into the foundation model, hallucinations can still occur [5]. To ensure the correctness of results, our platform offers the option to search for references for parts of the generated answer. By reusing the retrieval part of retrieval-augmented generation (RAG), we are searching for a context similar to a specific part of the answer selected for grounding. If such a similar context exists, the correctness of the partial answer can be ensured.