Skip to content

Banking & Finance

|

CS.124

AI chatbot security testing in banking

Testing an LLM-based banking chatbot against the OWASP Top 10 for LLM Applications 2025: prompt injection, data disclosure, safeguards in place before go-live.

Talk to an expertFree consultation, no strings attached.We reply within 1 business day

Client

Large financial institution deploying a customer-facing chatbot

Challenge

A financial institution was preparing to launch a chatbot built on a large language model (LLM) that answers customer questions about banking products. The chatbot used customer data and the bank's business logic, so on top of classic application vulnerabilities it brought LLM-specific risks: prompt injection, disclosure of another customer's data, and model responses manipulated by an attacker. The bank wanted these assessed before the system reached customers.

Approach

01

Threat modeling

Mapping the chatbot's architecture (model, system prompt, data sources, integrations with the bank's systems) to the OWASP Top 10 for LLM Applications 2025, including Prompt Injection, Sensitive Information Disclosure, Improper Output Handling, Excessive Agency and System Prompt Leakage.

02

Prompt injection testing

Attempts to bypass the model's guardrails and jailbreak it, to force disallowed actions, and to pull other customers' data through the chatbot's integrations.

03

Data disclosure and output handling

Verifying customer session isolation, attempting to extract the system prompt, and checking whether the application validates model output before displaying it.

04

Control recommendations

Data access authorized in the bank's systems in the context of the signed-in customer, independent of instructions in the prompt. No sensitive data in the system prompt, input validation, output filtering and query monitoring.

Results

Before the chatbot launched we found prompt injection access to other customers' data, and the team put safeguards in place.

  • Before launch we found prompt injection paths that gave access to other customers' data.

  • Tests confirmed that the chatbot disclosed parts of its system prompt.

  • The institution's team implemented the recommended safeguards before the chatbot went live to customers.

Planning a similar project?

30 minutes with a consultant who runs projects like yours. We agree the scope and tell you what the price depends on.

We'll discuss scope, methodology, and timeline.

Book a consultation

Free consultation, no strings attached.

We reply within 1 business day

Book a call