Banking & Finance
|CS.124
AI chatbot security testing in banking
Testing an LLM-based banking chatbot against the OWASP Top 10 for LLM Applications 2025: prompt injection, data disclosure, safeguards in place before go-live.
Client
Large financial institution deploying a customer-facing chatbot
Challenge
A financial institution was preparing to launch a chatbot built on a large language model (LLM) that answers customer questions about banking products. The chatbot used customer data and the bank's business logic, so on top of classic application vulnerabilities it brought LLM-specific risks: prompt injection, disclosure of another customer's data, and model responses manipulated by an attacker. The bank wanted these assessed before the system reached customers.
Approach
Threat modeling
Mapping the chatbot's architecture (model, system prompt, data sources, integrations with the bank's systems) to the OWASP Top 10 for LLM Applications 2025, including Prompt Injection, Sensitive Information Disclosure, Improper Output Handling, Excessive Agency and System Prompt Leakage.
Prompt injection testing
Attempts to bypass the model's guardrails and jailbreak it, to force disallowed actions, and to pull other customers' data through the chatbot's integrations.
Data disclosure and output handling
Verifying customer session isolation, attempting to extract the system prompt, and checking whether the application validates model output before displaying it.
Control recommendations
Data access authorized in the bank's systems in the context of the signed-in customer, independent of instructions in the prompt. No sensitive data in the system prompt, input validation, output filtering and query monitoring.
Results
Before the chatbot launched we found prompt injection access to other customers' data, and the team put safeguards in place.
Before launch we found prompt injection paths that gave access to other customers' data.
Tests confirmed that the chatbot disclosed parts of its system prompt.
The institution's team implemented the recommended safeguards before the chatbot went live to customers.
Planning a similar project?
30 minutes with a consultant who runs projects like yours. We agree the scope and tell you what the price depends on.
Related case studies
CS.106
Adversary Emulation Security Testing
The bank's SOC tested what it detects across 7 APT scenarios and received recommendations to strengthen monitoring.
CS.121
Security Architect as a Service: a new change-review process for a bank in 3 weeks
The bank cleared a backlog of more than 4 months, and after 2 weeks it was reviewing new change requests as they arrived.
CS.101
Secure Retail Banking Migration to AWS Cloud
The retail bank moved its services to AWS with no disruption for customers and classified its data to KNF requirements.
We'll discuss scope, methodology, and timeline.