A support chatbot that can read account records can also repeat them to the wrong person. OWASP ranks that risk second in its 2025 Top 10 for LLM applications, under the name sensitive information disclosure, and its guidance is blunt about one popular fix: telling the model in its system prompt not to reveal certain data may not hold. For a fintech product, the more durable controls sit outside the model, and several of them line up with rules a US financial company already follows.
What OWASP means by sensitive information disclosure
OWASP’s LLM02:2025 entry covers personal information, financial details, health records, confidential business data, security credentials and legal documents. It describes three common forms: personal data leaking during a conversation, model outputs exposing proprietary algorithms or training data, and responses that include confidential business information.
Its example scenarios are ordinary rather than exotic. In one, a user receives another user’s personal data because of inadequate sanitization. In another, an attacker gets past input filters to pull out sensitive information. In the third, careless inclusion of data in training leads to disclosure later.
The entry also says restrictions in the system prompt about what data the model may return “may not always be honored” and can be bypassed through prompt injection or other methods. Our separate guide covers prompt injection defenses; this piece is about what the model can see in the first place.
The system prompt is not a vault
A related OWASP entry, LLM07:2025 System Prompt Leakage, states that a system prompt should not be considered a secret or used as a security control. It tells developers to keep API keys, database names, user roles and permission structures out of prompts and in systems the model does not directly access.
OWASP’s own example is a banking chatbot whose prompt includes a daily transaction limit and a total loan limit. If the prompt leaks, an attacker learns the thresholds to work around. OWASP’s point is that the real flaw is an application that delegates limits and authorization to the model at all. It says privilege separation and authorization checks must happen in a deterministic, auditable way outside the LLM.
Controls that work outside the model
OWASP’s prevention list for LLM02 is long. For a product team, the parts that change system design fall into four groups.
| Control | What OWASP recommends | How it shows up in a fintech app |
|---|---|---|
| Least privilege | Grant access only to data a user or process needs | Retrieval runs with the signed-in customer’s permissions, never a service account that can read every account |
| Restrict data sources | Limit what external sources the model can reach | The assistant queries a narrow API, not the core ledger or a raw document store |
| Sanitize and redact | Scrub or mask sensitive content; tokenize and redact with pattern matching before processing | Account and card numbers are masked before text reaches the model or its logs |
| Transparency | Clear retention, use and deletion policies; let users opt out of training use | The privacy notice matches what the vendor contract actually allows |
The third column is our application of OWASP’s general guidance, not text from OWASP. The same logic applies to retrieval pipelines: an index built from every customer’s documents can return one customer’s statement to another if access filtering happens after retrieval rather than before. Our analysis of retrieval over financial documents covers the accuracy side of that design.
OWASP also lists differential privacy, federated learning and homomorphic encryption. Those matter mainly to teams training or fine-tuning their own models. A company calling a hosted model through an API will usually get more from access design and redaction.
Where the Safeguards Rule comes in
For non-bank financial companies under the Federal Trade Commission’s jurisdiction, the Safeguards Rule at 16 CFR 314 already requires several of these controls for customer information. The rule does not mention language models, and it applies whether or not an LLM is involved. Several of its requirements in section 314.4 map closely to an LLM feature:
- Access controls (
314.4(c)(1)): authenticate users and limit each one to the customer information needed for their duties, or, for customers, to their own information. - Encryption (
314.4(c)(3)): encrypt customer information in transit over external networks and at rest, or use compensating controls approved by the company’s Qualified Individual. - Monitoring and logging (
314.4(c)(8)): monitor and log the activity of authorized users and detect unauthorized access or use. Prompt and response logs that hold customer data are themselves customer information to protect. - Service provider oversight (
314.4(f)): select providers capable of maintaining appropriate safeguards, require those safeguards by contract, and periodically assess providers based on risk.
The last item covers a hosted model vendor that receives customer information. Retention terms, training use and subprocessors are contract questions under that paragraph, not settings to discover after launch. New York-regulated firms have a parallel set of duties under the state’s cybersecurity regulation, which our explainer on NYDFS Part 500 covers.
A short design checklist
Before a model touches customer records, a team can answer five questions from its own architecture documents:
- Which identity does the retrieval or tool call run as, and can it reach records the user cannot?
- What sensitive fields are masked before text reaches the model, and before it reaches logs?
- What does the system prompt contain, and would leaking it expose anything beyond wording?
- Where are authorization and transaction limits enforced, and does that path bypass the model entirely?
- What does the vendor contract say about retention, training use and assessment, and who reviews it?
None of these depends on the model refusing a request. That is the practical lesson of OWASP’s guidance: instructions to the model are a layer of behavior, not a boundary.
