LLMs in the Enterprise: Security and Governance First

The gap between the pilot and the policy

Most organizations did not decide to adopt large language models. Their employees did, one browser tab at a time. By the time leadership asks the security team for an AI strategy, marketing is already pasting campaign data into a consumer chatbot, an engineer is feeding proprietary code to a coding assistant, and someone in finance is piping customer records into a public model API. The pilot happened without you. The job now is governance.

That is not a reason to ban the technology. A blanket ban simply pushes usage underground, where you have zero visibility. The goal is to make the safe path the easy path: approved tools, guardrails around sensitive data, and rules a normal employee can follow. This post covers the risks that matter, the controls that contain them, the choice between RAG and fine-tuning, and how to turn all of it into an acceptable-use policy.

The real risks, named

Vague anxiety about "AI safety" does not help. Name the concrete failure modes and you can control them.

• Data leakage to third-party models. The most common and most expensive risk. When an employee pastes source code, contracts, PII, or PHI into a consumer LLM, that text leaves your boundary. Depending on the vendor and plan, it may be retained, logged by a human reviewer, or used to train future models. Enterprise API tiers typically exclude your data from training; free consumer tiers often do not — and employees rarely know the difference.
• Shadow AI. The unsanctioned tools nobody approved. Industry breach reporting now tracks heavy generative-AI use on corporate systems, most of it through personal accounts on non-corporate infrastructure. You cannot apply DLP, logging, or retention rules to a tool you do not know exists.
• Prompt injection. When an LLM reads untrusted content — a web page, an email, an uploaded PDF, a support ticket — that content can carry hidden instructions that hijack the model. An agent with tool access can be tricked into exfiltrating data or acting on the attacker's behalf. This is the top entry on the OWASP Top 10 for LLM Applications, and it has no clean fix; it is mitigated, not eliminated.
• Hallucination in decisions. Models produce fluent, confident text that is sometimes simply wrong. That is tolerable for a first draft and dangerous for a legal summary, a medical instruction, or a compliance answer. The risk is not that the model errs occasionally — it is that a human downstream treats the output as authoritative and acts on it.

Governance controls that actually hold

Controls beat policy documents. Here is the stack that contains the risks above.

1. Approved tools, not a free-for-all. Publish a short list of sanctioned platforms — an enterprise LLM tier with a no-training commitment, or a privately hosted model — and make them genuinely convenient. If the approved tool is slower than the consumer app, people route around it.
2. Data loss prevention on the egress path. DLP inspects what leaves your environment and blocks the patterns that must never go to an external model: source code repositories, PII, PHI, cardholder data, credentials. Route AI traffic through a gateway or SASE broker so those inspection rules apply to chatbot prompts the same way they apply to email and file uploads. Our data loss prevention practice extends the same classification policy to generative-AI destinations rather than treating them as a separate, unmonitored channel.
3. Access control and least privilege for AI systems. Treat an internal LLM application like any other system that touches sensitive data. The model, and especially any agent with tool access, should hold only the permissions its task requires. An AI assistant that can read every SharePoint site the requesting user cannot is an access-control failure waiting to become a breach.
4. Logging and retention. Capture prompts and responses for sanctioned tools, with appropriate privacy controls, and feed that telemetry into your monitoring so misuse is detectable after the fact. If you cannot answer "what did people send to which model last month," you cannot investigate an incident or satisfy an auditor.

Building these into one coherent program — not four disconnected point fixes — is where an enterprise AI practice earns its place beside the security team.

RAG or fine-tuning: pick for the data, not the demo

To make a model answer from your own information, there are two paths — and teams routinely pick the wrong one because fine-tuning sounds more impressive.

• Retrieval-augmented generation (RAG) keeps your data in a governed store — a vector database or search index — and retrieves the relevant passages at query time, inserting them into the prompt. The model itself never absorbs your data permanently.
• Fine-tuning bakes patterns from your data into the model's weights through additional training. It changes how the model behaves rather than what facts it can look up.

For most enterprise use cases — answering from policy documents, knowledge bases, or tickets — RAG is the better first choice, for governance reasons:

• Access control survives. Because the data stays in a store you control, you can enforce per-document permissions at retrieval time, so a user only gets passages they are entitled to see. Fine-tuned weights cannot enforce row-level access — once a fact is baked in, anyone who can query the model may surface it.
• Freshness and correction. Update a document and the next answer reflects it. A fine-tuned model is frozen at training time; removing a fact means retraining.
• Auditability. RAG can cite the source passage it used — decisive for the hallucination problem and for any regulated decision.
• Cost and effort. RAG needs no training runs and adapts as your corpus changes. Fine-tuning earns its keep for style, tone, and format — not for teaching the model facts it should be looking up.

A practical default: start with RAG plus strong retrieval permissions, fine-tune only when you need consistent behavior a prompt cannot reliably produce.

Keeping sensitive data private

This comes down to where inference happens and what the vendor may do with the input.

• Use enterprise tiers with contractual data protection. Confirm in writing that prompts are not used for training and that retention is bounded. Get it from the contract, not the marketing page.
• Consider self-hosted or private-cloud models for your most sensitive workloads — regulated data, trade secrets, anything under HIPAA or export control. An open-weight model inside your own boundary means no prompt ever leaves it.
• Minimize before you send. Redact or tokenize identifiers so the model gets only what it needs to reason.
• Map it to your compliance obligations. GDPR, HIPAA, and contractual data agreements do not pause for AI — feeding regulated data to an unvetted model can itself be the violation. Fold AI data flows into your existing security compliance evidence so the auditor sees one controlled program, not an unmanaged exception.

Write an acceptable-use policy people will follow

The policy that works is short, specific, and readable in five minutes. At minimum, cover:

• Approved tools. Name the sanctioned platforms and state plainly that unapproved consumer AI tools are not permitted for company data.
• Data classification rules. Spell out what may never be entered into any external model — customer PII, PHI, source code, credentials, unreleased financials, legal matters — with concrete examples, not just categories.
• Human review requirement. Any AI output used in a decision, a customer message, or a published document must be reviewed and owned by a person. The model drafts; a human is accountable.
• Disclosure. State when AI-generated content must be labeled, internally or to customers.
• Reporting. Give people a fast, blameless way to report a suspected leak — you want to hear about mistakes early.

Pair the document with training so it is understood rather than merely acknowledged. The failure mode of every AUP is the employee who never read it.

Start with visibility

You cannot govern what you cannot see. The first step is nearly always discovery: find which AI tools are in use, what data flows to them, and where the exposures are today. From there the controls — approved tooling, DLP on the AI egress path, retrieval-based architecture, and a policy people follow — turn shadow AI into a program you run on purpose.

intSignal helps enterprises adopt LLMs without giving up control: AI usage discovery, governance and DLP design, private RAG architecture, and an acceptable-use policy built to your compliance obligations. Talk to our team and put security and governance first — before the next pilot becomes the next incident.