AI that never leaves the building – and why that is only half of security

Cloud AI is not the right fit for everything. When data is confidential, networks are closed or response times must be guaranteed, language models and agents can run on your own hardware. Moving the model inside the walls solves one problem, though, and creates several others.

Why run AI locally?

In industry, the most valuable data is often the most sensitive: process recipes, equipment performance, maintenance histories and production volumes. Sending it to an external AI service is a data governance decision that many companies have never explicitly made.

There is also a network argument. Production and control networks are deliberately isolated or tightly firewalled. Getting their data to a cloud AI service means opening paths that security teams resist for good reason. A local model works inside the existing security boundary rather than around it.

Local AI also keeps running when the connection drops, gives predictable response times, and keeps the full audit trail of inputs and outputs in your own systems.

Local, on-premise and air-gapped are not the same thing

The terms are often mixed up, but the difference matters for both security and compliance:

Model Where the data goes Typical use
Cloud AI Processed outside your environment Non-sensitive analysis, frontier models
On-premise, connected Stays inside; updates and monitoring over a controlled connection Most industrial use cases
Air-gapped Nothing in or out; updates delivered offline Classified and safety-critical environments

Many organisations end up with a hybrid: local AI for operational and sensitive data, cloud for the rest.

Sizing the hardware to the use case

Local AI no longer requires a data centre. The right hardware depends on the task:

Hardware Fits
High-performance laptop A narrow use case, such as analysing factory logs and prioritising alarms
GPU workstation or server More capable models and larger datasets, locally
GPU cluster Many users or strict response-time requirements

What makes small hardware work is tuning: open-weight models adapted and fine-tuned to the chosen tasks, and libraries optimised for the available GPU capacity.

Local does not mean secure

The security community's view of AI has shifted. The OWASP Top 10 for LLM Applications, updated in 2026 for the first time partly on real incident data, still ranks prompt injection first and has moved excessive agency up to third place. Its central advice is to stop trying to build a model that cannot be fooled, and instead design the system so that nothing important breaks when it is.

None of these risks disappear when the model runs on your own server:

  • The model file itself. Open-weight models are downloaded from public hubs. Security researchers have repeatedly found malicious or tampered models there, including files that run code when loaded. A model is software supply chain and needs the same scrutiny.

  • Prompt injection. A local model still reads documents, logs and messages. Any of them can carry hidden instructions that the model may follow.

  • Excessive agency. An agent that can read systems, write tickets or change settings is only as safe as its permissions. Every capability it does not need is attack surface.

  • Internal data leakage. "The data stays in the building" does not mean every employee should see it. A local assistant can expose maintenance contracts or HR files to the wrong internal user just as easily as a cloud one.

  • The server is a system too. The inference server, its libraries and its interfaces need patching, access control and monitoring like any other production system.

What a verified workflow looks like

In closed environments we build security into the workflow from the start rather than adding it afterwards:

  • Vetted models only. Models come from known sources, in safe file formats, with versions pinned and checksums verified.

  • Signed, controlled updates. In air-gapped environments, updates are delivered offline and validated before use.

  • Least privilege for agents. Each agent gets only the tools and data its task needs; actions with real consequences require human approval.

  • Access control on answers. The model only retrieves what the asking user is allowed to see.

  • Logging and traceability. Every query, data source and agent action is recorded, so decisions can be explained afterwards.

  • Monitoring. The AI system is part of the same continuous monitoring as the rest of the environment.

What it looks like in practice

Examples from our recent deployments, anonymised:

  • Predictive maintenance. A fault in a pump system was detected 12 days before the vibration alarm, and the fault type was identified.

  • Anomaly detection. An agent identified a tampered energy meter and pinpointed the day the anomaly began.

  • Closed environments. Deployments in classified, air-gapped environments.

Regulation is pushing in the same direction

NIS2 and Finland's Cybersecurity Act require organisations in many sectors to manage the risks of their information systems, and AI systems are no exception.

Local AI brings the benefits of language models to places the cloud cannot reach. Done well, it is not just a model on a server, but a system where hardware, model, permissions and monitoring are designed together.

Thinking about AI for data that cannot leave the building?
Book a 30-minute conversation with our expert.

Previous
Previous

Tamtron One Cloud — Evolving a core digital platform.

Next
Next

Turn Intention Into Action