Air-gapped AI: how to run AI models with no internet connection

NVIDIA's DGX documentation describes an air-gapped system as one that is not connected to an unsecured network, such as the public internet, an unsecured LAN, or other computers connected to an unsecured network. For AI, that means models, prompts, documents, outputs and logs never cross the boundary, and nothing outside can reach the system.

Air-gapped is stricter than on premise. An on-premise system may still download updates or send telemetry. An air-gapped system does neither, so every update has to arrive through a controlled process.

Open-weight language models, embedding models and speech models run on local GPUs without any internet connection once their files are installed. So do the surrounding services: inference engines, an OpenAI-compatible API gateway for internal applications, vector databases for document search, notebooks, single sign-on against your internal directory, monitoring and audit logs.

Guardrails matter more in a closed environment, because you cannot call an external moderation service. Choose a platform whose PII masking, content filtering and other guardrails run locally.

Most system software normally updates over the internet. NVIDIA documents two alternatives for air-gapped DGX systems: reimaging from an ISO image copied to removable media, or updating from a local mirror of the NVIDIA and Canonical repositories. Container images are loaded manually rather than pulled from NVIDIA's registry.

Models follow the same pattern. Weights are downloaded on a connected system, checked against your security policy and transferred through your approved process. Plan who approves each transfer, how files are scanned and how often you update.

Vendors usually troubleshoot over a remote connection. In an air-gapped site, support happens on site or through logs your team exports after review. Ask how the vendor handles this, how quickly engineers can attend, and how replacement parts are delivered.

Because adding capacity later means another controlled installation, size with headroom for growth. GPU memory decides which models fit and how many people can use them; fractional GPU sharing lets several teams use one system. Facility power and cooling must be planned for 8-GPU servers, while a deskside NVIDIA DGX Station runs on standard office power for smaller sites.

Ask every vendor: