Private Inference

Overview

The INCOG LLM module provides a privacy-preserving inference runtime for language model capabilities within the INCOG Browser. It addresses a specific and emerging category of privacy risk: the systematic behavioral profiling enabled by large language model service providers who retain, analyze, and in some cases commercially exploit the content of user-submitted prompts.

When a user submits a query to a cloud-hosted language model, the following information is typically observable by the service provider:

• The exact text of the prompt, including all contextual information the user included
• The timestamp of the query
• The session context linking the query to prior interactions
• In authenticated sessions, the user's account identity and aggregate query history
• Network metadata linking the query to the user's IP address

This information, accumulated at scale, constitutes an extraordinarily rich behavioral profile: what questions a user asks, what problems they are trying to solve, what information they are seeking, and what topics they engage with over time. For users whose queries involve sensitive personal, financial, medical, or political content, the retention of this data by a centralized third party represents a substantial privacy risk.

---

Inference Architecture

Three Execution Modes

The INCOG LLM module supports three inference execution modes, selectable based on the user's privacy requirements, hardware capability, and latency tolerance:

Mode 1: Local Inference
The language model is downloaded and executed entirely on the user's device, within the INCOG Browser's isolated execution environment. No query data leaves the device. This mode provides the strongest privacy guarantees but requires sufficient local hardware capability (CPU/GPU, RAM) and model download.

Mode 2: Relay-Routed Inference
Queries are submitted to a compatible inference provider through the INCOG relay network, preventing the provider from observing the user's IP address. The provider still observes query content, but cannot link it to the user's real identity. This mode is appropriate when local hardware is insufficient and the user accepts query content exposure to the provider while requiring network-layer anonymity.

Mode 3: Private Relay Inference
Queries are submitted through the relay network to inference providers that operate under strict zero-retention policies and verify this through cryptographic means (where technically feasible). This mode combines network-layer anonymity with provider-side data minimization.

Mode Selection

Mode selection is configurable per session and per query. The default mode is determined by the user's hardware profile:

• Hardware capable of running ≥7B parameter models at acceptable performance: defaults to local inference
• Hardware capable of running smaller models: defaults to local inference with smaller models, with relay-routed fallback for queries requiring larger model capability
• Insufficient hardware: defaults to relay-routed inference with notification of content exposure

---

Local Inference Runtime

Model Execution Environment

Local inference executes within the INCOG Browser's sandboxed execution environment, using WebAssembly (WASM) and WebGPU where available for hardware-accelerated inference. The execution environment:

• Is isolated from web content contexts — models cannot access or be accessed by arbitrary web applications
• Does not retain inference state between sessions — context windows are cleared at session end
• Does not write inference results to persistent storage without explicit user action
• Does not transmit any inference data over the network in local mode

Model Management

Models are downloaded through a signed distribution channel and verified against published hashes before use. Model files are stored in the browser's isolated storage and are not accessible to web content. Model updates are applied through the same signed distribution channel.

The model library is curated to include models appropriate for local inference on consumer hardware. Models are selected for:

• Compatibility with WASM/WebGPU runtime
• Acceptable inference quality at hardware-constrained sizes
• Licensing that permits local, private deployment

Context Management

Local inference context (the conversation history maintained between a user and the model within a session) is stored exclusively in memory during the session. The context:

• Is not written to disk
• Is not shared across tabs or browsing contexts
• Is not retained when the session ends
• Cannot be accessed by web content through any API

Users can optionally export conversation history to local storage through explicit action, but this export is not automatic. The default behavior produces zero persistent record of inference interactions.

---

Data Exposure Analysis

What the Local Inference Mode Does Not Expose

In fully local inference mode, zero query data is transmitted outside the user's device. The following are not observable by any external party:

• Query content (prompts)
• Model responses
• Conversation history
• Inference timing (beyond network-observable session activity)
• Topics or subjects of queries

Residual Exposure in Relay-Routed Mode

In relay-routed inference mode, the inference provider observes query content and response. The relay network prevents IP attribution, but the provider has full access to:

• The text of each query
• The text of each response
• Aggregate session statistics (query count, session duration)

Users selecting relay-routed mode should understand that this mode provides network-layer anonymity only — not content privacy relative to the inference provider.

---

Browser Integration Architecture

Isolation from Web Content

The INCOG LLM module is accessible from web applications only through a defined browser API that enforces:

• Explicit user permission for each application's model access request
• Scoped access to a specific model capability tier (not all available models)
• No access to inference history from other applications or sessions
• User control over each inference request (applications cannot batch-submit queries without individual user confirmation)

This isolation ensures that the LLM capability cannot be exploited by web applications as a data exfiltration mechanism or as a cross-context tracking signal.

In-Browser UI Integration

The browser provides a first-party inference interface accessible via a dedicated browser surface (not embedded in web page content). This interface:

• Renders in a browser chrome context isolated from web page execution
• Provides mode selection controls visible to the user
• Displays network routing status when relay-routed mode is active
• Does not expose the underlying model or inference API to web page context

Applications wishing to use model inference must use the defined browser API with the user's explicit consent, rather than having direct access to the inference surface.