Browser LLM Integration

Overview

The integration of language model inference capabilities into the INCOG Browser is designed around a strict separation between model capability and web content access. The browser provides inference as a browser-native service — analogous to how browsers provide spell-checking or form autofill — rather than as a web platform API freely accessible to arbitrary web content.

This architectural choice is deliberate. Making inference capabilities freely accessible to web content would create a powerful new data exfiltration surface: a malicious web application could silently submit sensitive page content, browsing context, and user inputs to a remote inference provider, bypassing all other privacy protections the browser enforces. By restricting inference access to a controlled browser API with user permission gating, the integration preserves the browser's overall privacy posture.

Architecture Layers

Layer 1: Inference Runtime

The inference runtime is a self-contained module within the browser's privileged process space. It:

• Manages model storage, loading, and execution
• Provides an internal IPC interface for the browser API layer
• Enforces execution isolation between the model and all web content
• Manages the context window for active inference sessions
• Handles mode selection (local vs. relay-routed) and execution accordingly

The runtime does not expose a JavaScript API directly. Web content cannot interact with the runtime except through the permission-gated browser API.

Layer 2: Browser Inference API

The browser exposes a JavaScript API to web content that provides:

// Request inference permission
const permission = await navigator.incog.inference.requestPermission({
  capability: 'text-generation',
  purpose: 'User-visible description of intended use'
});

// Submit inference request (requires granted permission)
const response = await navigator.incog.inference.generate({
  prompt: userPrompt,
  maxTokens: 512,
  temperature: 0.7
});

Each API call:

• Requires explicit user confirmation in the browser UI (not suppressible by web content)
• Is scoped to the active browsing context (cannot be used to query cross-context state)
• Does not expose the underlying model parameters or capabilities beyond the requested capability tier
• Does not allow access to inference history from prior requests

Layer 3: Browser Chrome Interface

The browser provides a first-party inference UI surface in the browser chrome (outside the web content area). This surface:

• Is not accessible to web content execution
• Provides unrestricted inference capability to the user
• Displays current inference mode and network routing status
• Allows users to export, share, or save inference results through explicit action

Model Registry

Model Selection Criteria

The model registry includes models selected for compatibility with the browser's privacy and technical constraints:

Technical criteria

• Compatible with WebAssembly and WebGPU execution model
• Parameter count appropriate for target hardware tier (sub-7B for consumer hardware, larger for hardware-accelerated environments)
• Quantization-compatible for reduced memory footprint (GGUF format for WASM runtime)

Privacy criteria

• Permissive licensing that allows local, private, unmodified deployment
• No telemetry or "phone home" mechanisms in model inference code
• Open weights with verifiable provenance

Quality criteria

• Benchmark performance appropriate for intended use cases
• Instruction-following capability for user-facing applications
• Sufficient context window for conversational use

Model Distribution

Models are distributed through a content-addressed distribution network. Each model is identified by its cryptographic hash, and the browser verifies this hash before using a downloaded model. This prevents model substitution attacks where a malicious distribution endpoint provides a different model than advertised.

Model updates are delivered as differential patches where possible to minimize download size. Users can opt out of automatic model updates and use specific model versions for reproducibility.

Inference Session Management

Session Lifecycle

An inference session corresponds to a user-initiated conversation within the browser's inference interface. Sessions:

• Are initialized when the user opens the inference interface or an application is granted inference access
• Maintain context (conversation history) in memory during the session
• Are terminated when the user closes the inference interface, the browsing context is closed, or the session times out
• Leave no persistent record of conversation content after termination

Context Window Handling

The context window — the maintained conversation history — grows with each exchange. When the context window approaches the model's maximum length:

1. The browser alerts the user that the context is near capacity
2. The user can choose to summarize and truncate the context (summary generated locally by the model)
3. Or start a new session with an empty context

The context window is never automatically transmitted to a remote service to extend its size — this would constitute an implicit data disclosure that the user may not have consented to.

Multi-Session Isolation

Multiple simultaneous inference sessions (e.g., different web applications each granted inference access, and the user's own browser chrome inference session) are fully isolated:

• Each session maintains an independent context window
• Sessions cannot read each other's context
• Inference results from one session are not visible to others
• Session state is not shared between origins

Privacy Properties of the Integration

What the Integration Prevents

The browser LLM integration is specifically designed to prevent the following privacy violations that are common in cloud-hosted inference services:

1. Silent prompt collection: Web applications cannot submit prompts without explicit user confirmation of each request
2. Cross-session behavioral profiling: Session isolation prevents inference providers (even with relay-routed mode) from accumulating a behavioral profile across multiple visits
3. Model-based fingerprinting: The model API does not expose implementation details that could be used to fingerprint the user's hardware or browser configuration
4. Inference result exfiltration: Applications cannot automatically transmit inference results to remote endpoints without user knowledge (results are returned to the application's JavaScript context, not to an external endpoint)

What the Integration Does Not Prevent

Application-layer collection: An application granted inference access receives the model's response as a JavaScript string. The application can subsequently transmit this response to any endpoint it chooses. The browser cannot prevent an application from submitting inference results to a server after receiving them. Users should only grant inference access to applications they trust.

User error: A user who includes sensitive personal information in a prompt submitted through relay-routed inference exposes that information to the inference provider, regardless of network-layer protections. The browser provides mode indicators to inform users of the current exposure level.

Future Protocol Extensions

Confidential Inference

As hardware confidential computing (secure enclaves, trusted execution environments) becomes more accessible in inference infrastructure, the protocol will support verifiable confidential inference: a mode in which inference executes in a hardware-attested enclave on the provider's infrastructure, with cryptographic guarantees that prompt content is not observable by the provider's software layer or personnel. This mode would combine provider-side privacy with hardware-accelerated inference capability beyond what local execution can provide.

Federated Fine-Tuning

A long-term extension of the LLM integration is federated model fine-tuning: allowing users to locally improve model performance on their specific use patterns without sharing fine-tuning data with a central party. Federated learning techniques allow model improvements to be aggregated across users in a privacy-preserving manner — improving model quality for all users without collecting individual user data. This capability requires significant additional protocol infrastructure and is a research-stage direction rather than a near-term deliverable.