Does Remio's AI run in the cloud?

No. Every AI capability in Remio runs on-device or on your host machine. Adaptive quality reads telemetry on the host and the client; the planned agent platform routes language-model commands through Remio's encrypted peer-to-peer tunnel; predictive input pre-renders frames locally on the client; MetalFX neural upscaling uses Apple's Neural Engine on the device that decodes the stream. Remio's relay never sees session content — it forwards DTLS-SRTP encrypted bytes when peer-to-peer is impossible, and it cannot decrypt them. There is no Remio cloud endpoint that touches screen pixels, keystrokes, or inference output.

Can I use Claude or GPT to control my Mac through Remio?

Not yet — this is the AI-agent platform capability, currently in research. The underlying infrastructure is already in place: ScreenCaptureKit captures host frames for the agent to read, UnifiedInputHandler injects native mouse and keyboard events into the host, and FlatBuffers carries commands at sub-millisecond parse time over Remio's encrypted data channel. What is missing is a stable API surface for an agent to call. The agent platform will let any language model that supports tool use — Claude, GPT-4o, Gemini, local models — drive a real desktop through the same encrypted connection a human user gets.

What is adaptive quality and how does it differ from manual quality settings?

Adaptive quality is the only AI capability currently shipping in Remio. It reads three signals — network bandwidth from WebRTC bandwidth estimation, on-screen content type via the encoder content-type hint, and battery level on the client — then tunes encoder bitrate, codec choice, and resolution on the host in real time. Manual settings ask the user to pick a fixed bitrate and codec at session start; adaptive quality re-tunes every few hundred milliseconds without any UI interaction. The result is smoother fallback when Wi-Fi degrades, longer battery on the client, and crisper text when the host frame contains a document instead of motion.

When will predictive input ship in Remio?

Predictive input is in research — no shipping date yet. The technique pre-renders the local cursor position and keystroke echo before the host frame arrives over the network, masking 20 to 60 milliseconds of round-trip latency. The hard problem is rollback when the prediction does not match the host frame — a misprediction has to fade out smoothly enough that the eye does not catch the correction. Predictive input is grouped in the same research phase as the agent platform and MetalFX neural upscaling. Adaptive quality ships first because it is the only capability that does not need on-device model inference.

Does Remio's AI collect screen data for training?

No. Remio does not collect screen data, keystrokes, or inference output for training or any other purpose. The adaptive quality model that ships today runs on the host with no telemetry sent to Remio. The planned on-device models for predictive input, neural upscaling, and behavioural security all execute inside the client app sandbox and never write screen content to disk. The agent platform routes commands through the encrypted peer-to-peer channel; the language model that drives it is whichever model you connect to (Anthropic, OpenAI, Google, local), and Remio does not see the prompts or responses. The relay sees only opaque encrypted bytes.

AI Remote Desktop — Agents, Adaptive Quality

The six capabilities

One shipping, three in research, two planned.

A working remote desktop has six places where AI can earn its keep: the encoder, the agent layer, the input loop, the decoder, the voice path, and the session signal. Remio is building one capability into each of those six layers — status below is honest, not aspirational.

Building now 1 of 6

Adaptive quality — encoder tuning

Reads available bandwidth, on-screen content type, and client battery level every few hundred milliseconds. Re-tunes bitrate, codec, and resolution on the host without any UI interaction. The only AI capability that ships today.

Live bandwidth read H.265 / AV1 Host-side Zero config

Read deep dive →

In research 3 of 6

Agent platform — AI that drives your desktop

Lets a language model see and drive a real desktop through Remio's end-to-end encrypted connection. Most of the plumbing exists: native screen capture feeds frames to the agent, native input injection moves the mouse and keyboard, and a fast command channel carries actions at sub-millisecond parse time. Missing: a stable agent-facing surface for Claude, GPT, Gemini, and local models.

Native screen capture Native input injection Fast command channel End-to-end encrypted

Read deep dive →

Predictive input — local render-ahead

Pre-renders the cursor and keystroke echo on the client before the host frame arrives, masking 20–60 ms of round-trip. Hard part is rolling back gracefully when the host frame disagrees with the prediction — the correction has to fade smoothly enough that the eye misses it.

16 ms prediction Client render Smooth rollback Stream-aware

Neural upscaling — sharper at lower bitrate

Stream at a lower native resolution and upscale on the client using Apple's MetalFX spatial scaler and the Neural Engine. Target is meaningful bandwidth reduction with imperceptible quality loss on text and UI surfaces — precise numbers depend on content type.

MetalFX Neural Engine On-device Client-side

Read deep dive →

Planned 2 of 6

Voice control — speak to your remote desktop

Speak commands to your remote host — open an app, switch a Space, take a screenshot. Speech-to-text runs on-device using the platform's native dictation engine; recognised intents route through the same keyboard and dock paths the touch UI already uses.

On-device speech-to-text Intent routing Dock actions Keyboard actions

Behavioural security — session anomaly

Learns the shape of your normal sessions — typical client devices, typical hours, typical app patterns — and flags drift. Baseline lives on the host. New geographies or unusual session timing surface as an alert before any session data moves.

On-host baseline Geo anomaly Time drift Session entropy

Why native matters

AI lives in the pipeline, not in a sidecar.

Bolting AI onto a web-wrapped remote desktop is window dressing. Remio is being built so the model runs where the frame is — on the same Apple Neural Engine and on-device acceleration the stream already uses. The result is fewer hops, no cloud round-trip, and no screen data leaving the device that owns it.

Remio

Native intelligence in the pipeline

AI is integrated into capture, transport, and render — not a sidecar process. Same on-device frameworks the streaming engine already uses.

✓On-device Neural Engine and on-device AI

✓MetalFX spatial upscaling on-device

✓Sub-millisecond command parsing

✓Direct, end-to-end encrypted bridge for AI agents

✓Zero screen data leaves your devices

✓Native screenshot, input, and app-launch hooks

Web-wrapped remote desktops

AI as a bolt-on sidecar

Wrapped browsers and cross-platform UI toolkits push every request to a cloud endpoint, lose access to Apple frameworks, and pay a latency tax on every command.

×Cloud-side AI → screen data leaves device

×No access to MetalFX or Neural Engine

×Slow text-over-browser → tens of ms per command

×Relay-only path for agent traffic

×Usage data returns to vendor cloud by default

×Screenshot and input via accessibility shims

Roadmap

Three phases, no vapourware.

Each phase brings one more capability into the streaming pipeline. Status mirrors the board above — this section adds the time axis. Phases sequence rather than overlap so each capability gets a real shipping window.

Phase 01 — 2026 · in development

Encoder intelligence

Adaptive quality. Bitrate, codec, and resolution tuned in real time from network bandwidth, on-screen content type, and client battery level. Host-side, zero configuration. Currently rolling out across Mac and Windows hosts.

Active

Phase 02 — 2026–2027 · research

Pipeline intelligence

Agent platform for Claude, GPT, Gemini, and any tool-using model. Predictive input that masks 20–60 ms of round-trip locally. Neural upscaling for sharper frames at lower bitrate. Each capability ships independently as it stabilises — no monolithic release.

Planned

Phase 03 — 2027 · roadmap

Interface intelligence

Voice control mapped to host-side actions through the same dock and keyboard paths the touch UI uses. Behavioural session security that learns the shape of normal sessions on the host and flags geographic, temporal, or pattern drift before screen data moves.

Roadmap

Six AI capabilities, one streaming pipeline.

One shipping, three in research, two planned.

Adaptive quality — encoder tuning

Agent platform — AI that drives your desktop

Predictive input — local render-ahead

Neural upscaling — sharper at lower bitrate

Voice control — speak to your remote desktop

Behavioural security — session anomaly

AI lives in the pipeline, not in a sidecar.

Native intelligence in the pipeline

AI as a bolt-on sidecar

Three phases, no vapourware.

Get adaptive quality today, the rest as it ships.