Stream less, see more: how on-device super-resolution cuts remote desktop bandwidth in half

The bandwidth problem nobody talks about

Remote desktop has a dirty secret: it is a bandwidth hog.

Streaming a 1080p desktop at 60 fps takes roughly 8 to 15 Mbps with modern codecs. That is fine on fiber. It is fine on strong home Wi-Fi. But step onto hotel Wi-Fi, tether from your phone, or try to work from a coffee shop in a busy city — and everything falls apart.

The current solution is depressingly simple. When bandwidth drops, reduce quality. Lower the resolution. Drop the framerate. Accept blurry text and choppy video until conditions improve. Every remote desktop app does this. It has been the only option for twenty years.

What if there were a better way?

What if, instead of sending a crisp 1080p stream and degrading it when conditions get rough, you could send a small 1080p stream all the time — and make it look like 4K on the other end?

That is AI super-resolution. And it is not science fiction. Gaming has been doing it for years.

The super-resolution streaming pipeline

The translation from gaming to remote desktop is surprisingly clean. In gaming, you render at low resolution and upscale. In streaming, you encode at low resolution and upscale. Four stages, left to right, end to end.

Same perceived sharpness. Roughly a quarter of the bandwidth. And because the host is encoding a smaller image, it works less too — lower CPU usage, less heat, more battery on laptops.

The gaming industry already solved this

In 2018, NVIDIA introduced DLSS — Deep Learning Super Sampling. The concept was almost too good to believe: render a game at lower resolution, then use a neural network to upscale it to look like native resolution. Less work for the GPU, same visual quality.

Gamers were skeptical. How could an AI-generated image look as good as natively rendered pixels? But the results spoke for themselves. By 2026, DLSS 4.5 uses transformer models that produce images many players actually prefer to native rendering — sharper edges, better temporal stability, fewer artifacts.

AMD followed with FSR. Apple introduced MetalFX. Intel shipped XeSS. The entire industry converged on the same insight:

The smartest pixel is the one you never have to render.

It took the gaming industry about five years to go from "this is a gimmick" to "this is mandatory." Remote desktop has not even started.

On-device AI, private by design

Here is where it gets really interesting — and where we differ from how you might expect AI to work.

The super-resolution model runs entirely on your device. Not in the cloud. Not on some server. On the phone or laptop in your hands.

Modern devices are absurdly powerful for on-device AI. Apple's Neural Engine on M-series chips handles 15.8 trillion operations per second. Qualcomm's Hexagon NPU on recent Snapdragon chips sits in the same league. These dedicated AI processors sit idle most of the time. We are putting them to work.

The inference happens in the rendering pipeline, between decode and display. The model sees a 1080p frame and outputs a 4K frame. Total added latency? Under 3 milliseconds. You will never notice it.

And because everything runs locally:

• No data leaves your device — your screen content stays private
• No internet required for AI — the model works offline
• No subscription for AI features — it is built into the app
• No server costs — scales infinitely because every user brings their own compute

This aligns with Remio's privacy-first philosophy. We do not want your data. We do not even want to see your screen content. On-device AI lets us deliver cutting-edge features while knowing literally nothing about what you are doing.

Why this only works on native apps

Here is the part that ties everything together — and why we wrote an entire post about going native.

To run a neural network in the rendering pipeline without adding visible latency, you need:

1. Access to the Neural Engine or NPU. This requires native APIs — CoreML on Apple, NNAPI on Android. There is no web API for the Neural Engine. Electron apps cannot reach it.
2. GPU pipeline integration. The upscaled frame must move directly to the display without extra copies. Metal and Vulkan let you do this. WebGL does not give you this level of control.
3. Sub-3 ms timing. Every extra millisecond matters in a real-time streaming pipeline. JavaScript's garbage collector alone can introduce unpredictable pauses. Native code gives deterministic timing.

An Electron app would need to decode the video in Chromium's media pipeline, extract the frame to JavaScript, somehow invoke the Neural Engine (impossible without native bindings), get the upscaled frame back, and display it through Chromium's compositor. Each step adds latency and complexity. The total overhead would likely run 15 to 25 ms — too slow for real-time use.

Super-resolution in the streaming pipeline is not just an AI problem. It is a systems engineering problem. And systems engineering demands native code.

This is why, two years into building Remio, no Electron-based remote desktop has shipped AI upscaling. It is not that they do not want to. The architecture will not let them.

The technical challenges, and how we are solving them

We will not pretend this is easy. There are real challenges.

Model latency. The model must run in under 3 ms per frame at 60 fps. That leaves zero room for bloated architectures. We are using optimized convolutional models — not transformers, they are too slow for per-frame inference — specifically tuned for the Neural Engine's execution units.

Content diversity. Remote desktop content is wildly different from gaming content. Text, spreadsheets, code editors, video playback, design tools — the model needs to handle all of it. We are training on datasets that specifically include desktop UI elements, not just natural images.

Temporal stability. Frame-by-frame upscaling can cause flickering. Gaming solved this with temporal accumulation — reusing information from previous frames. We are exploring similar techniques, feeding motion vectors from the video codec into the upscaling model.

Power efficiency. Running a neural network 60 times per second on a phone sounds expensive. But the Neural Engine is designed for exactly this — it is more power-efficient than the GPU for matrix operations. Early tests show under 5 percent additional battery drain.

The honest challenge: why this is harder than gaming

We have talked about DLSS and gaming super-resolution as inspiration — and they are. But intellectual honesty demands we explain why remote desktop upscaling is a fundamentally harder problem.

Gaming DLSS has unfair advantages. Game engines provide the AI with rich data that does not exist in our world:

• Motion vectors. The engine knows exactly where every object moved between frames. We get compressed video with no scene-level motion data.
• Depth buffers. DLSS knows which pixels are near and which are far. We see a flat 2D image.
• Temporal history. DLSS accumulates detail across multiple frames using engine data. We work from individually compressed frames that may carry different artifacts each time.
• Dedicated hardware. NVIDIA's Tensor Cores deliver 384+ TOPS specifically optimized for this workload. Mobile Neural Engines are powerful but serve a different purpose.

In other words, gaming super-resolution is a guided reconstruction with rich auxiliary data. Remote desktop upscaling is blind super-resolution from lossy, compressed video — a much harder starting point.

This does not mean the approach is wrong. Even modest improvements in visual quality at lower bitrates deliver real bandwidth savings. But we want to be transparent: the quality ceiling for blind upscaling is lower than what DLSS achieves with full engine integration. We are building something genuinely useful, not claiming parity with a different technology.

What this means for you

When AI super-resolution ships in Remio, you will not need to think about it. There is no toggle, no setting, no "AI mode." The app will automatically:

• Detect your available bandwidth
• Choose the optimal encode resolution
• Upscale on-device to match your screen
• Adapt in real time as conditions change

On fast connections, it may not need to upscale at all. On slow connections, it will be the difference between "unusable" and "feels like I am sitting in front of my computer."

That is the goal. Not AI you have to think about. AI that just makes everything work better, silently, privately, on the device you already own.

We are building it right now. And we are building it alongside a whole suite of AI features that will make Remio the most intelligent remote desktop app on the planet.

Stream less. See more. That is the future.