Running a Video Encoder in the Browser: The Four Walls We Hit

SqueezeVid compresses video inside the browser. Roughly 59,500 jobs and 13 TB of input have gone through it. The interesting part was never that FFmpeg compiles to WebAssembly — that has been true for years. It is the four walls you hit when you try to run it as a product instead of a demo.

Wall one: WebAssembly has a memory ceiling, and a demo never finds it

WebAssembly linear memory is a single contiguous address space, and the bundled FFmpeg core ships with a fixed 1 GiB heap. Emscripten's default in-memory filesystem loads the entire input into that heap. A 90-second phone clip works fine. A 6 GB screen recording kills the tab.

The fix is not a bigger heap. It is never putting the file in the heap. We wrote a seekable Emscripten filesystem backed by OPFS synchronous access handles. The handles are acquired asynchronously in the worker before FFmpeg starts, because a sync access handle cannot be opened from synchronous code. After that, every read and write goes to disk through two fixed-size buffers, one for reading and one for writing. The encoded file never lands in the heap, including during MP4 faststart, which moves the moov atom to the front of the file and is the step most likely to try to hold an entire output in memory at once.

Two details cost us real time. First, no file offset may be coerced to a 32-bit integer anywhere in the path. A single | 0 truncates at 2 GiB and produces silent corruption rather than an error, which is the worst possible failure mode for a tool people trust with footage they cannot re-shoot. Second, OPFS does not accept views onto the shared wasm heap in every browser, so both buffers are ordinary ArrayBuffers and every transfer is a copy. That copy is the price of files above 2 GB working at all.

Wall two: multithreaded encoding costs you every third-party embed

Multithreaded FFmpeg in WebAssembly needs SharedArrayBuffer. Since Spectre, SharedArrayBuffer requires the document to be cross-origin isolated, which means two response headers:

• Cross-Origin-Opener-Policy: same-origin
• Cross-Origin-Embedder-Policy: require-corp

Send those and your encoder gets threads. You also lose the ability to embed any cross-origin resource that does not explicitly opt in through CORP or CORS. In practice that means embedded video tutorials stop rendering, third-party support widgets stop rendering, and advertising creatives from arbitrary advertisers stop rendering. A loader script may well survive on its own, because those are often served with Cross-Origin-Resource-Policy: cross-origin. The frames that loader then injects generally do not.

This is a product decision wearing the costume of a response header. We chose threads. The lesson worth passing on is that the choice exists, that you should make it deliberately rather than discover it when some third-party integration quietly fails to appear, and that the headers belong on the routes that actually run the encoder rather than across an entire domain.

Feature detection has to agree with the decision. Our check tests for WebAssembly and then for either SharedArrayBuffer or crossOriginIsolated. When it fails, the interface routes the visitor to explicit server processing up front, rather than failing cryptically part-way through a 2 GB file.

Wall three: picture size does not tell you the H.264 level

An H.264 level is not a function of resolution. It is chosen from macroblocks per picture, macroblocks per second, two aspect-ratio constraints, and a bitrate ceiling. Pick a level from resolution alone and 3840×2160 looks like Level 5.1. At 60 fps it is not: macroblocks per second exceeds the 5.1 budget and you need 5.2. Get it wrong and the encoder either rejects your configuration or, worse, accepts it and emits a stream that some decoders refuse to play.

Our plan builder walks the Annex A table with the frame rate included, and applies the two constraints that stop an unusually wide or tall frame from claiming a level it cannot use. If no level fits, the job falls back to the compatibility encoder instead of guessing. The reference implementation in FFmpeg's h264_levels.c is the clearest description of this we found.

One thing worth stating plainly, because the control looks like a quality slider: the native path uses a bitrate model, not CRF equivalence. It also caps against the source bitrate, so an already-small input never gets allocated more bits than it arrived with. Without that cap, "compression" occasionally produces a larger file, which is a very hard result to explain to somebody.

Wall four: devices lie, and pinning matters more than parallelism

navigator.hardwareConcurrency may be reduced by the browser. deviceMemory is optional, rounded and clamped. Neither reports free memory or CPU speed. We treat both as coarse scheduling hints only, sorting devices into four tiers that select a thread count and an I/O chunk size between 256 KiB and 4 MiB.

The non-obvious rule is the cap: threads are clamped to one fewer than the reported core count. Decoder and encoder pools share CPUs, and the core stalled when both sides were allowed to request the full reported number. Asking for fewer threads made encodes complete that had previously hung. Nothing here reserves a core. It only stops both halves of the pipeline from believing they own the machine.

The guarantee that made the rest shippable

A browser encoder can hang. WebAssembly can trap, a worker can die, and the browser can deny storage. The one thing a person must never have to wonder about is whether their original file survived.

So every engine operation — loading, mounting the input, preparing the output, executing, reading the result back and cleaning up — is wrapped in an activity watchdog. Log, progress and activity events refresh a timestamp. If 60 seconds pass with no activity at all, the attempt is rejected, the worker is terminated, and its credit is released so the next attempt starts on a fresh engine. The message a person sees is deliberately boring and always ends the same way: your file was not modified. That is literally true, because the input is only ever opened for reading.

Storage failures get their own messages rather than one generic apology, because the remedies genuinely differ. Running out of temporary browser storage means freeing disk space. An output exceeding what the browser will hold means using a smaller input or the server path.

One measured run

A single run on a single machine, so treat this as an existence proof and not a benchmark. Apple M3 Pro, Chrome 153, native browser beta, original resolution, strength 26:

• Input 8,673,132 bytes, output 2,527,381 bytes: about 70.9% smaller for this source.
• 2,079 ms from button click to completion, including the interface path.
• All 300 frames and the AAC audio stream survived a packet audit.
• A network recording captured no video upload. Small metadata requests were present.

We publish the fixture, checksums and environment alongside the explanation of how browser compression works, because a compression percentage with no source attached is not a claim anybody should accept.

Where the server still wins

Client-side processing is not a universal answer, and pretending otherwise would be dishonest. A phone is slower than dedicated hardware. Resizing still runs through FFmpeg because the native transform path is not validated yet. Files above the browser limits, unsupported codecs, missing colour metadata and browsers without cross-origin isolation all route to an explicit server upload, and that upload is always a stated choice rather than a silent fallback.

The honest summary: FFmpeg in the browser is a solved problem at demo scale and an engineering project at product scale. The walls are the memory model, the isolation trade-off, the codec level table, and the fact that device hints are only ever hints. You can try the compressor or read about the 10 GB browser beta, and both keep your video on your own device.