Workers and responsiveness

Native OpenCV operations are synchronous. A promise around the call does not move its computation off the UI thread. Create the OpenCV instance inside an ES module worker when processing frames interactively.

vision.worker.ts

import { COLOR_RGBA2GRAY, CV_8UC4, cvtColor, initOpenCV, Mat, matFromArray, toImageData } from '@banou/opencv-wasm'
import wasmUrl from '@banou/opencv-wasm/opencv_js.wasm?url'

await initOpenCV({ wasmUrl })
self.onmessage = ({ data }) => {
  using rgba = matFromArray(data.height, data.width, CV_8UC4, data.pixels)
  using gray = new Mat()
  cvtColor(rgba, gray, COLOR_RGBA2GRAY)
  const result = toImageData(gray)
  self.postMessage({ pixels: result.data, width: result.width, height: result.height }, {
    transfer: [result.data.buffer],
  })
}

main.ts

const worker = new Worker(new URL('./vision.worker.ts', import.meta.url), { type: 'module' })
const image = context.getImageData(0, 0, canvas.width, canvas.height)
worker.postMessage({ pixels: image.data, width: image.width, height: image.height }, [image.data.buffer])

Transferring the array buffer detaches the sender’s copy. Only transfer ordinary image buffers; a native matrix belongs to the WASM instance that created it.

Keep a bounded queue

For a live preview, process one frame at a time and retain at most the newest waiting frame. A queue of every captured frame turns temporary slowness into growing latency. Offline video processing may need a different policy that preserves every frame.

Workers isolate synchronous computation from rendering, but each OpenCV instance consumes its own heap. Multiple workers are not free native parallelism. Benchmark the memory and throughput of the actual pipeline before adding workers.

The image laboratory implements this pattern, including cleanup, error reporting and parameter changes while a frame is running.