Web

@vidrip/mimic-web opens the camera, runs the face tracker in WebAssembly (on the GPU when the browser allows), and feeds the gesture engine. It works in any framework because it needs nothing but a page.

Requirements

A secure context: https:// or http://localhost. Browsers only allow camera access there.
A current Chrome, Edge, Firefox or Safari (desktop or mobile).
Your bundler can serve five static files (the tracker’s runtime and the face model).

Install

Configure access first, then:

npm install @vidrip/mimic-web @mediapipe/tasks-vision

Serve the runtime files

The tracker needs four WebAssembly files from @mediapipe/tasks-vision and the face model from @vidrip/mimic-web. Copy them into your static folder so they’re served from your own origin:

From	File
`node_modules/@mediapipe/tasks-vision/wasm/`	`vision_wasm_internal.js`, `vision_wasm_internal.wasm`, `vision_wasm_nosimd_internal.js`, `vision_wasm_nosimd_internal.wasm`
`node_modules/@vidrip/mimic-web/model/`	`face_landmarker.task` (about 4 MB)

With Vite, a ten-line plugin does it on every start; the web starter has one in vite.config.ts. Exclude @mediapipe/tasks-vision from dependency pre-bundling (optimizeDeps.exclude).

Bind gestures

import { Mimic } from '@vidrip/mimic-web';

const base = '/mimic/';
const mimic = await Mimic.create({
  modelUrl: `${base}face_landmarker.task`,
  wasm: {
    simd: { loader: `${base}vision_wasm_internal.js`, binary: `${base}vision_wasm_internal.wasm` },
    nosimd: { loader: `${base}vision_wasm_nosimd_internal.js`, binary: `${base}vision_wasm_nosimd_internal.wasm` },
  },
  preview: document.querySelector('video'),   // optional, shown mirrored
  blendshapes: true,                           // needed for eyes.up and expression.match
  on: {
    'blink':     (e) => { if (e.phase === 'start') fire(); },
    'smile':     (e) => setGrin(e.phase === 'start' ? e.strength : 0),
    'head.*':    (e) => { if (e.phase === 'start') steer(e.id); },
    'face.lost': (e) => { if (e.phase === 'start') pause(); },
  },
});

startButton.onclick = async () => {
  await mimic.start();                         // opens the camera: call it from a click on iOS
  const { profile } = await mimic.calibrate(); // about 2.5 s of a resting face
  if (profile) localStorage.setItem('mimic-profile', JSON.stringify(profile));
};

Mimic.create loads the tracker but doesn’t touch the camera. Call start() (or calibrate(), which starts it) from a user gesture, since iOS Safari requires one.

Calibration and profiles

Every gesture is judged relative to the player’s resting face, so nothing fires until calibration finishes. While calibrate() runs, mimic.calibration reports progress (0 to 1) and quality (noFace, offCenter, eyesClosed, mouthMoving, headMoving or good), which you can turn into prompts. See Calibration.

Store the profile and pass it back next session to skip calibration:

const saved = JSON.parse(localStorage.getItem('mimic-profile') ?? 'null');
const mimic = await Mimic.create({ …, profile: saved });

Laptops and left/right

By default (spec: 'auto') Mimic applies the browser’s left/right conventions and, on devices with a mouse, the laptop threshold set, which suits webcams further from the face. If a particular camera reports your own left as head.right, pass invert: { head: true } (or eyes: true) and tell us about the device.

The full web API reference.
The gesture catalog.

Web