FrameFind
corereactutils

Face signals,
processed where they happen.

Add gaze, blink, head-pose and face-attribute signals to your product without streaming camera frames to a server.

Try the live demoGitHub
Data stays on-device
Sub-2 ms model inference*
WASM · WebGPU · Node.js

*ONNX model-only benchmark; full frame latency depends on the device and runtime.

import { useRef } from 'react';
import {
  useGlassesDetector,
  useHeadPoseDetector,
  useBlinkDetector,
} from '@framefind/react';

export function App() {
  const videoRef = useRef(null);

  const glasses  = useGlassesDetector({ videoRef });
  const headPose = useHeadPoseDetector({ videoRef });
  const blink    = useBlinkDetector({
    videoRef,
    onBlink: (ear) => console.log('blink!', ear),
    onFaceLost: () => console.log('face lost'),
  });

  return (
    <div>
      <video ref={videoRef} autoPlay playsInline muted />
      {glasses.loading && <p>Loading…</p>}
      {glasses.result  && <p>Glasses: {glasses.result.glasses ? 'yes' : 'no'}</p>}
      {headPose.result && <p>Yaw: {headPose.result.yaw.toFixed(1)}°</p>}
      {blink.result    && <p>Blinking: {blink.result.isBlinking ? 'yes' : 'no'}</p>}
    </div>
  );
}
$npm install @framefind/core @framefind/react
Frame 02 / Modular detectors

Pick the signal your interface needs.

All detectors live in @framefind/core with React hooks in @framefind/react. Choose the browser or Node.js runtime and keep the integration focused on the signals you actually use.

Available now

GlassesLive
@framefind/core

Identify whether a face is wearing glasses and return a calibrated confidence score.

Eye-region crop → 6.1 MiB ONNX classifier → smoothed probability
Head PoseLive
@framefind/core

Track yaw, pitch and roll to understand how a face is oriented.

MediaPipe landmarks → solvePnP → ZYX Euler angles
BlinkLive
@framefind/core

Detect blink events with adaptive per-eye calibration and temporal smoothing.

Blendshapes + EAR geometry + asymmetry → drop-rate gate → blink event
MaskLive
@framefind/core

Classify a face as masked, unmasked or wearing a mask incorrectly.

Face-bbox crop → 112×112 ONNX classifier → softmax(with/without/incorrect)
GazeLive
@framefind/core

Estimate gaze direction and map it to a normalized screen region.

Iris landmarks → eye-bbox ratio → head-pose compensation → gaze vector + region

On the roadmap

LivenessPhase 2
@framefind/core

Anti-spoof challenge for KYC and onboarding flows

Blink + head turn + smile challenge → texture analysis → liveness score
TalkingPhase 2
@framefind/core

Mouth-open detector for meeting UX and speaker indicators

Lip landmarks → Mouth Aspect Ratio → temporal gate → talking event
DrowsinessPhase 2
@framefind/core

Fatigue scoring from blink rate, yawns, and eye closure

Blink events + PERCLOS + yawn rate → temporal window → drowsiness score
AttentionPhase 2
@framefind/core

Engagement score from head pose and eye state

abs(yaw) < 20° && abs(pitch) < 15° && eyesOpen → 0–1 score
Pulse (rPPG)Phase 3
@framefind/core

Heart-rate estimation from facial micro-color changes

Forehead/cheek ROI → temporal RGB signal → POS / CHROM → BPM
Frame 03 / Why FrameFind

A clearer foundation for camera features.

Real-time interaction needs local feedback

A network round-trip on every frame adds latency where it matters most. Local inference keeps feedback immediate and responsive.

Camera data deserves a smaller surface

FrameFind processes video in the browser or on your server-side runtime. Raw camera frames are not sent to a FrameFind service.

Use only the signals you need

Detectors are modular and runtimes remain peer dependencies, so you can choose the browser or Node.js path that fits your application.

A small, typed integration

Use the core classes directly or start with the React hooks. Every detector returns predictable typed results with faceDetected and confidence data.

Frame 04 / How it works

From frame to signal, locally.

FrameFind detects landmarks once, derives the signal you need and returns a typed result to your application.

  1. Input frame

    Camera · image · Node.js

  2. Face landmarks

    Shared tracking layer

  3. Signal extraction

    Eyes · face region · pose

  4. Local inference

    ONNX · geometry · WASM

  5. Typed result

    Class · angle · event

5

live detectors

0 bytes

data sent to server

112²

ONNX input pixels

Frame 05 / Quick start

From install to first result.

Choose your runtime, initialize a detector and start reading typed results.

1

Choose a runtime and install the package

Core + React
bash
npm install @framefind/core @framefind/react onnxruntime-web
Node.js
bash
npm install @framefind/core onnxruntime-node
2

Create the detector

tsx
const { videoRef, result, loading } = useGlassesDetector();

Or use the vanilla API: new GlassesDetector()

3

Read the result in your application

tsx
if (result?.glasses) {
  console.log(`Glasses! ${(result.probability * 100).toFixed(1)}% confidence`);
}
import { useRef, useEffect } from 'react';
import { useGlassesDetector, useBlinkDetector } from '@framefind/react';

export default function App() {
  const videoRef = useRef<HTMLVideoElement>(null);

  const { result: glasses, loading } = useGlassesDetector({
    videoRef,
    threshold: 0.35,
  });

  const { result: blink } = useBlinkDetector({
    videoRef,
    onBlink: (ear)      => console.log('blink!', ear),
    onFaceLost: ()      => console.log('face lost'),
    onEARChange: (ear)  => console.log('ear:', ear),
  });

  useEffect(() => {
    let stream: MediaStream;
    navigator.mediaDevices
      .getUserMedia({ video: true })
      .then((s) => {
        stream = s;
        if (videoRef.current) videoRef.current.srcObject = s;
      });
    return () => stream?.getTracks().forEach((t) => t.stop());
  }, []);

  return (
    <div className="max-w-sm p-4">
      <video ref={videoRef} autoPlay playsInline muted className="w-full" />
      {loading && <p>Loading model…</p>}
      {glasses && (
        <p>Glasses: {glasses.glasses ? 'Yes' : 'No'} ({(glasses.probability * 100).toFixed(1)}%)</p>
      )}
      {blink && (
        <>
          <p>State: {blink.isBlinking ? 'closed' : 'open'}</p>
          <p>EAR: {blink.smoothedEar?.toFixed(3) ?? '—'}</p>
          <p>Baseline: {blink.baselineEar?.toFixed(3) ?? '—'}</p>
        </>
      )}
    </div>
  );
}

Frame 06 / Use cases

Signals that fit real products.

Use the detector that matches your interface, with a processing model that respects the camera frame.

EdTech

Study interfaces

Build optional gaze and blink signals into learning tools without uploading classroom video.

Telemedicine

Remote assessments

Use local face signals to support guided assessments while keeping sensitive video on the device.

Gaming

Hands-free controls

Turn gaze and head pose into lightweight interaction signals for browser-based experiences.

Accessibility

Accessible interfaces

Prototype gaze-assisted navigation and alternative controls with a local processing path.

Automotive

Attention-aware tools

Explore blink, gaze and head-pose signals in controlled, user-consented interfaces.

Video Conferencing

Meeting experiences

Add optional presence and interaction cues without making raw video part of your backend.