EXPERIMENT_011 // SPATIAL.BRAND.MIRROR
Your webcam feeds a generative art system that responds to your presence and motion in real time. The output is not a mirror — it is an abstract transformation. Movements become shape, colour, and rhythm on a p5.js canvas tuned to a specific visual palette. Diana builds this physical-digital feedback loop for experiential clients. This experiment shows it running in a browser tab, responding to whoever stands in front of it.
LOADING EXPERIMENT...
MolmoAct 2 (Allen Institute, 2025) demonstrated a 37× latency improvement in real-time vision-to-action feedback loops for robotics. The design insight — low-latency physical-digital coupling is now tractable — translates directly to a browser context. This experiment realises it using a readily available vision API: your webcam frames are analysed at up to 4fps by Gemini, which returns a coarse descriptor of your motion direction, shape quality, colour temperature, and presence level. A p5.js canvas maps those five fields to a flowing generative system keyed to Diana’s brand palette. The output reads as intentional. It responds to you. This is the brief Diana pitches to retail and experiential clients — a brand identity that reacts to physical presence rather than displaying a static logo.
HOW IT WORKS
The browser samples consecutive webcam frames at 250ms intervals using a hidden HTMLCanvasElement (not OffscreenCanvas — Safari compatibility). Each frame is downsampled to 80×60 greyscale pixels and compared against the previous frame via mean absolute difference. When motion exceeds a threshold (0.015), or when 4 frames have elapsed since the last API call (enforcing a 1fps floor regardless of motion), the current frame is captured at 320×240 pixels as a JPEG (quality 0.6) and sent to the analysis API. If a call is already in flight, the next trigger is skipped — no concurrent calls. When the scene is static for more than 10 frames, the canvas transitions to an idle state locally without an API call.
The captured JPEG is sent to POST /api/experiments/brand-mirror/analyse, which calls gemini-3.5-flash via @diabolicallabs/llm-client (Google provider). The model returns a FrameDescriptor: motion direction, motion intensity (0–1), dominant shape quality (angular, curved, scattered, dense, or sparse), colour temperature (warm, cool, neutral), and presence level (absent, peripheral, present, close). The schema is intentionally coarse — the minimum signal needed to drive the canvas with meaningful variation. API calls at 1–2fps with a 320×240 JPEG input run at approximately 60–120 calls per minute at peak. On API error, the canvas continues on the last known FrameDescriptor.
A p5.js particle system maps the FrameDescriptor to visual parameters: motionDirection drives the flow field vector; motionIntensity controls velocity and trail opacity; dominantShapeQuality determines whether particles render as sharp line segments, bezier trails, or scattered dots; colourTemperature tilts the active palette toward warm or cool anchor colours; presenceLevel governs canvas density. State transitions interpolate over 1.5 seconds. Four palette presets are selectable via pill buttons: Diana Brand (chartreuse, electric cyan, near-black), Monochrome, Warm (amber, rose, bone), and Dusk (deep violet, slate, pale gold). With prefers-reduced-motion enabled, the particle system is replaced with a static colour wash that still updates on new FrameDescriptor values.
WHAT THIS PROVES
A brand identity can respond to physical presence. Not a logo. Not a tagline. A living visual system with a consistent character that reacts to who is in front of it — their motion speed, direction, and proximity. This experiment demonstrates that the technical infrastructure for that experience now fits in a browser tab: WebRTC for capture, a vision API for scene interpretation, and a generative art system for output. The latency is perceptible (800ms–2s per analysis call) but bridged by continuous p5.js interpolation, so the canvas never freezes.
The coarseness of the FrameDescriptor is deliberate. Five discrete fields produce richer visual output than a continuous pose estimation stream because the generative system can interpolate between states smoothly. The API describes the scene; the canvas animates the gap. That separation — coarse structured signal + continuous generative interpolation — is the architectural pattern this experiment proves at production quality.