About
CONNECTING // WEST STREET @ W23 STXWalk Keyboards used traffic camera video feeds to transform crosswalks into piano keyboards. As pedestrians stepped on the white stripes of the crosswalk, the app plays the corresponding notes.
After running with live detection for a few months, the live app is in retirement. To view the experience, see the video on the homepage.
HOW IT STARTED
This project won first place at AI Tinkerers' NYC Vision Hack v.2 in Aug 2026. The challenge: leverage AI, computer vision via Roboflow, and deploy via Google Cloud Run to transform raw NYC data into intelligent vision agents.
HOW IT WORKS
The web app used Roboflow to detect pedestrians in a traffic cam video in real time. An AI agent running on Cloud Run orchestrated a calibration cycle and used Roboflow to detect crosswalk stripes on regular intervals. When the web app detected a pedestrian inside a crosswalk stripe boundary, it played the note corresponding to the stripe's position on a piano keyboard.
MORE DETAILS
Get more details on how the app worked at vinceallen.com!