GELU (GESTOLUMINA)
A high-fidelity sensory expansion wearable engineered for Deaf & Hard-of-Hearing musical agency. Combines on-silicon 16kHz I2S FFT audio analysis, vibration-matched RGB illumination, and directional bi-directional H-bridge haptics.
Vibration-Matched Illumination
One of the hardest challenges in haptic engineering is demonstrating physical tactile vibrations on video. If you record a bracer vibrating against someone's wrist, the camera sees virtually nothing.
To solve this, I designed GeLu's addressable RGB lighting engine to synchronize precisely with the incoming frequency harmonics and motor attack envelope. When heavy sub-bass hits, the LEDs pulse in deep hues matching the tactile surge. When higher acoustic transients arrive, the LEDs flash in rapid harmony.
This design breakthrough allowed both Deaf performers and sighted audiences to share the same intuitive sensory experience, directly catalyzing my first peer-reviewed academic publication at NIME 2024.
GeLu Simulator
Test the GeLu DSP engine directly in your browser. This simulator replicates the exact 16kHz FFT frequency splitting, transient detection, dual-mode FastLED pixel illumination, and bi-directional motor gating running inside Dillon's ESP32 firmware (ESP32HapticControl_INMP441).
Firmware Architecture: ESP32HapticControl_INMP441
Located in public/ESP32Codes/PlatformIO/02_Audio_Haptics/ESP32HapticControl_INMP441, this firmware is written in C++ for PlatformIO and represents the production evolution of Dillon's wearable haptic systems.
| Subsystem | Hardware Pin | Firmware Implementation | Operational Role |
|---|---|---|---|
| I2S Audio Input | SCK: 20, WS: 10, SD: 21 | ESP32 I2S DMA Driver (I2S_NUM_0) | Continuous 16,000 Hz, 32-bit DMA audio capture from INMP441 omnidirectional MEMS mic. |
| FFT DSP Engine | On-Silicon Core | arduinoFFT<double> (256 Samples) | Decomposes audio frames into Bass (<250Hz), Mids (250–2000Hz), and Treble (>2000Hz) band energy. |
| Addressable LEDs | GPIO 1 | FastLED WS2812 Controller (5 Pixels) | Visualizes frequencies with asymmetric attack/release filtering to match tactile haptics. |
| Flexible Noodle LED | GPIO 8 | ESP32 Hardware LEDC PWM (5 kHz) | Provides continuous ambient luminance proportional to overall audio magnitude. |
| Bi-Directional H-Bridge | FWD: 9, REV: 7 | Dual-Channel Complementary PWM | Enables Forward, Reverse, and Alternating polarity vibration for distinct tactile sensations. |
| Captive Portal AP | Internal Wi-Fi Radio | WebServer + DNSServer | Hosts Wi-Fi network DillonSimeoneGeLu with captive redirect for live smartphone parameter tuning. |
Transient Kick Detection Logic
The firmware compares the instantaneous audio band energy against a smoothed exponential moving average. When a sharp attack occurs (such as a snare strike or bass drop where energy / smoothEnergy > transientThreshold), the system injects an immediate transientBoostDuty burst for 150ms before returning to normal power-curve tracking. This prevents haptic lag and gives percussive beats crisp physical punch.
The Evolution of GeLu
From conceptual research at CymaSpace under Universal Music Design to independent production, custom PCBs, and wearable field deployments.
Initial wearable exploration pairing wireless ESP32-S3 microcontrollers with embedded TensorFlow Lite gesture machine learning models and preliminary vibrotactile nodes.
Iterative custom PCBs, integrated thumbpad capacitive triggers, tactile motor drivers, and Bambu Lab ergonomic wrist enclosures for live musical performances.
Commercial-ready refinement featuring the 16kHz I2S INMP441 FFT engine, vibration-matched RGB illumination, and standalone captive portal configuration.
Hardware hacker Andrew "bunnie" Huang testing the GeLu Prototype 3 at Teardown Portland. Engineers praised its ability to deliver seamless ambient awareness of acoustic spaces.