Architecture: ESP32 I2S Audio Runtime#

Overview#

The robot-face firmware uses the ESP32's two I2S peripherals to capture microphone audio and play voice-agent responses without blocking the 30 fps face renderer.

• I2S0 is dedicated to receive-only capture from an INMP441 microphone.
• I2S1 is dedicated to transmit-only playback through a MAX98357A amplifier.
• DMA moves samples between each peripheral and memory.
• Two FreeRTOS tasks perform chunk-level capture and playback work on core 0.
• Audio.h is the only public firmware boundary that exposes audio-driver behavior.

For the electrical rationale and wiring, see ESP32 Voice Hardware. For the render-loop constraint this architecture protects, see ESP32 Parametric Face Animation.

---

I2S Framing#

I2S is a fixed-rate PCM audio stream rather than a general-purpose bus. Standard Philips-format I2S uses three signal roles:

One word-select period always contains a left slot followed by a right slot. The INMP441 and MAX98357A are used as mono devices, but mono operation is a slot-selection policy rather than a different wire format.

One WS period is one stereo frame, always — left slot then right slot — even though both peripherals are mono. Leaving a slot silent is a configuration choice, not the bus format changing.

The microphone is configured with I2S_STD_SLOT_LEFT, matching its L/R pin tied to ground. The amplifier receives the playback representation expected by its mono mixing behavior.

---

Peripheral and Task Boundaries#

The classic ESP32 exposes two independent I2S peripherals with separate DMA engines. This project assigns one direction to each peripheral so listening and speaking can operate simultaneously without reconfiguring a shared channel.

Capture is hardware-paced and needs no buffering. Playback arrives over the network in bursts, so it is queued through a RingBuffer sized for half a second before playbackTask drains it at a steady rate.

AppController connects audio lifecycle to VoiceSession. The selected Agent implementation receives microphone chunks. When firstAudio arrives from Agent or StubAgent, the controller can stop diagnostic output and begin normal playback.

---

Capture Path#

captureTask reads DMA-paced samples from I2S0 at 16 kHz. The INMP441 places a signed 24-bit sample at the most-significant end of a 32-bit slot. The task keeps the top 16 bits with the source expression (int16_t)(raw[i] >> 16).

This is a bit-selection operation, not floating-point rescaling. It discards lower precision and retains the signed high-order portion expected by the downstream speech interface without adding rounding work or bias.

Each delivery contains 320 samples:

320 samples / 16,000 samples per second = 0.020 seconds = 20 ms

Capture is hardware-paced and forwarded immediately through Agent::sendAudio(). It does not need a jitter buffer because the local microphone and I2S clock produce samples steadily.

---

Playback Path#

Voice-agent audio can arrive over the network in bursts even though I2S1 must transmit at a steady 24 kHz. queuePlayback() therefore writes incoming samples to a RingBuffer. playbackTask drains that buffer in chunks of 240 samples.

The configured buffer holds 12,000 samples:

12,000 samples / 24,000 samples per second = 0.5 seconds

Before transmission, scaleVolume() limits each sample according to VOICE_VOLUME_PERCENT, which defaults to 35. This is a project safety ceiling; the I2S protocol itself has no volume concept, and the amplifier's GAIN pin does not replace per-sample limiting.

An active I2S transmitter cannot pause its clock while waiting for the network. If the ring buffer underruns, playbackTask writes zero-valued samples. During diagnostics it can write the configured test tone instead. It never skips the I2S write.

---

Feature Gates#

Two flags deliberately control different layers:

When VOICE_I2S_ENABLED is off, the Audio.h functions compile to no-ops. Separating the gates allows UI, state transitions, and turn-taking to be developed and tested before the microphone and amplifier are connected.

---

Configuration Reference#

The PDF source presents some pin and DMA constants as grouped names. They are expanded here for readability; confirm the exact macro spelling in Config.example.h before copying values into firmware.

---

Runtime Invariants and Failure Modes#

The renderer must not block#

Neither capture nor playback may run on the render loop. DMA handles sample transport, while FreeRTOS audio tasks perform chunk-level work on core 0. Network operations remain behind the Agent boundary.

Sample alignment must stay explicit#

Changing the microphone slot, bit width, or alignment without updating the right shift produces clipped, quiet, or corrupted capture. Treat the raw >> 16 conversion as part of the INMP441 wire-format contract.

Playback must remain continuous#

Network jitter is normal. The ring buffer absorbs bursts; underruns emit silence. Skipping writes or blocking until new data arrives breaks the hardware-paced stream.

Volume limiting is safety behavior#

Raising VOICE_VOLUME_PERCENT changes the maximum power delivered to an 8-ohm, 1 W speaker. Validate the power path and speaker rating before changing it.

Hardware and application gates must remain independent#

Tests that use StubAgent should not require I2S initialization. Conversely, enabling I2S should not silently imply that a real voice-agent connection is configured.

---

Practical Extension Guide#

Change the microphone format#

1. Confirm the device's slot, alignment, and signed sample format from its datasheet.
2. Update I2S channel configuration and the capture conversion together.
3. Recalculate chunk duration and network byte rate.
4. Verify representative positive, negative, quiet, and near-clipping samples.

Change the output rate or buffering#

1. Confirm that the amplifier accepts the new sample rate.
2. Recalculate playback chunk duration.
3. Size the ring buffer from an explicit jitter-duration target.
4. Confirm that underrun still produces exactly one chunk of silence.
5. Measure memory use and end-to-end voice latency.

Add another audio backend#

Keep I2S details inside Audio.cpp. Preserve the Audio.h contract so AppController, VoiceSession, and agent implementations remain transport-independent.

---

Referenced Files and Interfaces#

The paths and symbols below are reported by the source document. The firmware repository is not part of this documentation workspace.