Architecture: ESP32 Parametric Face Animation#

Overview#

The ESP32 robot face contains no pre-rendered facial images. Its visible state is generated from a FaceState structure containing 21 floating-point parameters. Every frame, the engine determines the current value of each parameter and draws a 128 by 64 monochrome face from those values.

The face behaves like a marionette with 21 control strings. An expression is a target set of values, not an image. Motion comes from changing those values over time, and rendering comes from simple geometry.

Every face on this page is produced by running the firmware's own drawing primitives on that preset's 21 numbers, at the panel's real 128 × 64. Nothing here is an illustration of the face; it is the face.

---

The FaceState Contract#

The parameters are grouped by visible responsibility:

Every member must remain a float. The interpolation code walks the structure as a contiguous list of floats rather than naming each member. Adding an integer or other differently sized field would misalign that traversal and corrupt subsequent values.

Face.h therefore contains a static_assert that rejects an incompatible structure layout, and test_face checks the same invariant. This is an architectural contract enforced by both compiler and test, not a naming convention developers must remember.

---

Expression Presets#

An expression is a row of target values. Most fields can retain neutral defaults; the preset specifies what changes.

For example, the transition from neutral to happy changes seven parameters:

The emotional reading can depend on a single sign. Angry and sad faces use similar eyes and frowns, but browTilt around +7 reads as anger while a value around -6 reads as sadness. Comparing those two renders above, the eyes and mouths are nearly identical; the entire difference in feeling is one number's sign.

Features can also be hidden without adding flags. Neutral, happy, and sleepy place browY around -24, outside the screen. The renderer detects the out-of-bounds geometry and skips it.

---

Interpolation Between Expressions#

Hard-cutting from one preset to another looks mechanical. The engine retains a source pose and a target pose, then computes normalized progress k from 0 to 1.

result = from + (to - from) × k

For mouthCurve moving from neutral 0 to happy 13:

Conceptually, the same loop applies the same k to all 21 fields:

for (size_t field = 0; field < fieldCount; ++field) {
    out[field] = from[field] + (to[field] - from[field]) * k;
}

Nobody authored the three middle frames. Each is the same line of arithmetic applied to all 21 fields at once — the eyes shortening from 30 to 18, mouthCurve growing from 0 to 13, the lid arc carving deeper, all at one pace.

The source is intentionally field-agnostic. A future float parameter joins interpolation automatically when it is appended to the structure and included in its validated float count.

---

Easing and Motion Character#

Linear progress starts and stops at full speed. Passing k through an easing function adds acceleration, deceleration, or overshoot without changing the start and target states.

The project uses four curves:

Same start, same finish, different journey. easeOutBack is the only curve that travels past its target before settling, and that overshoot is what separates a startle from a sigh.

The complete piecewise in-out curve is:

easeInOutCubic(t) = t < 0.5
    ? 4t^3
    : 1 - (-2t + 2)^3 / 2

Cubing explains why an ease-in starts slowly:

Each scripted expression carries a snappy choice. Sharp reactions such as happy, wink, surprise, anger, knocking, and hiccups use overshoot. Sad, sleepy, suspicious, and fogging transitions use the smooth curve.

---

Continuous Ambient Motion#

Interpolation makes the face change but would otherwise leave it frozen between expressions. Breathing, blinking, and saccades run beneath the selected pose.

Breathing#

The whole face moves vertically on a small sine wave:

faceY = faceY + sin(seconds × 1.8) × 0.9

The period is about 3.5 seconds. Amplitude is less than one pixel, so integer rounding makes the face hold, shift by one pixel near the peak, then return. The viewer perceives life without seeing an obvious bob.

Blinking#

A blink lasts 160 ms but is intentionally asymmetric:

The fast close and slower open imitate eyelid mechanics. Ordinarily the next blink is randomized between 2.2 and 5.2 seconds. One time in four, another blink is scheduled only 180 to 400 ms later, creating natural clusters rather than metronomic timing.

Saccades#

Every 0.7 to 2.2 seconds, the gaze chooses a new random position and reaches it over 220 ms. Neutral gaze wanders by about four pixels; the dedicated look-around expression expands the range to about 11 pixels.

---

Procedural Talking Motion#

A single sine wave produces even chewing. The talking expression adds two waves with frequencies that do not align neatly:

a = 0.5 + 0.5 × sin(seconds × 11.0)
b = 0.5 + 0.5 × sin(seconds × 6.3 + 1.1)

mouthOpen = 2 + a × 9 + b × 4

Sometimes both waves peak together and the mouth opens widely; sometimes they oppose each other and it barely parts. Their combined pattern takes roughly 63 seconds to repeat, which reads as irregular speech during normal use.

---

Timed Set Pieces#

Some expressions are performances rather than static poses. After easing in, an elapsed-time clock drives staged behavior such as sneezing, knocking, fogging the glass, dozing off, and hiccups.

Sneeze timeline#

Hiccup impulse#

A hiccup is a decaying oscillation:

impulse = (1 - k) × cos(k × 18)

cos(k × 18) oscillates about 2.9 times. Multiplying by 1 - k reduces each bounce until none remains, producing the motion of a plucked string.

Permanent terminal state#

With the demo loop disabled, a user can leave an expression selected indefinitely. Every timeline function must therefore return a stable resting value after its story ends.

if (elapsed < 600) {
    return 90 - 52 * easeOutCubic(elapsed / 600.0f);
}
if (elapsed < 3200) {
    return 38;
}
if (elapsed < 3900) {
    return 38 + 52 * easeInCubic(/* normalized drop progress */);
}
return 90; // safely off-screen forever

Without the terminal return, a position could continue growing until it overflows after the pose has been held for a long time.

---

Procedural Drawing#

At the end of a frame, the renderer turns the 21 final values into monochrome geometry.

There are no bitmap face assets in this design. Every eye, brow, mouth, and effect is produced from primitives and arithmetic.

Eyes#

An eye is assembled by drawing and erasing:

Two of the three stages remove pixels rather than add them, which is why the order is part of the contract.

For the happy eye, a 30 by 18 rounded rectangle with radius 9 remains uncovered by the lid, while a lidArc value of 11 carves upward at the sides to leave a squint.

Lid closure from 0 through 1 simply increases the fraction painted black from the top. Blink animation is that value moving to 1 in 56 ms and back to 0 in 104 ms.

Mouth and eye-arc parabola#

For each horizontal column, the renderer computes normalized position t, where -1 is the left edge, 0 is the center, and 1 is the right edge.

The mouth uses:

shape = 1 - t^2

This is zero at the corners and one at the center. Multiplying mouth curve and opening by shape creates a natural center bulge that tapers at both ends. The renderer subtracts mouthCurve × shape from the mouth's vertical midline, so a positive value lifts the center above the corners and a negative value pushes it below. mouthOpen is scaled by the same shape, which is why the lips part widest at the center and meet at the corners.

The happy eye carve uses the complementary shape:

cut = lidArc × t^2

The carve is smallest in the center and largest at the sides, producing the upward arch of a squint. The same parabola is reused in opposite directions.

The sign of mouthCurve alone decides whether the mouth's midline sits above or below its corners; shape only controls how the offset tapers toward the ends.

---

The Frame Budget#

The screen is redrawn at approximately 30 fps, giving each frame about 33 ms. Within that budget the firmware must:

1. Read touch input.
2. Blend all 21 fields.
3. Run expression-specific behavior.
4. Layer the blink and ambient motion.
5. Draw the face and effects.
6. Push 1,024 framebuffer bytes to the OLED over I2C.

Nothing may block the render loop. A 400 ms network request would drop about 12 frames. Weather fetches therefore run on the second processor core, and audio uses dedicated I2S peripherals and DMA. See ESP32 I2S Audio Runtime.

The animation relies on bounded arithmetic such as interpolation, cubic easing, sines, cosines, and simple geometry because those operations complete predictably in microseconds.

---

Expression Timing Reference#

These durations express character rather than merely configuring speed. Surprise reaches its pose quickly and overshoots; sadness eases in slowly; sleep is the slowest arrival.

---

How to Add or Change an Expression#

1. Define the target FaceState using only float fields.
2. Change the minimum number of parameters needed to communicate the expression.
3. Select smooth or overshooting easing based on the motion's character.
4. If the expression is timed, ensure every behavior function has a stable value after its last beat.
5. Keep per-frame work bounded and non-blocking.
6. Test transitions from more than one source pose, not only from neutral.
7. Verify that ambient blinking and gaze do not invalidate deliberate lid or eye behavior.
8. Render at the real 128 by 64 resolution to catch geometry that looks acceptable only when enlarged.

---

Referenced Files#

The paths and symbols below are reported by the source document. The firmware repository is not part of this documentation workspace.