Overview#

Note: This document was generated by an AI agent.

Boto is an expressive desk character running on a classic dual-core ESP32. A 128×64 monochrome OLED shows either a set of procedural facial expressions or a 2.5D face rig; one capacitive touch pad navigates the character and a set of information panels; a microphone and a small amplifier let you hold a conversation with it.

That is the whole product. Everything else in these documents exists because a microcontroller is a genuinely tight place to build it.

Read the rest of this section in order and every box above gets its own explanation. This is the only page that shows them all at once.

---

The five constraints#

Almost every design decision in this project is downstream of one of five facts about the hardware. They are worth stating plainly and once, because later documents assume them rather than re-explaining them.

1. The panel is one bit deep#

Each pixel is on or off. There is no grey, no alpha, no anti-aliasing. A curve is a staircase, and a shape that looks smooth does so because its staircase was chosen carefully.

This is why the face is drawn by arithmetic rather than stored as images. A stored bitmap at one bit per pixel cannot be dimmed, blended, or tweened — you can only cut to the next one. Expressions instead exist as numbers that are interpolated, and the drawing code turns whatever numbers it is handed into pixels. Rendering and effects covers the consequence; the 2.5D rig takes it furthest, deriving each frame from a projection rather than storing any frame at all.

2. The panel is fed over I2C, and that is slow#

I2C (Inter-Integrated Circuit, said "eye-squared-C") is a two-wire bus: one data line (SDA) and one clock line (SCL), shared by every chip on it. Instead of each peripheral getting its own dedicated pins, devices on an I2C bus are addressed by a fixed ID and only one talks at a time. The trade for that simplicity — two wires, any number of devices — is speed: every byte crosses one bit at a time, acknowledged by the receiver before the next can go. I2S is not I2C contrasts it with the very different bus this board uses for audio.

The framebuffer is 128 × 64 ÷ 8 = 1,024 bytes. Pushing it to the SSD1306 over I2C at 400 kHz costs, near enough:

1,024 bytes × 9 bits   = 9,216 bits      (8 data bits + 1 ACK each)
9,216 bits / 400 kHz   ≈ 23.0 ms
plus addressing and control bytes per chunk
                       ≈ 25 ms in practice

The frame interval is 33 milliseconds, about 30 fps. So roughly three quarters of every frame is spent waiting on the panel. That single number explains a surprising amount:

• Information panels redraw at only 10 fps, because static text does not need the bus and the face does.
• Pre-rendered clips, authored at 64 fps, play back at double speed, since a 16 ms frame time is not achievable when the transfer alone is 25 ms.
• Audio runs on the other core. A blocking write sharing core 1 would be competing for the eight milliseconds a frame has left.

3. There is no filesystem#

SPIFFS (SPI Flash File System) is a small filesystem that can live directly on a microcontroller's own flash chip — no SD card needed — letting firmware read and write ordinary named files the way a desktop program opens one off disk. There is a SPIFFS partition reserved on this board, but the firmware never mounts it.

Instead, the web UI and the animation clips are compiled directly into the firmware binary as data embedded in the program itself, rather than stored as separate files that could go missing. A file you can open is a file you can be missing, and recovering from a missing asset on a device with no screen worth apologising on is not worth the trouble. The cost is that changing the web page means reflashing rather than just uploading a new file. Settings, persistence and assets covers what genuinely does need to survive a reboot, and the very different mechanism it uses.

4. RAM is small, and the largest contiguous block matters more than the total#

The ESP32 has 320 KB of DRAM available to a sketch. This firmware's static allocation is already 108 KB of it, leaving roughly 220 KB of heap for everything dynamic — TLS sessions, the web server, audio buffers, the display buffer, the JSON documents.

The trap is not the total. It is fragmentation:

A successful memory allocation is not evidence that it was safe. The Wi-Fi stack needs contiguous memory too, and starving it does not fail at the allocation — it fails minutes later as a device that silently stopped answering, taking with it the only channel you had for asking what went wrong.

That has happened twice on this device. Both times the fix was to budget against the largest contiguous free block, which the device can report about itself, rather than against total free heap — free heap still looks roomy after the largest block is gone. The audio recorder now refuses to start rather than take a buffer that would leave too little behind; audio diagnostics tells that story in full.

5. Flash is 4 MB, and over-the-air updates halve it#

Flashing over Wi-Fi requires two application slots: the running firmware, and somewhere to write the new one before switching. That is what this board's partition layout buys, and what it costs:

The alternative would be a single 3 MB application and no second slot — more room, but every update would then require a USB cable. Data, network and OTA covers the swap itself.

---

The resource envelope#

These are measured from the tracked build, not estimated.

Read those four rows as the budget the rest of this documentation spends. Each later document itemises its own share: the I2S runtime accounts for 24 KB of DMA buffers and 8 KB of playback ring; the local agent for the TLS session that sizes everything around it; the 2.5D rig for its share of the eight millisecond slack.

What running out looks like, since none of these fail cleanly:

---

What it does#

The tracked configuration starts in 3D face style with manual scene advancement, and falls into ambient scenes after inactivity. Classic style offers twenty expressions; the 3D registry holds 55 entries, eight of which are reserved user slots whose checked-in assets are empty.

The panel set is Clock, Weather, System, Network, Battery, Voice, Speak and Setup. Visibility comes from stored settings seeded from configuration. Clock and weather need valid data to show anything; battery is disabled unless wired and configured. Voice, I2S, Whisper, Ollama, Piper and wake listening all default off in the tracked configuration, so a fresh build runs without any of the speech hardware present.

The device also serves a management website over HTTP, which requires a non-empty configured password even on its setup access point. Its Network, Display, Animations, Audio Test and Firmware tabs reflect the device's compiled capabilities and its active face style, rather than offering controls that would do nothing.

Touch controls#

One pad, so everything is a tap or a hold. The long-press threshold is 600 ms.

To leave a voice tab for the face while voice is enabled, tap through to an ordinary panel and long-press there — the hold gesture is taken by the turn.

Hands-free operation detects sustained sound, sends the utterance to Whisper, and checks whether the transcript begins with Boto's name. It is transcript-based gating, not an on-device keyword recogniser, which is an important distinction for both latency and privacy; the conversational agent covers why it was built that way and what it costs.

---

Dependencies#

The firmware is a single Arduino sketch: one small entry point, with all real implementation organized underneath it by responsibility — the face, audio, the agent, networking, and so on — compiled together as one tree.

The web page is plain HTML, CSS and JavaScript with no build step. External model services are optional and are not deployed as part of this project.

---

How to read the rest#

This is meant to be read in order:

---

Adding something new#

The recurring question, when extending a device like this, is where a change belongs. In order:

1. Check what already exists first, so a change never becomes a second copy of a helper that's already there.
2. Group it with its domain — the face, audio, the agent, networking, and so on — rather than at the top level. The module map gives the domains and what each owns.
3. Decide which core it runs on. Anything that can block for longer than a few milliseconds — network, filesystem, inference, anything with a timeout — belongs on the background core behind a task, never in the render loop.
4. Put every setting in one place, a central configuration kept in sync with its template.
5. Keep it testable without hardware if it's pure logic. The gesture, easing, face-state and animation-script logic is deliberately kept free of the Arduino API, so it compiles and runs in a host test suite with no board attached.
6. Render absence honestly. Missing hardware and missing network state get a placeholder that says so. Nothing in this firmware invents a plausible value.