Architecture: Settings, Persistence, and Assets#

Note: This document was generated by an AI agent.

Overview#

There are three kinds of state on this device, and the distinction between them decides where each one lives:

A saved preference takes precedence over its compiled default, but cannot create a feature omitted from the image — so the local configuration file is not a complete description of a device that already has NVS values.

The third row is the one that shapes this page. Volatile state is not persisted because it changes at frame rate, and flash is a medium where a write is never free.

---

What NVS actually is, and what a write costs#

Arduino's preferences API is a thin wrapper over ESP-IDF's NVS, which is not a key-value database but a log on flash. Flash cannot rewrite a byte in place; it can only clear bits within a page and erase a whole 4 KB sector back to ones. NVS works with that rather than against it.

The partition is mostly empty on purpose: the free space is what the append-only log churns through before a page has to be erased.

The structure is worth knowing because it explains all three of the elision tricks in this firmware:

partition            0x5000          = 20,480 B  = five 4 KB pages
page                 32 B header + 32 B state bitmap + 126 entries x 32 B
usable               ~four pages     ≈ 504 entries ≈ 16 KB

Every value is at least one 32-byte entry. A primitive up to eight bytes fits inside one entry alongside its key (NVS caps a key at 15 characters, which is why setting keys are terse and a dedicated test guards the limit). A string or blob takes one entry for its header plus one per 32 bytes of data.

An update does not overwrite. It appends a new entry and marks the old one erased. The page is reclaimed only when NVS compacts it, which is the wear-levelling — and which is why writing the same value repeatedly is not free even though nothing about the stored state changed.

What is actually stored#

Under eight percent of the usable space. The remaining headroom is not waste — it is what the append-only log consumes between compactions.

Why endurance is not the binding constraint#

Flash sectors are rated around 100,000 erase cycles. A page absorbs on the order of 126 appended entries before it has to be compacted, and there are several pages, so the bound on total writes is in the tens of millions. That number is irrelevant here for a better reason than its size:

Nothing in this firmware writes to NVS on a timer, and nothing writes per frame. Every write is caused by a person — a web PUT, a Setup menu selection, a scene reorder, a credential change.

All three stores that can be driven repeatedly elide an identical write before it reaches flash: each one checks whether the incoming value already matches what was last saved, and skips the write entirely when it does. The face-style store does the same for its single byte. Those three guards are the entire wear strategy, and they work because the write rate is already human-scale. The change that would break this is a periodic write — anything that persists state on a schedule rather than on a decision.

Persistence failure is a degradation, not an error#

Every store applies the new value in RAM regardless, and reports separately whether it also persisted. Writing a setting over the web API surfaces that as session_only with HTTP 200, because RAM genuinely changed and the value is live — it simply will not survive a reboot. The scene-order store documents matching behaviour.

Two caveats on reading saved back:

• saved can mean "the store was readable and returned the default for an absent key". It does not prove a key was physically written.
• The Wi-Fi HTTP response acknowledges a staged reconnect, not durable storage: the network module saves the credentials without checking the result, and the store itself only checks the SSID write. Do not present that response as proof that both credentials survived a reboot.

These stores live outside both application slots, so an ordinary firmware update retains every one of them; only erasing flash or changing the partition layout loses them. There is no all-state factory reset — settings reset, scene-order reset, style selection and Wi-Fi saving are four separate operations.

---

The runtime setting catalogue#

The API exposes 17 settings. A validation layer checks names and ranges, and refuses a write that would set all nine panel-visibility values to zero — because the only input is one touch pad, and a user with no visible panel has no way back to the menu. Panel entries must stay contiguous with each other internally: a tab maps to its setting by position rather than by a lookup table, and anything inserted among the panel-visibility entries shifts every later one.

Five settings have no consumer at all. They are stored, validated, persisted and returned by the API, and nothing reads them — the compiled defaults for weather units, screen frame timing, ambient idle timeout, and the two animation speed settings are still what the firmware uses. A 200 with saved:true proves the preference was written, not that anything acts on it. These are implementation gaps, catalogued rather than hidden.

anim_speed is also not a 3D speed control — 3D playback uses a separate compiled speed value (78), which is not a setting at all.

Two API notes. Validated external writes should go through the range-checked entry point; a lower-level operation exists that does not range-check, for internal use. And a stored-but-invalid value falls back to the compiled default at load, so a range narrowed in a later firmware cannot strand a device on a value it no longer accepts.

The table cannot exceed 32 entries without a change elsewhere: the saved-state tracking uses a 32-bit bitmask. At 17 there is room; at 33 the mask silently stops working.

Reading settings is streamed one entry at a time, so the table can grow without the response having to fit anywhere. It did not always work that way — see the web API.

Face style is chosen in the on-device Setup menu and saved separately; there is no HTTP endpoint for it. The web page discovers Classic versus 3D through the status response and the scene route's 409.

The scene-order blob#

Order and rotation mask are stored as one blob, not two, so a partial write can never leave them disagreeing:

version (1) + count (1) + order[55] + mask[7]  =  64 bytes exactly

The count field is load-bearing. The registry has already grown once in this project — 47 scenes to 55 — and a stale blob applied blindly would scramble the order on every device that updated, because the same index now names a different scene. So a blob only loads when its count still matches the live registry; otherwise the order resets to default.

That check does not detect a semantic id reassignment at an unchanged count. If ids are ever renumbered without the count changing, preserve the existing meanings or bump VERSION explicitly.

---

Generated assets#

All of them follow one convention: generated headers are checked in and regenerated by hand, rather than produced at build time. Compilation does not regenerate anything, which means a page change that is not regenerated ships the old page — and is also why there is no Python dependency in the build.

The web generator gzips with a fixed modification time so an unchanged page produces an identical header instead of a spurious diff. When verifying an asset, compare decompressed content: gzip output bytes can differ across Python and zlib versions even with a fixed timestamp. The sizes and the reasoning for embedding rather than serving from a filesystem are in the web API.

The generated user-animation header currently declares zero user scenes, with eight empty slots reserved. It refers to external animation-editing tools that are not tracked in this checkout, so do not send contributors to run them. User tracks store a timestamp and a value per keyframe, with a per-track float scale and a channel enum, reserving eight scenes and a 24-key track capacity — see the animation format for the full contract. There is no HTTP upload or editor for user animations in this firmware.

There is no longer a generated TLS trust-bundle header. It held the Mozilla CA bundle for direct HTTPS from the board, and went when cloud replies moved to the LAN proxy — 88 KB of flash recovered, and one fewer generated artefact to keep current.

---

Local configuration#

The environment sync script reads literal KEY=value lines. It is not a dotenv implementation: it does not source a shell file and does no quoting or interpolation, so surrounding a value with shell quotes includes the quotes. The last matching key wins. It requires a non-empty Wi-Fi network name, escapes backslashes and quotes for C++, and emits Wi-Fi credentials plus any non-empty OTA password, web password and web username.

The build script neither runs that sync step nor copies the configuration template; both are deliberate one-time setup steps.

• For a credentials-free face build, omit the generated credentials header entirely.
• For an AP-only first boot with web access, set a web password in the local configuration file and leave Wi-Fi credentials empty — the sync script cannot generate an empty-SSID header.

---

Adding a setting#

1. Add the default to both the local configuration file and the tracked template.
2. Add the enum entry and the setting-table entry. NVS keys are capped at 15 characters, and panel entries must stay contiguous with the screen list.
3. Add the matching default-values entry in the settings store — a missing entry shifts every later default silently.
4. Wire a real consumer to the setting-read API. Persisting a value is not implementing a control; five settings on this page are the proof.
5. Add policy coverage to the settings test suite, including any new invariant.
6. Update the web page's labels and grouping, and regenerate the asset.
7. Check the widest entry still fits the streaming response's 192-byte item buffer — the settings test suite asserts this.
8. Verify apply, reboot retention, reset, and storage-failure reporting.

For strings or compound data, add a dedicated adapter the way Wi-Fi and scene order do. Do not put mutable strings into the integer settings table. If a worker on core 0 must read the value, make that ownership explicit.

---

Failure modes#

---

Files referenced#

See also#

• Web management and HTTP API — the routes that drive every write here
• Data, network and weather and OTA — why these values survive an update
• Animation asset format — the user-scene contract
• Runtime — the volatile state that is deliberately not here