techpad
Building a Unified Firmware for the Waveshare ESP32-S3-Touch-LCD-3.49

Building a Unified Firmware for the Waveshare ESP32-S3-Touch-LCD-3.49

· 10 min read
Tutorials arduino esp32 firmware iot lvgl voice-assistant

The hardware

The Waveshare ESP32-S3-Touch-LCD-3.49 is an unusual board: a 172×640 IPS touch strip display (tall and narrow, not a normal rectangle), driven by an ESP32-S3R8 with 8MB of PSRAM. It packs a lot into a small footprint:

  • AXS15231B QSPI all-in-one LCD + touch controller
  • ES8311 (speaker DAC) + ES7210 (mic ADC) audio codec
  • QMI8658 IMU
  • PCF85063 battery-backed RTC
  • TCA9554 GPIO expander
  • Native USB-Serial/JTAG — no separate UART bridge chip

The goal: one piece of firmware that's simultaneously a clock/status display, a Home Assistant dashboard (in progress), and a full push-to-talk voice assistant — plus alarms, timers, and a push-notification API, all built with the Arduino IDE.

Toolchain

  • arduino-cli with the esp32:esp32:esp32s3 core, v3.3.x (ESP-IDF 5.x based) — required because the vendored I2C glue code (i2c_bsp.c) uses the new driver/i2c_master.h API (i2c_new_master_bus), which doesn't exist in the older 2.x cores.
  • LVGL v8, not v9 — deliberately. Waveshare's own display/touch glue (lvgl_port.c) is written against v8's driver API (lv_disp_drv_t). Rewriting it for v9's lv_display_t API wasn't worth it just to chase "latest."
  • Full FQBN, worth recording since it's easy to get wrong on this board:
esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=cdc,CPUFreq=240,
FlashMode=qio,FlashSize=16M,PartitionScheme=app3M_fat9M_16MB,DebugLevel=none,
PSRAM=opi,LoopCore=1,EventsCore=1

USBMode=hwcdc matters — this board has no UART bridge chip, so the native USB-C connects straight to the S3's built-in USB-Serial/JTAG peripheral. It enumerates as usbmodemXXXX, not the usual usbserial-*/SLAB_USBtoUART.

Bring-up: the backlight mystery

The first real bug wasn't in application code — it was hardware. Following Waveshare's own 01_ADC_Test example (which toggles IO-expander pin 1 as part of ADC setup) caused the backlight to die intermittently. Flashing back to a completely stock example confirmed the hardware itself was fine, which narrowed it to firmware.

Root cause, found by bisecting subsystems one at a time (compile → flash → test, disabling one thing per cycle): TCA9554 IO-expander pin 1, when toggled, disables the backlight circuit — on this specific board revision. Waveshare's own reference code uses that same pin as an ADC gate elsewhere, so the two examples directly contradict each other. Once battery_bsp.c stopped touching pin 1, the backlight became rock solid.

That pin is now documented with a loud comment and in docs/pinout.md as a "never touch" — a good reminder that vendor reference code isn't gospel when multiple examples in the same repo disagree.

Also confirmed empirically: the backlight PWM is active-low. LCD_PWM_MODE_255 computes to a duty cycle of 0 (brightest), and LCD_PWM_MODE_0 computes to 255 (fully off) — the macro names are inverted relative to what they actually do.

Landscape rotation

The panel is native portrait (172 wide × 640 tall). For a clock/dashboard use case, landscape made more sense. Two paths were possible:

  1. LVGL8's built-in sw_rotate + lv_disp_set_rotation()
  2. Waveshare's own compile-time Rotated == USER_DISP_ROT_90 macro, which does a manual buffer transpose in the flush callback

Option 1 crashed reliably (Guru Meditation Error: StoreProhibited @ 0x00000000), traced to a real bug in the vendored LVGL8: lv_draw_sw_layer_create() calls lv_memset_00() on a buffer before checking whether the allocation actually succeeded. Rather than patch LVGL internals, option 2 (the vendor's own proven manual-transpose macro) was used instead.

LVGL memory tuning

Two library-level patches were needed in the shared lv_conf.h:

  • LV_FONT_MONTSERRAT_48 enabled (for the alarm/timer digit displays)
  • LV_MEM_SIZE bumped from 48KB → 96KB

The crash mentioned above also stemmed partly from LV_MEM_SIZE being too small — LVGL's internal heap lives in internal SRAM, not PSRAM, so it's a genuinely scarce resource shared with WiFi, the audio codec, and everything else. 192KB was tried first and reverted — it pushed RAM usage to 75% and arduino-cli started warning about stability.

A custom 165px digit-only bitmap font (lv_font_conv, generated from Arial Bold, --no-compress since LV_USE_FONT_COMPRESSED isn't set) was used for the big clock display instead of relying on transform_zoom, which was part of what triggered the LVGL crash above.

Battery power and the PWR button

Two separate issues here:

Power latch. IO-expander pin 6 has to be driven high to keep the board powered on battery — without it, the device powers off the instant the physical PWR button is released. io_expander_bsp_init() latches this immediately at boot.

Long-press power off. GPIO16 (active-low) is polled in loop(); holding it for 5 seconds (PWR_BUTTON_OFF_HOLD_MS) clears the same latch pin, cutting power immediately. This is intentionally all firmware-driven — powering on is a hardware/analog concern (holding PWR boosts VBAT_5V long enough for the ESP32 to boot and self-latch), so there was nothing to build for that half.

Later, the PWR button gained a second job: a short press (held less than the 5-second threshold, but more than a 30ms debounce) now cycles to the next page in the UI, wrapping back to the clock after the last page — a physical alternative to swiping.

OTA — the dev-cycle speedup

With no UART bridge chip, every USB flash requires manually holding BOOT, pressing RESET, then releasing BOOT — tedious for a fast iteration cycle. ArduinoOTA mostly solved this:

arduino-cli upload -p <device-ip> --fqbn "<FQBN>" \
  --upload-field "password=<ota-password>" firmware/display349_hub

It's not perfectly reliable — uploads occasionally stall at a random percentage and time out. No root cause was found; the workaround is simply retrying, which usually succeeds on the second attempt. USB flashing remains the fallback when OTA won't cooperate.

The voice assistant pipeline

This was the most involved piece: record → transcribe → chat → speak, entirely through OpenRouter's API, triggered by either the physical BOOT button or an on-screen mic button.

Audio hardware. esp_codec_dev, vendored from Waveshare's 08_Audio_Test, wraps the ES8311 (playback) and ES7210 (record) over I2S in TDM mode, board profile "S3_LCD_3_49". All audio buffers are allocated explicitly from PSRAM (heap_caps_malloc(..., MALLOC_CAP_SPIRAM)) — Arduino's default heap is only ~320KB total, nowhere near enough for WAV-sized buffers.

Three bugs, found one at a time via serial/network logging:

  1. No sound, HTTP 400. The TTS model slug openai/gpt-4o-mini-tts-2025-12-15 didn't actually exist on OpenRouter. Tested several model slugs directly against the live API via curl; hexgrad/kokoro-82m with response_format: "pcm" was the one that actually worked. (OpenAI's own tts-1/tts-1-hd/gpt-4o-mini-tts all 400'd despite being documented; Mistral's Voxtral only supports mp3, not pcm.)
  2. Stuck forever on "Speaking...". The chunked-transfer read loop had no timeout and relied on WiFiClient::connected(), which stays true on a keep-alive connection even after the full response body has arrived. Fixed with a 2-second idle timeout instead.
  3. Audio played but sounded like scratching/static. HTTPClient::getStreamPtr() doesn't decode HTTP chunked transfer-encoding — the raw chunk-size framing bytes were being read as if they were PCM audio samples. Fixed with http.useHTTP10(true) before the request, which avoids chunked encoding entirely.

Playback volume was also boosted via software gain, since the codec's own volume control was already maxed out and still too quiet.

Alarm, Timer, Alerts, and a push-notification API

The most recent addition: a reusable notification primitive plus three features built on top of it.

app_alert is the shared building block — a two-beep synthesized chime (sine wave with an attack/decay envelope, generated straight into a PSRAM buffer and pushed through audio_bsp_play()) plus a top-layer LVGL banner that auto-dismisses after 8 seconds. It's careful about locking: app_alert_show() must be called from a context that does not already hold the LVGL mutex (i.e. loop() or a background task), since it takes the lock itself — calling it from inside an LVGL event callback would deadlock.

Alarm (app_alarm) — hour/minute steppers, an on/off switch, and NVS persistence via Preferences, so it survives a reboot. app_alarm_poll() runs from loop() and fires app_alert_show() once per matching minute.

Timer (app_timer) — minute stepper (1–60 min, ±1/±5 buttons), Start/Cancel, live countdown display, fires the same alert at zero.

Push API (net_api) — a tiny WebServer on port 80:

POST http://<device-ip>/notify
Body: raw text, or JSON {"text": "..."}

One rough edge worth noting: a plain curl -d "some text" sends Content-Type: application/x-www-form-urlencoded by default, which the ESP32 WebServer parses as a form field rather than leaving the body intact — so arg("plain") comes back empty. The handler now falls back to treating the parsed form key itself as the message when that happens, so both curl -d "text" and curl -H "Content-Type: text/plain" -d "text" work without the caller needing to know the difference.

UI navigation: from vertical to horizontal paging

The screens (Clock, Weather, Notify, Info, HA placeholder, Voice, Alarm, Timer, Settings) live in an lv_tileview. They started out stacked vertically (swipe up/down) and were later switched to horizontal (swipe left/right) — which surfaced two touch-handling bugs worth detailing, since they're a good example of how a display's raw geometry can quietly break assumptions baked into earlier fixes.

Bug 1 — swipes weren't registering at all. LVGL8's tileview only switches pages once a drag crosses roughly 50% of the tile's width, measured from the drag's start point. That threshold was invisible in the old vertical layout, where each tile was only 172px tall — an ordinary swipe covers well over half of that. Horizontally, tiles are the full 640px-wide screen, so the same physical swipe distance falls far short of the ~320px needed. Fix: intercept PRESSED/RELEASED on every tile, track the tileview's scroll offset between the two, and force a page switch with lv_obj_set_tile_id() past a much smaller fixed threshold (50px) instead of relying on the built-in center-snap behavior.

Bug 2 — after that fix, slow swipes worked but fast ones didn't. An earlier fix (for a jumpy touch-slider) had added an outlier filter that rejects any single touch-poll jump larger than 60px, on the theory that such jumps were electrical glitches rather than real finger movement. That filter was tuned for the old 172px-tall axis. On the new 640px-wide axis, a fast swipe legitimately moves more than 60px between polls — so the filter was silently discarding real movement, indistinguishable from noise. The fix was to split the threshold per axis: 200px on X (the long, 640px axis) and keep 60px on Y (the short, 172px axis) — wide enough for a real fast swipe, still tight enough to catch genuine glitches on the narrow axis.

Both bugs are a reminder that when you rotate or re-orient a UI, every distance-based heuristic tuned against the old geometry (scroll thresholds, outlier filters, tap-vs-drag detection) needs re-checking against the new one — they don't fail loudly, they just quietly stop matching the physical reality of the screen.

Where it stands

Working and tested on real hardware: landscape display, big custom-font clock, weather, notifications, an on-device info/status page, battery voltage + PWR button (short-press-to-page, long-press-to-power-off), OTA flashing, the full voice pipeline, alarm, timer, the push-notification API, and left/right paging. Home Assistant/MQTT integration remains a placeholder screen — the next phase. Backlight dimming (as opposed to on/off) turned out not to work on this board's circuit despite the PWM registers writing and reading back correctly — the leading theory is that the backlight enable circuit is binary rather than truly analog-dimmable, and that thread was deliberately set aside in favor of the features above.

The overall lesson from this build: on a board this integrated, most of the hard bugs weren't in the "interesting" application code (the voice pipeline, the UI) — they were in the boring plumbing underneath it: which GPIO pin quietly disables what, which library API assumes an axis that no longer matches after a rotation, which HTTP client silently mishandles chunked encoding. Systematic bisection and direct hardware logging (a raw TCP log mirror on port 23, since the USB cable isn't always plugged in) did more to find these than reading documentation ever did.