DUAL_CORE_ARCHITECTURE.md

ESP32-S3 Dual-Core Architecture

How SPNFY uses the ESP32-S3's dual-core architecture for smooth animation while handling network operations.

ESP32-S3 Core Overview

The ESP32-S3 has two Xtensa LX7 cores:

Core	Name	Default Usage
Core 0	Protocol CPU (PRO_CPU)	WiFi/Bluetooth stack, background tasks
Core 1	Application CPU (APP_CPU)	Arduino setup() and loop()

The Problem: Blocking Network Calls

HTTP requests to the Spotify API can take 200-500ms. In a single-threaded model:

loop: [render][render][─── HTTP wait 500ms ───][render][render]
                            ↑ Animation freezes here

The Solution: Task Separation

We run network operations on Core 0, keeping Core 1 free for smooth rendering:

┌─────────────────────────────────────────────────────────────┐
│                        ESP32-S3                             │
├─────────────────────────┬───────────────────────────────────┤
│        Core 0           │            Core 1                 │
│    (Background Tasks)   │       (Arduino loop)              │
├─────────────────────────┼───────────────────────────────────┤
│ • Spotify API polling   │ • Rotation rendering              │
│ • HTTP requests         │ • Touch handling                  │
│ • Image download        │ • JPEG decode (on new image)      │
│ • Token refresh         │ • Status overlay                  │
│ • Display DMA transfer  │                                   │
│                         │                                   │
│ [Blocking OK]           │ [Smooth 30+ FPS animation]        │
└─────────────────────────┴───────────────────────────────────┘

Implementation

1. Spotify Task (Core 0)

Handles all Spotify API communication:

void spotifyTask(void* parameter) {
    while (true) {
        // Check WiFi, handle rate limits
        if (WiFi.status() != WL_CONNECTED) {
            vTaskDelay(pdMS_TO_TICKS(1000));
            continue;
        }
        
        // Handle playback commands (play/pause, skip)
        if (playbackToggleRequested) {
            sendPlaybackCommand(playbackToggleTarget);
            playbackToggleRequested = false;
        }
        
        // Poll now playing
        String imageUrl = getNowPlaying();
        if (!imageUrl.isEmpty()) {
            downloadImageToPending(imageUrl);
        }
        
        vTaskDelay(pdMS_TO_TICKS(50));  // Check frequently for commands
    }
}

// Create task on Core 0
xTaskCreatePinnedToCore(
    spotifyTask, "SpotifyTask", 8192, NULL, 1, &spotifyTaskHandle, 0
);

2. Send Task (Core 0)

Transfers frame buffer to display via DMA:

static void sendTask(void *param) {
    while (true) {
        if (xSemaphoreTake(sendSemaphore, portMAX_DELAY) == pdTRUE) {
            if (bufferToSend) {
                sendBufferToDisplay(bufferToSend);
                bufferToSend = nullptr;
            }
            xSemaphoreGive(sendDoneSemaphore);
        }
    }
}

// Create task on Core 0
xTaskCreatePinnedToCore(
    sendTask, "SendTask", 4096, nullptr, 2, &sendTaskHandle, 0
);

3. Main Loop (Core 1)

Handles rendering and touch:

void loop() {
    handleTouch();
    
    // Check for new album art from Spotify task
    if (newImageReady && xSemaphoreTake(spotifyMutex, 0)) {
        // Decode and display new image
        displayJpeg(pendingJpegBuffer, pendingJpegSize);
        xSemaphoreGive(spotifyMutex);
    }
    
    // Render frame
    renderRotatedAlbumArt();
    renderStatusOverlay(true);
    
    // Async send (returns immediately, Core 0 handles transfer)
    swapAndSendAsync();
}

Thread-Safe Data Exchange

Both cores accessing the same memory causes corruption. We use mutexes:

// Shared state
SemaphoreHandle_t spotifyMutex = nullptr;
volatile bool newImageReady = false;
uint8_t *pendingJpegBuffer = nullptr;

// Core 0 (writer) - in spotifyTask
if (xSemaphoreTake(spotifyMutex, portMAX_DELAY)) {
    pendingJpegBuffer = downloadedData;
    pendingJpegSize = size;
    newImageReady = true;
    xSemaphoreGive(spotifyMutex);
}

// Core 1 (reader) - in loop()
if (newImageReady && xSemaphoreTake(spotifyMutex, 0)) {  // 0 = don't wait
    jpegBuffer = pendingJpegBuffer;
    pendingJpegBuffer = nullptr;
    newImageReady = false;
    xSemaphoreGive(spotifyMutex);
    
    displayJpeg(jpegBuffer, jpegBufferSize);
}

Key points:

• portMAX_DELAY: Wait forever (OK for background task)
• 0 timeout: Don't wait, return immediately (keeps animation smooth)

Double Buffering

Render and send run in parallel:

Core 1 (Main Loop)           Core 0 (Send Task)
─────────────────            ──────────────────
Render Frame N   ──────────► Wait for semaphore
                             │
Swap buffers ◄───────────────┤
Give semaphore  ─────────────► Send Frame N-1 to display
                             │
Render Frame N+1             │ (parallel execution)

void swapAndSendAsync() {
    xSemaphoreTake(sendDoneSemaphore, portMAX_DELAY);  // Wait for prev
    bufferToSend = frameBuffers[currentRenderBuffer];
    currentRenderBuffer = 1 - currentRenderBuffer;
    frameBuffer = frameBuffers[currentRenderBuffer];
    xSemaphoreGive(sendSemaphore);  // Trigger send
}

FreeRTOS vs Arduino Delays

Function	Behavior	Use Case
delay(ms)	Blocks the core	Simple Arduino code
vTaskDelay(ticks)	Yields to other tasks	FreeRTOS tasks
pdMS_TO_TICKS(ms)	Converts ms to ticks	Use with vTaskDelay

In FreeRTOS tasks, always use vTaskDelay() to allow proper scheduling.

Memory Considerations

• Task stacks allocated from heap (8192 bytes for Spotify, 4096 for Send)
• Image buffers use PSRAM (MALLOC_CAP_SPIRAM)
• Mutex is a small kernel object (~80 bytes)
• DMA buffer in internal RAM for fast transfer

Performance Results

Configuration	FPS	Notes
Single-threaded	15	Animation freezes during API calls
Dual-core, sync send	20-25	Better, but send still blocks
Dual-core, async send	30+	Render and send in parallel

Debugging Tips

Check which core is running:

ESP_LOGI(TAG, "Running on core %d", xPortGetCoreID());

Monitor task stack usage:

UBaseType_t stackHighWater = uxTaskGetStackHighWaterMark(spotifyTaskHandle);
ESP_LOGI(TAG, "Stack remaining: %d bytes", stackHighWater * 4);