Question: recordRoborazziDebug takes very long (22 min.)

[simonvassili-qm]

Hi @takahirom,

I'm setting up roborazzi in a large multimodule project.
We have 112 ui-modules where we apply the RoborazziPlugin and have to record snapshots.

A recordRoborazziDebug run takes up to 22 min. on a M4 Pro. This would be also not feasible for our CI.

I don't know how to improve it? What I'm doing wrong?

Here is my current setup:

class RoborazziPlugin : Plugin<Project> {

    override fun apply(target: Project): Unit = with(target) {
        val isAndroidModule = plugins.hasPlugin("com.android.application") || plugins.hasPlugin("com.android.library")
        if (!isAndroidModule) return@with

        val android = extensions.findByType(CommonExtension::class.java) ?: return@with
        val isComposeEnabled = android.buildFeatures.compose == true

        if (isComposeEnabled && name.contains("internal")) {
            configureRoborazzi(this, android)
        }
    }

    @OptIn(ExperimentalRoborazziApi::class)
    private fun configureRoborazzi(project: Project, android: CommonExtension) = with(project) {
        val libs = extensions.getByType<VersionCatalogsExtension>().named("libs")
        pluginManager.apply(libs.findPlugin("roborazzi").get().get().pluginId)

        extensions.configure<RoborazziExtension> {
            generateComposePreviewRobolectricTests {
                enable.set(true)
                includePrivatePreviews.set(true)
                packages.set(listOf("...."))
                generatedTestClassCount.set((Runtime.getRuntime().availableProcessors() / 2).coerceAtLeast(1))
            }
        }

        dependencies.apply {
            add("testImplementation", libs.findBundle("roborazzi").get())
            add("testRuntimeOnly", libs.findLibrary("test-junit-vintage-engine").get())
        }

        // Required so the generated Robolectric tests can resolve Android resources and strings.
        android.testOptions.unitTests.isIncludeAndroidResources = true
        android.testOptions.unitTests.isReturnDefaultValues = true

        val isRoborazziRun = gradle.startParameter.taskNames.any { it.contains("roborazzi", ignoreCase = true) }
        tasks.withType<Test>().configureEach {
            useJUnitPlatform {
                if (isRoborazziRun) {
                    // Roborazzi run: execute ONLY the generated screenshot tests.
                    includeTags(ROBORAZZI_PREVIEW_CATEGORY)
                } else {
                    // Normal unit-test run: skip the screenshot tests.
                    excludeTags(ROBORAZZI_PREVIEW_CATEGORY)
                }
            }
            // Advised by Roborazzi for image fidelity; harmless on normal runs and checked at configuration time.
            systemProperties["robolectric.pixelCopyRenderMode"] = "hardware"
            // The memory-heavy settings are only needed while actually rendering previews.
            if (isRoborazziRun) {
                maxHeapSize = "4096m"
                jvmArgs("-noverify")
                // The compare baseline is downloaded into build/outputs/roborazzi by CI and is not a tracked
                // task input, so a cached or up-to-date test result could silently skip rendering against a
                // new baseline. Always re-render during Roborazzi runs (compilation stays incremental).
                doNotTrackState("Roborazzi screenshot rendering must always re-run")
            }
        }
    }

    private companion object {
        // JUnit Platform tag that JUnit Vintage derives from the generated tests'
        // `@Category(RoborazziComposePreviewTestCategory)` (the tag equals the category's FQCN).
        const val ROBORAZZI_PREVIEW_CATEGORY = "com.github.takahirom.roborazzi.RoborazziComposePreviewTestCategory"
    }
}

I register it in buildSrc:

gradlePlugin {
    plugins {
        register("roborazzi-plugin") {
            id = "roborazzi-plugin"
            implementationClass = "....customgradleplugins.RoborazziPlugin"
        }
   }
}

Currently I apply it in the projects build.gradle.kts:

subprojects { 
        val modulesToIncludeForRoborazzi = listOf(":feature:", ":core:")
        if (modulesToIncludeForRoborazzi.any(path::startsWith) && path.endsWith(":internal")) {
            apply(plugin = "roborazzi-plugin")
        }

And this is the bundle in libs.versions.toml:

[bundles]

# Applied by RoborazziPlugin to every Compose-enabled :feature/:core *:internal module.
roborazzi = [
    "test-robolectric", 
    "test-roborazzi", // 1.64.0
    "test-roborazzi-annotations",
    "test-roborazzi-compose",
    "test-roborazzi-junit-rule",
    "test-roborazzi-previewscannersupport",
    "test-composepreviewscanner",
    "test-compose-ui-junit4",
    "test-junit4",
    "test-jupiter-api",
]

Thank you for your help!

[takahirom]

@simonvassili-qm Thanks for the details!

If the test execution itself is slow locally, I’d first try to identify where it is spending time before changing the configuration.

One easy way is to run the screenshot test from Android Studio using the debugging setup described in the README. When it looks stuck or very slow, pause the debugger. The current stack trace usually tells you what Roborazzi/Robolectric/Compose is waiting for.

From past cases, the common causes are usually:

1. The first run is slow because Robolectric/Android SDK artifacts are being downloaded.
2. An animation is still running, so Roborazzi waits for Compose/Robolectric to become idle.
3. An Activity or some state is leaking between tests.

I’d also recommend running the test with VisualVM attached. It is often enough to start VisualVM, select the Gradle/test JVM process, and check where CPU/time is being spent. That can quickly show whether the time is spent in rendering, waiting for idleness, downloading/loading SDK artifacts, or something else.

So as a first step, could you try pausing the debugger while the test is slow, or checking the process with VisualVM?

[simonvassili-qm]

Thank you for you answer!
I'll investigate it and get back to you.

[takahirom]

@simonvassili-qm I think you should try using Robolectric 4.17-beta-1. It seems that the new version is very fast.