diff --git a/docs/Engine-Testing-Patterns.md b/docs/Engine-Testing-Patterns.md new file mode 100644 index 00000000000..803131770ae --- /dev/null +++ b/docs/Engine-Testing-Patterns.md @@ -0,0 +1,255 @@ +# Engine Testing Patterns + +Patterns and gotchas for writing tests against the Terasology engine, +especially integration tests using the ModuleTestingEnvironment (MTE). + +For module-level testing basics, see [Testing-Modules.md](Testing-Modules.md). + +## Test Hierarchy + +Tests form a natural progression from fast/isolated to slow/integrated: + +| Level | Runner | Speed | Example | +|-------|--------|-------|---------| +| Unit (no engine) | Plain JUnit, mocks | Fast | `PojoEventSystemTests` | +| Unit (engine libs) | `ModuleManagerFactory`, manual Context | Medium | `ContextImplTest` | +| Integration (MTE, single player) | `@IntegrationEnvironment` | Slow | `ComponentSystemTest` | +| Integration (MTE, multiplayer) | `@IntegrationEnvironment(networkMode = NetworkMode.LISTEN_SERVER)` | Very slow | `ClientConnectionTest` | + +Prefer higher levels only when the thing you're testing genuinely requires +the engine or network stack. + +## MTE Basics + +The `@IntegrationEnvironment` annotation starts a headless engine instance. +Use `@In` (not `@javax.inject.Inject`) for field injection in test classes — +the MTE harness uses `InjectionHelper` which looks for `@In`. + +```java +@IntegrationEnvironment +public class MyTest { + @In + private EntityManager entityManager; // injected by MTE + + @Test + public void testSomething(MainLoop mainLoop) { + // MainLoop injected via JUnit parameter resolution + } +} +``` + +For multiplayer tests, use `NetworkMode.LISTEN_SERVER` and create clients +via `ModuleTestingHelper`: + +```java +@IntegrationEnvironment(networkMode = NetworkMode.LISTEN_SERVER) +public class MyNetworkTest { + @In + private ModuleTestingHelper helper; + + @Test + public void testWithClient() throws IOException { + Context clientContext = helper.createClient(); + // client has its own EntityManager, NetworkSystem, etc. + } +} +``` + +## Network Event Testing + +### The Event Registration Problem + +Network events (`@BroadcastEvent`, `@OwnerEvent`, `@ServerEvent`) require +two things to replicate over the network: + +1. Registration in the `EventSystem` (for local dispatch) +2. Registration in the `EventLibrary` (for network metadata — direction, serialization) + +The `NetworkEventSystemDecorator` handles both: its `registerEvent()` method +registers the event and, if the event has a network annotation, adds it to +the `EventLibrary`. + +**Inner-class events defined in test files** get registered in the `EventSystem` +by the module classpath scan, but may NOT get their network metadata populated +in the `EventLibrary`. This means they fire locally but never replicate across +the network. + +### Working Patterns + +**Use existing engine events for network propagation tests.** Events like +`ChatMessageEvent` (`@OwnerEvent`) are already fully registered and +will replicate correctly: + +```java +// This works — ChatMessageEvent is engine-registered with full metadata +// It's an @OwnerEvent, so it replicates to the entity's owner +clientEntity.send(new ChatMessageEvent("hello", senderInfo)); +``` + +**Use `TestEventReceiver` for local event testing.** MTE provides this +helper to listen for events without defining a full ComponentSystem: + +```java +try (TestEventReceiver receiver = new TestEventReceiver<>(context, MyEvent.class)) { + entity.send(new MyEvent()); + assertTrue(receiver.hasReceived()); +} +``` + +**Register a probe system for multiplayer event verification.** When you +need to verify an event arrived on a specific context (host or client), +register a `BaseComponentSystem` as a probe via `ComponentSystemManager`: + +```java +MyProbe probe = new MyProbe(); +context.get(ComponentSystemManager.class).register(probe); +``` + +`ComponentSystemManager.register()` handles event handler registration +internally — do not also call `EventSystem.registerEventHandler()` as +this causes duplicate event delivery. + +### What Does NOT Work + +Defining `@BroadcastEvent`/`@OwnerEvent`/`@ServerEvent` inner classes in +test files and expecting them to replicate over the network. They register +locally but the `EventLibrary` doesn't pick up their network metadata, +so `NetworkEventSystemDecorator.networkReplicate()` silently skips them. + +## Singleton State in Tests + +Several engine classes are global singletons (`PathManager`, `CoreRegistry`) +that tests must mutate. This is inherently fragile — any test that changes +singleton state affects all subsequent tests in the same JVM. + +### The save/restore pattern + +Always save the original state in `@BeforeEach` and restore in `@AfterEach`: + +```java +private Path originalHomePath; + +@BeforeEach +void setUp(@TempDir Path tempHome) throws IOException { + originalHomePath = PathManager.getInstance().getHomePath(); + PathManager.getInstance().useOverrideHomePath(tempHome); +} + +@AfterEach +void tearDown() throws IOException { + // Guard: @TempDir cleanup may have already deleted the original path + if (originalHomePath != null && Files.isDirectory(originalHomePath)) { + PathManager.getInstance().useOverrideHomePath(originalHomePath); + } +} +``` + +The `Files.isDirectory` guard is important — `@TempDir` cleanup runs before +`@AfterEach` in some JUnit configurations, so the path you saved in setup +may already be deleted. Without the guard, the restore itself throws +`NoSuchFileException`. + +### Why this matters + +If a test class mutates `PathManager` without restoring, the next test class +in the same JVM sees a home path pointing at a deleted temp directory. This +causes `NoSuchFileException` failures that are: +- **Non-deterministic**: they depend on test execution order +- **Environment-specific**: may pass locally but fail in CI (different JVM + reuse, cleanup timing, OS) +- **Hard to diagnose**: the failing test is correct — the bug is in a + *different* test class that ran earlier + +Gradle's default is one JVM per test worker, so parallel test *classes* +are usually isolated. Parallel *methods* within a class share state — +avoid `@Execution(CONCURRENT)` on tests that mutate singletons. + +### CoreRegistry Isolation + +`CoreRegistry` is the same pattern — a deprecated static singleton: + +```java +@BeforeEach +void setUp() { + originalContext = CoreRegistry.get(Context.class); + CoreRegistry.setContext(new ContextImpl()); +} + +@AfterEach +void tearDown() { + CoreRegistry.setContext(originalContext); +} +``` + +MTE tests handle this automatically — don't manually set `CoreRegistry` +in `@IntegrationEnvironment` tests. + +### Service vs System + +Distinguish between: +- **Services** (in `Context`/`CoreRegistry`): always available via `@In`, + e.g. `EntityManager`, `NetworkSystem` +- **ComponentSystems** (in `ComponentSystemManager`): event-driven, process + events via `@ReceiveEvent`, e.g. `ChatSystem` + +Some functionality (like chat) requires an ECS system to be registered and +processing events — just having the service in the context isn't enough. + +## Test Timing Diagnostics + +MTE integration tests — especially those creating clients — are prone to +timeout-related flakiness in CI. Use JUnit 5's `TestReporter` to publish +structured timing data that appears in JUnit XML reports: + +```java +@Test +@Tag("flaky") +public void testWithClient(TestReporter reporter) throws IOException { + long start = System.currentTimeMillis(); + + Context clientContext = helper.createClient(); + reporter.publishEntry("client_connect_ms", + String.valueOf(System.currentTimeMillis() - start)); + + // ... test logic ... + + reporter.publishEntry("total_ms", + String.valueOf(System.currentTimeMillis() - start)); +} +``` + +The published entries appear in JUnit XML output and are accessible via +the Jenkins test report API (`/testReport/api/json`), making them +queryable by both humans and automation without console log scraping. + +**When to add timings:** Any test tagged `@Tag("flaky")`, and especially +any test that calls `helper.createClient()` — the client connection +handshake is the most common source of timeout failures. + +**Recommended entry names:** `client_connect_ms`, `client2_connect_ms`, +`both_registered_ms`, `messages_sent_ms`, `total_ms`, `ticks_received`. + +## Gradle Test Execution + +```bash +# Run all tests +./gradlew test + +# Run a specific test class (subproject is required) +./gradlew :engine-tests:test --tests "*.ClientNetworkStateTest" + +# Force fresh run (clear cached results) +./gradlew :engine-tests:cleanTest :engine-tests:test --tests "*.MyTest" + +# Via ws CLI (auto-discovers subproject and clears cache) +ws test terasology ClientNetworkStateTest +``` + +Always use `cleanTest` for targeted test runs — Gradle's UP-TO-DATE cache +can serve stale results from a previous failure. + +## See Also + +- [Testing-Modules.md](Testing-Modules.md) — module-level testing basics +- `engine-tests/src/main/java/org/terasology/engine/integrationenvironment/` — MTE source +- `engine-tests/src/test/java/org/terasology/engine/integrationenvironment/` — existing MTE tests diff --git a/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientNetworkStateTest.java b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientNetworkStateTest.java new file mode 100644 index 00000000000..b9737d1e6d1 --- /dev/null +++ b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientNetworkStateTest.java @@ -0,0 +1,74 @@ +// Copyright 2026 The Terasology Foundation +// SPDX-License-Identifier: Apache-2.0 +package org.terasology.engine.integrationenvironment; + +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestReporter; +import org.terasology.engine.context.Context; +import org.terasology.engine.core.GameEngine; +import org.terasology.engine.core.TerasologyEngine; +import org.terasology.engine.core.subsystem.EngineSubsystem; +import org.terasology.engine.core.subsystem.common.NetworkSubsystem; +import org.terasology.engine.integrationenvironment.jupiter.IntegrationEnvironment; +import org.terasology.engine.network.NetworkMode; +import org.terasology.engine.network.NetworkSystem; +import org.terasology.engine.registry.In; + +import java.io.IOException; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Verifies that a client created via MTE has the correct network state: + * NetworkSystem in CLIENT mode and a NetworkSubsystem registered. + */ +@IntegrationEnvironment(networkMode = NetworkMode.LISTEN_SERVER) +public class ClientNetworkStateTest { + + @In + private ModuleTestingHelper helper; + + @Test + @Tag("flaky") + public void testClientNetworkMode(TestReporter reporter) throws IOException { + long start = System.currentTimeMillis(); + Context clientContext = helper.createClient(); + reporter.publishEntry("client_connect_ms", String.valueOf(System.currentTimeMillis() - start)); + + NetworkSystem networkSystem = clientContext.get(NetworkSystem.class); + assertNotNull(networkSystem, "NetworkSystem should exist in client context"); + assertEquals(NetworkMode.CLIENT, networkSystem.getMode(), + "Client NetworkSystem should be in CLIENT mode"); + + reporter.publishEntry("total_ms", String.valueOf(System.currentTimeMillis() - start)); + } + + @Test + @Tag("flaky") + public void testClientHasNetworkSubsystem(TestReporter reporter) throws IOException { + long start = System.currentTimeMillis(); + Context clientContext = helper.createClient(); + reporter.publishEntry("client_connect_ms", String.valueOf(System.currentTimeMillis() - start)); + + GameEngine gameEngine = clientContext.get(GameEngine.class); + assertNotNull(gameEngine, "Client should have a GameEngine in context"); + assertInstanceOf(TerasologyEngine.class, gameEngine, "GameEngine should be a TerasologyEngine"); + TerasologyEngine engine = (TerasologyEngine) gameEngine; + + boolean hasNetworkSubsystem = false; + for (EngineSubsystem subsystem : engine.getSubsystems()) { + if (subsystem instanceof NetworkSubsystem) { + hasNetworkSubsystem = true; + break; + } + } + assertTrue(hasNetworkSubsystem, + "NetworkSubsystem should be registered in the client engine"); + + reporter.publishEntry("total_ms", String.valueOf(System.currentTimeMillis() - start)); + } +} diff --git a/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientSystemUpdateTest.java b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientSystemUpdateTest.java new file mode 100644 index 00000000000..9d1bcbb364f --- /dev/null +++ b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/ClientSystemUpdateTest.java @@ -0,0 +1,64 @@ +// Copyright 2026 The Terasology Foundation +// SPDX-License-Identifier: Apache-2.0 +package org.terasology.engine.integrationenvironment; + +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestReporter; +import org.terasology.engine.context.Context; +import org.terasology.engine.core.ComponentSystemManager; +import org.terasology.engine.entitySystem.systems.BaseComponentSystem; +import org.terasology.engine.entitySystem.systems.UpdateSubscriberSystem; +import org.terasology.engine.integrationenvironment.jupiter.IntegrationEnvironment; +import org.terasology.engine.network.NetworkMode; +import org.terasology.engine.registry.In; + +import java.io.IOException; + +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Verifies that a system registered on a client receives update ticks + * through the MTE game loop. + */ +@IntegrationEnvironment(networkMode = NetworkMode.LISTEN_SERVER) +public class ClientSystemUpdateTest { + + @In + private ModuleTestingHelper helper; + + @Test + @Tag("flaky") + public void testClientSystemReceivesUpdates(TestReporter reporter) throws IOException { + long start = System.currentTimeMillis(); + Context clientContext = helper.createClient(); + reporter.publishEntry("client_connect_ms", String.valueOf(System.currentTimeMillis() - start)); + + ComponentSystemManager csm = clientContext.get(ComponentSystemManager.class); + assertNotNull(csm, "Client should have a ComponentSystemManager"); + assertTrue(csm.isActive(), "Client ComponentSystemManager should be active"); + + TickCountingSystem tickCounter = new TickCountingSystem(); + csm.register(tickCounter); + + int targetTicks = 5; + helper.runUntil(() -> tickCounter.updateCount >= targetTicks); + + reporter.publishEntry("ticks_received", String.valueOf(tickCounter.updateCount)); + reporter.publishEntry("total_ms", String.valueOf(System.currentTimeMillis() - start)); + + assertTrue(tickCounter.updateCount >= targetTicks, + "Client system should have received at least " + targetTicks + + " updates, got " + tickCounter.updateCount); + } + + public static class TickCountingSystem extends BaseComponentSystem implements UpdateSubscriberSystem { + int updateCount; + + @Override + public void update(float delta) { + updateCount++; + } + } +} diff --git a/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/TwoClientChatTest.java b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/TwoClientChatTest.java new file mode 100644 index 00000000000..0789447936b --- /dev/null +++ b/engine-tests/src/test/java/org/terasology/engine/integrationenvironment/TwoClientChatTest.java @@ -0,0 +1,111 @@ +// Copyright 2026 The Terasology Foundation +// SPDX-License-Identifier: Apache-2.0 +package org.terasology.engine.integrationenvironment; + +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestReporter; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.terasology.engine.context.Context; +import org.terasology.engine.core.ComponentSystemManager; +import org.terasology.engine.entitySystem.entity.EntityRef; +import org.terasology.engine.entitySystem.systems.BaseComponentSystem; +import org.terasology.engine.integrationenvironment.jupiter.IntegrationEnvironment; +import org.terasology.engine.logic.chat.ChatMessageEvent; +import org.terasology.engine.network.Client; +import org.terasology.engine.network.ClientComponent; +import org.terasology.engine.network.NetworkMode; +import org.terasology.engine.network.NetworkSystem; +import org.terasology.engine.registry.In; +import org.terasology.gestalt.entitysystem.event.ReceiveEvent; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +/** + * Verifies that a ChatMessageEvent sent from the host reaches a connected + * client via network event propagation in a LISTEN_SERVER environment. + *

+ * ChatMessageEvent is an {@code @OwnerEvent} — it replicates to the entity's + * owner. The host sends it to each connected client entity individually, + * mimicking what ChatSystem does during normal chat. + */ +@IntegrationEnvironment(networkMode = NetworkMode.LISTEN_SERVER) +public class TwoClientChatTest { + private static final Logger logger = LoggerFactory.getLogger(TwoClientChatTest.class); + + @In + private ModuleTestingHelper helper; + + @Test + @Tag("flaky") + public void testChatReachesClient(TestReporter reporter) throws Exception { + long startTime = System.currentTimeMillis(); + + // Create two clients + helper.createClient(); + long client1Ms = System.currentTimeMillis() - startTime; + reporter.publishEntry("client1_connect_ms", String.valueOf(client1Ms)); + logger.info("Client 1 connected in {}ms", client1Ms); + + long client2Start = System.currentTimeMillis(); + Context client2Ctx = helper.createClient(); + assertNotNull(client2Ctx, "Client 2 context should be created"); + long client2Ms = System.currentTimeMillis() - client2Start; + reporter.publishEntry("client2_connect_ms", String.valueOf(client2Ms)); + logger.info("Client 2 connected in {}ms (total: {}ms)", + client2Ms, System.currentTimeMillis() - startTime); + + // Wait for both clients to register on the host + NetworkSystem hostNetwork = helper.getHostContext().get(NetworkSystem.class); + helper.runUntil(() -> { + int count = 0; + for (Client ignored : hostNetwork.getPlayers()) { + count++; + } + return count >= 2; + }); + assertThat(hostNetwork.getPlayers()).hasSize(2); + long registeredMs = System.currentTimeMillis() - startTime; + reporter.publishEntry("both_registered_ms", String.valueOf(registeredMs)); + logger.info("Both clients registered on host at {}ms", registeredMs); + + // Register a ChatMessageEvent probe on client 2 + ChatProbe probe = new ChatProbe(); + client2Ctx.get(ComponentSystemManager.class).register(probe); + + // Pick a sender and send ChatMessageEvent to each client entity from the host + // (mimics ChatSystem — sends to each client's entity individually) + Client senderClient = hostNetwork.getPlayers().iterator().next(); + EntityRef senderInfo = senderClient.getEntity().getComponent(ClientComponent.class).clientInfo; + String testMessage = "hello from host"; + + for (Client client : hostNetwork.getPlayers()) { + client.getEntity().send(new ChatMessageEvent(testMessage, senderInfo)); + } + reporter.publishEntry("messages_sent_ms", String.valueOf(System.currentTimeMillis() - startTime)); + logger.info("Chat messages sent at {}ms", System.currentTimeMillis() - startTime); + + // Wait for client 2's probe to receive the message + helper.runUntil(() -> probe.received); + + long totalMs = System.currentTimeMillis() - startTime; + reporter.publishEntry("total_ms", String.valueOf(totalMs)); + logger.info("Probe received at {}ms (total test time)", totalMs); + + assertThat(probe.received).isTrue(); + assertThat(probe.lastMessage).contains(testMessage); + } + + public static class ChatProbe extends BaseComponentSystem { + boolean received; + String lastMessage = ""; + + @ReceiveEvent(components = ClientComponent.class) + public void onChatMessage(ChatMessageEvent event, EntityRef entity) { + received = true; + lastMessage = event.getMessage(); + } + } +}