DogStatsD load generation by blt · Pull Request #1782 · DataDog/saluki

blt · 2026-06-01T02:02:06Z

Summary

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

How did you test this PR?

References

This commit introduces DogStatD load generation. If you're familiar with my work in lading you might notice this is very different. The implementation here is responsible for giving Antithesis choices between 'compliant' dogstatsd lines and 'feral' lines which are _mechanically_ allowable lines but are in practice not sensible. Also of interest control of contexts and send rates are jettisoned entirely, that's an antithesis scheduling concern. Metrics, events and service-checks are all present. Depending on the antithesis shots I may need to adjust the ratios of compliant to feral outcomes.

blt · 2026-06-01T02:02:22Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

DogStatsD load generation #1782 👈 (View in Graphite)
enhancement(antithesis): Vary datadog.yaml in test/antithesis, assert aliveness #1779
docs(agent-data-plane): Antithesis research scratchbook and bug ledger #1768 : 1 other dependent PR (#1769 )
chore(antithesis): Antithesis test harness and workload #1767
enhancement(correctness): Create test/antithesis sub-project #1774
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

datadog-official · 2026-06-01T02:02:41Z

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 2 Pipeline jobs failed

DataDog/saluki | check-licenses

🔧 Fix in code (Fix with Cursor).
Current 'LICENSE-3rdparty.csv' is not up to date.

Semantic PR Title Check | Check For Semantic PR Title

🛟 This job is unlikely to succeed on retry. Please review your pipeline configuration.
No release type found in pull request title. Required prefix missing for release indication.

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: a31a487 | Docs | Datadog PR Page | Give us feedback!}

pr-commenter · 2026-06-01T02:09:58Z

Binary Size Analysis (Agent Data Plane)

Baseline: 1bd1613 · Comparison: a31a487 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 37.89 MiB (baseline) vs 37.90 MiB (comparison)
Size Change: +760 B (+0.00%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
`anon.36eead261da73468b10600f578de22ed.87.llvm.14046994766694184360`	+2.83 KiB	1
`anon.0acc57ca412442189c7678857beeb25f.1.llvm.18230564023335910389`	-2.83 KiB	1
`anon.eba192a60e6caa742f85cd2f7b63cef6.18.llvm.10590908203511297131`	-2.46 KiB	1
`anon.e5f0d67f8d0d8b1c6e3888577e274890.511.llvm.6538778123177323897`	+2.46 KiB	1
`anon.423de405463045f5e6382613e58502c9.555.llvm.7227059468892596563`	+1.59 KiB	1
`anon.ed85dad8c2fdcfe5c906411ebc1601c6.1.llvm.11441704272744907195`	-1.59 KiB	1
`anon.0acc57ca412442189c7678857beeb25f.101.llvm.17623075813770587000`	+1.32 KiB	1
`anon.69220190dd54a7096abb5416a7fc5f5b.172.llvm.3001728827928409060`	-1.32 KiB	1
`anon.69220190dd54a7096abb5416a7fc5f5b.84.llvm.15805731423594874612`	+1.17 KiB	1
`anon.02b5255633f97219e3c81be4a33426c5.42.llvm.4649599560089562033`	-1.17 KiB	1
`anon.50abf489da332cac3fe9fcf0b1218cd2.353.llvm.436185206835468104`	+1.06 KiB	1
`anon.423de405463045f5e6382613e58502c9.623.llvm.2135413985037036474`	-1.06 KiB	1
`_RNvMs5_NtNtCsaz6eC2DG7lh_3std2io5errorNtB5_5Error4kind.llvm.12104049852236292769`	+830 B	1
`_RNvMs5_NtNtCsaz6eC2DG7lh_3std2io5errorNtB5_5Error4kind.llvm.15805731423594874612`	+830 B	1
`_RNvMs5_NtNtCsaz6eC2DG7lh_3std2io5errorNtB5_5Error4kind.llvm.3001728827928409060`	-829 B	1
`_RNvMs5_NtNtCsaz6eC2DG7lh_3std2io5errorNtB5_5Error4kind.llvm.4649599560089562033`	-829 B	1
`anon.0cbe5d1da6c19cbc1bc3d4c0ab437541.44.llvm.197401638548642842`	+823 B	1
`anon.f002492f45681213d5f2178507d8f8df.914.llvm.4043333310207047691`	-821 B	1
`anon.50abf489da332cac3fe9fcf0b1218cd2.48.llvm.9678418280774762212`	-732 B	1
`anon.50abf489da332cac3fe9fcf0b1218cd2.48.llvm.436185206835468104`	+731 B	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +3.43Ki  [NEW]    +456    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<saluki_components::destinations::dsd_stats::DogStatsDAPIHandlerState>>,axum::routing::path_router::PathRouter<saluki_components::destinations::dsd_stats::DogStatsDAPIHandlerState,_>::with_state<$LP$$RP$>::{{closure}}>>::h5a147bb683008716
  [NEW] +2.84Ki  [NEW]     +47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::h674725303a3ab093
  [NEW] +2.83Ki  [NEW]      +2    anon.36eead261da73468b10600f578de22ed.87.llvm.14046994766694184360
  [NEW] +2.46Ki  [NEW]     +80    anon.e5f0d67f8d0d8b1c6e3888577e274890.511.llvm.6538778123177323897
  [NEW] +2.17Ki  [NEW]    +129    core::ptr::drop_in_place<std::sync::poison::PoisonError<std::sync::poison::rwlock::RwLockWriteGuard<quick_cache::shard::CacheShard<alloc::string::String,saluki_components::sources::otlp::metrics::cache::NumberCounter,saluki_common::cache::weight::WrappedWeighter<saluki_common::cache::weight::ItemCountWeighter>,foldhash::quality::RandomState,saluki_common::cache::expiry::ExpiryCapableLifecycle<alloc::string::String>,alloc::sync::Arc<quick_cache::sync_placeholder::Placeholder<saluki_components::sources::otlp::metrics::cache::NumberCounter>>>>>>::he338aa4cd4e8609e
  [NEW] +2.17Ki  [NEW]    +315    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_components::sources::dogstatsd::forwarder::ForwardPacket>>::h76efc4f68c4fbede
  [NEW] +1.59Ki  [NEW]     +97    anon.423de405463045f5e6382613e58502c9.555.llvm.7227059468892596563
  [NEW] +1.57Ki  [NEW]    +703    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::poll_frame::hb2bbfae5f62c1b09
  [NEW] +1.38Ki  [NEW]    +691    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::poll_frame::h147ce3747417d704
  [NEW] +1.32Ki  [NEW]     +88    anon.0acc57ca412442189c7678857beeb25f.101.llvm.17623075813770587000
  +0.0%    +771  [ = ]       0    [4037 Others]
  [DEL] -1.32Ki  [DEL]     -88    anon.69220190dd54a7096abb5416a7fc5f5b.172.llvm.3001728827928409060
  [DEL] -1.38Ki  [DEL]    -691    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::poll_frame::h8d3d490742ef5359
  [DEL] -1.57Ki  [DEL]    -703    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::poll_frame::h4c56918971715b8b
  [DEL] -1.59Ki  [DEL]     -97    anon.ed85dad8c2fdcfe5c906411ebc1601c6.1.llvm.11441704272744907195
  [DEL] -2.17Ki  [DEL]    -129    core::ptr::drop_in_place<std::sync::poison::PoisonError<std::sync::poison::rwlock::RwLockWriteGuard<quick_cache::shard::CacheShard<stringtheory::MetaString,core::option::Option<saluki_components::transforms::dogstatsd_mapper::CachedMapResult>,saluki_common::cache::weight::WrappedWeighter<saluki_common::cache::weight::ItemCountWeighter>,foldhash::quality::RandomState,saluki_common::cache::expiry::ExpiryCapableLifecycle<stringtheory::MetaString>,alloc::sync::Arc<quick_cache::sync_placeholder::Placeholder<core::option::Option<saluki_components::transforms::dogstatsd_mapper::CachedMapResult>>>>>>>::hb84ff946adb48f71
  [DEL] -2.18Ki  [DEL]    -315    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_components::common::datadog::transaction::Transaction<saluki_common::buf::chunked::FrozenChunkedBytesBuffer>>>::h24eaa96d15a67cde
  [DEL] -2.46Ki  [DEL]     -80    anon.eba192a60e6caa742f85cd2f7b63cef6.18.llvm.10590908203511297131
  [DEL] -2.83Ki  [DEL]      -2    anon.0acc57ca412442189c7678857beeb25f.1.llvm.18230564023335910389
  [DEL] -2.84Ki  [DEL]     -47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::h66dcc7212f74260d
  [DEL] -3.44Ki  [DEL]    -456    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<saluki_components::sources::dogstatsd::replay::replay_control::DogStatsDReplayControl>>,axum::routing::path_router::PathRouter<saluki_components::sources::dogstatsd::replay::replay_control::DogStatsDReplayControl,_>::with_state<$LP$$RP$>::{{closure}}>>::h9e288300660bcc12
  +0.0%    +760  [ = ]       0    TOTAL

pr-commenter · 2026-06-01T02:25:21Z

Regression Detector (Agent Data Plane)

Run ID: 265d0344-fbd2-42a2-822c-8e7eda897017
Baseline: 1bd16137 · Comparison: a31a487f · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
dsd_uds_1mb_3k_contexts_cpu (erratic)	cpu	⚪ +8.50	metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored)	memory	⚪ +6.74	metrics profiles logs
otlp_ingest_metrics_5mb_memory	memory	⚪ +1.69	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic)	cpu	⚪ +1.59	metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored)	cpu	⚪ +1.38	metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput	throughput	⚪ -1.24	metrics profiles logs
dsd_uds_512kb_3k_contexts_memory	memory	⚪ +0.47	metrics profiles logs
dsd_uds_100mb_3k_contexts_memory	memory	⚪ +0.36	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ +0.12	metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic)	cpu	⚪ +0.08	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ +0.07	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput	throughput	⚪ -0.05	metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput	throughput	⚪ -0.01	metrics profiles logs
otlp_ingest_traces_5mb_memory	memory	⚪ +0.01	metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput	throughput	⚪ -0.00	metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput	throughput	⚪ -0.00	metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput	throughput	⚪ +0.00	metrics profiles logs
otlp_ingest_metrics_5mb_throughput	throughput	⚪ +0.01	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ -0.03	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput	throughput	⚪ +0.03	metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored)	throughput	⚪ +0.04	metrics profiles logs
dsd_uds_1mb_3k_contexts_memory	memory	⚪ -0.04	metrics profiles logs
otlp_ingest_traces_5mb_throughput	throughput	⚪ +0.05	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory	memory	⚪ -0.07	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.08	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.12	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory	memory	⚪ -0.15	metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic)	cpu	⚪ -0.17	metrics profiles logs
dsd_uds_500mb_3k_contexts_memory	memory	⚪ -0.30	metrics profiles logs
dsd_uds_10mb_3k_contexts_memory	memory	⚪ -0.32	metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic)	cpu	⚪ -0.66	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic)	cpu	⚪ -0.93	metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic)	cpu	⚪ -1.35	metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic)	cpu	⚪ -4.43	metrics profiles logs
dsd_uds_512kb_3k_contexts_cpu (erratic)	cpu	🟢 -9.70	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 123 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 39.6 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 60.5 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 181 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 26.7 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jun 1, 2026

blt mentioned this pull request Jun 1, 2026

enhancement(antithesis): Vary datadog.yaml in test/antithesis, assert aliveness #1779

Draft

4 tasks

reduce log load

a31a487

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DogStatsD load generation#1782

DogStatsD load generation#1782
blt wants to merge 2 commits into
blt/assert_adp_aliveness_on_bootup_add_sometimes_check_to_forwardingfrom
blt/dogstatsd-load-generation

blt commented Jun 1, 2026

Uh oh!

blt commented Jun 1, 2026

Uh oh!

datadog-official Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

blt commented Jun 1, 2026

Summary

Change Type

How did you test this PR?

References

Uh oh!

blt commented Jun 1, 2026

Uh oh!

datadog-official Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

pr-commenter Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Analysis (Agent Data Plane)

✅ Binary size difference within threshold

Uh oh!

pr-commenter Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector (Agent Data Plane)

Optimization Goals: ✅ No significant changes detected

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

datadog-official Bot commented Jun 1, 2026 •

edited

Loading

pr-commenter Bot commented Jun 1, 2026 •

edited

Loading

pr-commenter Bot commented Jun 1, 2026 •

edited

Loading