Skip to content

docs(dogstatsd): document dogstatsd_mem_based_rate_limiter.* as architectural divergence#1775

Open
jszwedko wants to merge 5 commits into
mainfrom
jszwedko/dogstatsd-mem-rate-limiter-docs
Open

docs(dogstatsd): document dogstatsd_mem_based_rate_limiter.* as architectural divergence#1775
jszwedko wants to merge 5 commits into
mainfrom
jszwedko/dogstatsd-mem-rate-limiter-docs

Conversation

@jszwedko
Copy link
Copy Markdown
Collaborator

@jszwedko jszwedko commented May 29, 2026

Summary

Expands the single .enabled stub in the DogStatsD configuration doc's "Not Planned" table to cover all 11 dogstatsd_mem_based_rate_limiter.* keys, and adds an explanation section that describes why these Go GC–specific mechanisms have no Rust equivalent in ADP and what ADP uses instead (static MemoryBounds declarations, a MemoryLimiter polling RSS, and bounded channels for structural backpressure).

Closes #1351

Test plan

  • Doc renders correctly (table alignment, header levels, admonition-free)
  • Explanation is accurate against current ADP memory architecture

🤖 Generated with Claude Code

…tectural divergence

Expands the single .enabled entry in the Not Planned table to cover all
11 keys under the prefix, and adds an explanation section describing why
these Go GC-specific mechanisms have no Rust equivalent in ADP and what
ADP uses instead (MemoryBounds, MemoryLimiter, bounded channels).

Closes #1351

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dd-octo-sts dd-octo-sts Bot added the area/docs Reference documentation. label May 29, 2026
Comment thread docs/agent-data-plane/configuration/dogstatsd.md Outdated
Comment thread docs/agent-data-plane/configuration/dogstatsd.md Outdated
jszwedko and others added 3 commits May 29, 2026 14:27
… row

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ble row

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 29, 2026

Binary Size Analysis (Agent Data Plane)

Baseline: 1bd1613 · Comparison: ddba05e · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 37.89 MiB (baseline) vs 37.89 MiB (comparison)
Size Change: +16 B (+0.00%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.85240eacea40817b540ad191ce7e90d0.1.llvm.13164839470218834878 +130 B 1
anon.85240eacea40817b540ad191ce7e90d0.1.llvm.24324102318726552 -127 B 1
anon.85240eacea40817b540ad191ce7e90d0.4.llvm.13164839470218834878 +115 B 1
anon.85240eacea40817b540ad191ce7e90d0.4.llvm.24324102318726552 -112 B 1
anon.85240eacea40817b540ad191ce7e90d0.3.llvm.13164839470218834878 +109 B 1
anon.85240eacea40817b540ad191ce7e90d0.3.llvm.24324102318726552 -106 B 1
anon.85240eacea40817b540ad191ce7e90d0.0.llvm.13164839470218834878 +97 B 1
anon.85240eacea40817b540ad191ce7e90d0.2.llvm.13164839470218834878 +95 B 1
anon.85240eacea40817b540ad191ce7e90d0.0.llvm.24324102318726552 -94 B 1
anon.85240eacea40817b540ad191ce7e90d0.2.llvm.24324102318726552 -92 B 1
[Unmapped] +1 B 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW]    +130  [NEW]     +40    anon.85240eacea40817b540ad191ce7e90d0.1.llvm.13164839470218834878
  [NEW]    +115  [NEW]     +25    anon.85240eacea40817b540ad191ce7e90d0.4.llvm.13164839470218834878
  [NEW]    +109  [NEW]     +19    anon.85240eacea40817b540ad191ce7e90d0.3.llvm.13164839470218834878
  [NEW]     +97  [NEW]      +7    anon.85240eacea40817b540ad191ce7e90d0.0.llvm.13164839470218834878
  [NEW]     +95  [NEW]      +5    anon.85240eacea40817b540ad191ce7e90d0.2.llvm.13164839470218834878
  +0.0%      +1  [ = ]       0    [Unmapped]
  [DEL]     -92  [DEL]      -5    anon.85240eacea40817b540ad191ce7e90d0.2.llvm.24324102318726552
  [DEL]     -94  [DEL]      -7    anon.85240eacea40817b540ad191ce7e90d0.0.llvm.24324102318726552
  [DEL]    -106  [DEL]     -19    anon.85240eacea40817b540ad191ce7e90d0.3.llvm.24324102318726552
  [DEL]    -112  [DEL]     -25    anon.85240eacea40817b540ad191ce7e90d0.4.llvm.24324102318726552
  [DEL]    -127  [DEL]     -40    anon.85240eacea40817b540ad191ce7e90d0.1.llvm.24324102318726552
  +0.0%     +16  [ = ]       0    TOTAL

@jszwedko jszwedko marked this pull request as ready for review May 29, 2026 21:33
@jszwedko jszwedko requested a review from a team as a code owner May 29, 2026 21:33
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d62df945b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- **Structural backpressure**: bounded channels between components provide back-pressure
independently of memory monitoring.

To set a process memory limit in ADP, use `memory_limit` (bytes). The `memory_slop_factor`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document the required memory mode

When a user follows this replacement guidance and only sets memory_limit, ADP does not actually enforce the static bounds or start the RSS-based limiter: memory_mode defaults to Disabled and initialize_memory_bounds returns a no-op limiter in that branch (lib/saluki-app/src/memory.rs lines 91-93 and 157-177). This section should mention that the described refusal/backpressure behavior requires memory_mode: strict or permissive (and enable_global_limiter for dynamic backpressure), otherwise the ignored DogStatsD limiter keys are not replaced by active memory limiting.

Useful? React with 👍 / 👎.

@datadog-prod-us1-5

This comment has been minimized.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 29, 2026

Regression Detector (Agent Data Plane)

Run ID: 6ac1f4a5-c69d-4351-adb9-a79e82417851
Baseline: 1bd16137 · Comparison: ddba05ec · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
otlp_ingest_metrics_5mb_memory memory ⚪ +3.73 metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored) memory ⚪ +2.75 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic) cpu ⚪ +2.64 metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic) cpu ⚪ +1.95 metrics profiles logs
dsd_uds_1mb_3k_contexts_cpu (erratic) cpu ⚪ +1.77 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic) cpu ⚪ +1.36 metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic) cpu ⚪ +0.73 metrics profiles logs
quality_gates_rss_idle memory ⚪ +0.62 metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic) cpu ⚪ +0.43 metrics profiles logs
dsd_uds_10mb_3k_contexts_memory memory ⚪ +0.38 metrics profiles logs
dsd_uds_500mb_3k_contexts_memory memory ⚪ +0.26 metrics profiles logs
dsd_uds_1mb_3k_contexts_memory memory ⚪ +0.20 metrics profiles logs
otlp_ingest_traces_5mb_throughput throughput ⚪ -0.11 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.10 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput throughput ⚪ -0.05 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +0.04 metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored) throughput ⚪ -0.02 metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_512kb_3k_contexts_memory memory ⚪ +0.00 metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput throughput ⚪ +0.00 metrics profiles logs
otlp_ingest_metrics_5mb_throughput throughput ⚪ +0.01 metrics profiles logs
dsd_uds_100mb_3k_contexts_memory memory ⚪ -0.02 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ -0.08 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory memory ⚪ -0.09 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput throughput ⚪ +0.11 metrics profiles logs
otlp_ingest_traces_5mb_memory memory ⚪ -0.12 metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput throughput ⚪ +0.16 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory memory ⚪ -0.31 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.33 metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored) cpu ⚪ -0.37 metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic) cpu ⚪ -1.02 metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic) cpu ⚪ -3.84 metrics profiles logs
dsd_uds_512kb_3k_contexts_cpu (erratic) cpu 🟢 -10.66 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 125 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 39.8 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 61.1 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 184 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 26.8 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Reference documentation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document dogstatsd_mem_based_rate_limiter.* as a known architectural divergence.

1 participant