chore(antithesis): Differential comparison between ADP and Datadog Agent context egress - #1891
Conversation
dd-octo-sts
Bot
added
the
area/test
This comment has been minimized.
This comment has been minimized.
Binary Size Analysis (Agent Data Plane)Baseline: 92cb9bd · Comparison: 1b94f06 · diff ✅ Binary size difference within thresholdChanges by Module
Detailed Symbol Changes |
87401a9 to
790b0a1
Compare
8eb47bd to
f4e1e05
Compare
790b0a1 to
cda6b84
Compare
Regression Detector (Agent Data Plane)Run ID: Optimization Goals: ✅ No significant changes detectedFine details of change detection per experiment (5)Experiments configured
Bounds Checks: ✅ Passed (5)
ExplanationA change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression ( |
cda6b84 to
b0895d9
Compare
82c6e40 to
f0e828a
Compare
03f2c03 to
cb55618
Compare
f0e828a to
a76a3d1
Compare
a76a3d1 to
db3a661
Compare
a20bc37 to
d7c3986
Compare
6b700ac to
32a0637
Compare
d7c3986 to
ac2e0a0
Compare
32a0637 to
c88e400
Compare
ac2e0a0 to
8077839
Compare
8d4a9ef to
2291f5a
Compare
55ba0bc to
e04ff3e
Compare
2291f5a to
0146358
Compare
0146358 to
66b9e71
Compare
66b9e71 to
ce908ea
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ec4e0fba14
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 76a96d20e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This draft PR introduces a new differential equivalence scenario focused only on context totals for now. There's enough change in here that I will pull alterations down-stack, more to come later.
| @@ -38,7 +41,7 @@ endif | |||
| export ADP_STANDALONE_IPC_CERT_FILE := /tmp/adp-ipc-cert.pem | |||
|
|
|||
| # macOS integration-test settings. | |||
| MACOS_TEST_AGENT_VERSION ?= 7.80.4 | |||
| MACOS_TEST_AGENT_VERSION ?= 7.80.3 | |||
| MACOS_TEST_AGENT_DMG_DIR ?= /tmp/saluki-dda-dmg-cache | |||
| MACOS_TEST_AGENT_DMG_URL ?= https://s3.amazonaws.com/dd-agent/datadog-agent-$(MACOS_TEST_AGENT_VERSION)-1.$(shell uname -m).dmg | |||
| MACOS_TEST_AGENT_INSTALL_DIR ?= /tmp/saluki-dda/datadog-agent | |||
| @@ -201,6 +204,7 @@ build-datadog-agent-image: build-adp-image ## Builds the converged Datadog Agent | |||
| @docker build \ | |||
| --tag saluki-images/datadog-agent:testing-devel \ | |||
| --tag local.dev/saluki-images/datadog-agent:testing-devel \ | |||
| --build-arg "DD_AGENT_VERSION=$(DATADOG_AGENT_VERSION)-full" \ | |||
| --build-arg ADP_IMAGE=saluki-images/agent-data-plane:testing-devel \ | |||
| --file ./docker/Dockerfile.datadog-agent \ | |||
| . | |||
| @@ -211,6 +215,7 @@ build-datadog-agent-image-release: build-adp-image-release ## Builds the converg | |||
| @docker build \ | |||
| --tag saluki-images/datadog-agent:testing-release \ | |||
| --tag local.dev/saluki-images/datadog-agent:testing-release \ | |||
| --build-arg "DD_AGENT_VERSION=$(DATADOG_AGENT_VERSION)-full" \ | |||
There was a problem hiding this comment.
While I don't think this change is wrong, per se -- and in some ways, it's better than the status quo -- we've been moving to have our dependency update stuff actually update the references directly so we can phase out needing .datadog-agent-version at all which is somewhat disconnected/lossy... so I don't want to propagate its usage unless there's a particular reason.
(The unsaid part here is that DD_AGENT_VERSION should already default to <version>-full in docker/Dockerfile.datadog-agent where <version> is the value in .datadog-agent-version... so I'm confused on why this was needed at all.)
There was a problem hiding this comment.
Ah, this would just be confusion on my part then. Will repair.
Summary
This commit introduces a 'differential' scenario to the antithesis scenario bank.
The approach here is to continuously compute the symmetric difference between
the set of ADP and Datadog Agent context egress sets, more details inline in the
scenario README. This does not yet assert that the counts are accurate, merely
that both SUTs egress the same contexts at some point. This commit was big enough,
I figured I'd add that sort of thing in a later line of work.
Change Type
How did you test this PR?
References