Skip to content

Update smoke test latest tool versions#11643

Open
dd-octo-sts[bot] wants to merge 1 commit into
masterfrom
ci/update-smoke-test-latest-versions-20260614
Open

Update smoke test latest tool versions#11643
dd-octo-sts[bot] wants to merge 1 commit into
masterfrom
ci/update-smoke-test-latest-versions-20260614

Conversation

@dd-octo-sts

@dd-octo-sts dd-octo-sts Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

This PR updates the pinned latest eligible stable tool versions used by CI Visibility smoke tests.
Only releases at least 48 hours old are eligible.

  • Gradle: 9.5.1 (unchanged)
  • Maven: 4.0.0-beta-3 (unchanged)
  • Maven Surefire: 3.6.0-M1 (published 2026-06-02)

Motivation

Keep smoke tests running against the latest eligible stable versions of build tools without adopting releases in their first 48 hours.

Contributor Checklist

  • Verify smoke tests pass with the new versions

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@dd-octo-sts dd-octo-sts Bot added tag: no release notes Changes to exclude from release notes tag: dependencies Dependencies related changes labels Jun 14, 2026
@dd-octo-sts dd-octo-sts Bot requested a review from a team as a code owner June 14, 2026 06:34
@dd-octo-sts dd-octo-sts Bot added tag: no release notes Changes to exclude from release notes tag: dependencies Dependencies related changes labels Jun 14, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 254916b51f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

# Updated automatically by the update-smoke-test-latest-versions workflow.
maven.version=4.0.0-beta-3
maven-surefire.version=3.5.5
maven-surefire.version=3.6.0-M1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pin the latest stable Surefire release

When the latest-maven-surefire smoke-test scenarios run, MavenSmokeTest reads this property and passes it as SMOKE_TEST_SUREFIRE_VERSION (dd-smoke-tests/maven/src/test/java/datadog/smoketest/MavenSmokeTest.java:240-245, 595-598), so this change makes CI use 3.6.0-M1. The -M1 suffix is a Maven milestone/pre-release, but this file and the update workflow are explicitly for the latest eligible stable versions; pinning a milestone means these smoke tests no longer cover the latest stable Surefire release and may exercise unreleased behavior instead.

Useful? React with 👍 / 👎.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 7 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-java | test_smoke: [25, 2/8]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-java | test_smoke: [8, 2/8]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-java | test_smoke: [11, 2/8]   View in Datadog   GitLab

View all 7 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 254916b | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.77 s 14.60 s [+0.4%; +1.9%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.47 s 13.65 s [-2.0%; -0.7%] (maybe better)
startup:petclinic:appsec:Agent 16.85 s 16.74 s [-0.2%; +1.6%] (no difference)
startup:petclinic:iast:Agent 16.89 s 16.91 s [-0.9%; +0.6%] (no difference)
startup:petclinic:profiling:Agent 16.66 s 16.87 s [-2.0%; -0.4%] (maybe better)
startup:petclinic:sca:Agent 16.83 s 16.74 s [-0.4%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.02 s 15.58 s [-1.5%; +7.1%] (no difference)

Commit: 26573a65 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: dependencies Dependencies related changes tag: no release notes Changes to exclude from release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants