Skip to content

LST: Remove Matrix Caps for MDs using Precompute#50856

Merged
cmsbuild merged 1 commit into
cms-sw:masterfrom
SegmentLinking:md_full_precompute
May 5, 2026
Merged

LST: Remove Matrix Caps for MDs using Precompute#50856
cmsbuild merged 1 commit into
cms-sw:masterfrom
SegmentLinking:md_full_precompute

Conversation

@GNiendorf
Copy link
Copy Markdown
Contributor

PR description:

Most LST objects (segments, triplets, quintuplets, ...) size their buffers from an upper-bound estimate computed in an initial counting pass before object creation (see previous PR's here: #50157, #48698, #47232). Memory for LST mini-doublets (two hits in a given module) are currently allocated using the minimum of a loose dynamic estimate (n hits lower sensor x n hits upper sensor) and static caps based on detector region. These static caps help reduce overallocation from the loose dynamic estimate but can cause truncation for high-occupancy events and have to be re-tuned by hand when hit selections or pT cuts change. This PR replaces these static caps with a precomputed exact-count: We first compute the number of mini-doublets that will be created in a counting pass and then allocate an exact-size buffer to store them. This eliminates possible truncation and reduces the memory allocated for mini-doublets for a small increase in overall timing. Mini-doublet memory drops from 31.4 MB to 7.4 MB on average (~22% decrease in total memory/event for LST) for a ~7% increase in LST time/event.

c.c @slava77

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 1, 2026

cms-bot internal usage

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 1, 2026

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50856/49211

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 1, 2026

A new Pull Request was created by @GNiendorf for master.

It involves the following packages:

  • RecoTracker/LSTCore (reconstruction)

@Moanwar, @cmsbuild, @jfernan2, @mandrenguyen, @srimanob can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @VinInn, @VourMa, @dgulhan, @elusian, @felicepantaleo, @gpetruc, @mmasciov, @mmusich, @mtosi, @rovere this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Copy Markdown
Contributor

slava77 commented May 1, 2026

test parameters:

  • enable = hlt_p2_integration, hlt_p2_timing
  • workflows = ph2_hlt

@slava77
Copy link
Copy Markdown
Contributor

slava77 commented May 1, 2026

@cmsbuild please test

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 1, 2026

-1

Failed Tests: HLTP2Timing
Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ff23d0/53013/summary.html
COMMIT: 327176e
CMSSW: CMSSW_17_0_X_2026-05-01-1600/el8_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/50856/53013/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
1000.0 step 2
1001.0 step 2
101.0 step 1
10224.0 step 1
11634.0 step 1
12434.0 step 1
12834.0 step 1
12846.0 step 1
13034.0 step 1
1306.0 step 1
13234.0 step 1
1330.0 step 1
135.4 step 1
136.731 step 2
136.793 step 2
136.874 step 2
139.001 step 2
140.56 step 2
14034.0 step 1
14234.0 step 1
16834.0 step 1
17034.0 step 1
18434.0 step 1
18634.0 step 1
2022.0010001 step 2
2023.0020001 step 2
2024.0000001 step 2
2024.0010001 step 2
2024.0020001 step 2
2024.0030001 step 2
2024.0040001 step 2
2024.0050001 step 2
2024.0060001 step 2
2024.0070001 step 2
2025.0000002 step 2
2025.0010001 step 2
25.0 step 1
2500.3001 step 2
250202.181 step 1
25202.0 step 1
312.0 step 1
34434.0 step 1
34434.75 step 1
34434.911 step 1
34496.0 step 1
34500.0 step 1
34634.999 step 1
4.22 step 2
4.53 step 2
5.1 step 1
7.3 step 1
8.0 step 1
9.0 step 1
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

  • You potentially added 21762 lines to the logs
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 16
  • DQMHistoTests: Total histograms compared: 616253
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 616253
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 15 files compared)
  • Checked 131 log files, 49 edm output root files, 16 DQM output files
  • TriggerResults: found differences in 5 / 15 workflows

@slava77
Copy link
Copy Markdown
Contributor

slava77 commented May 3, 2026

@cmsbuild please test

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 4, 2026

-1

Failed Tests: HLTP2Timing
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ff23d0/53037/summary.html
COMMIT: 327176e
CMSSW: CMSSW_17_0_X_2026-05-03-2300/el8_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/50856/53037/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 68
  • DQMHistoTests: Total histograms compared: 4803421
  • DQMHistoTests: Total failures: 836
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4802565
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 67 files compared)
  • Checked 284 log files, 246 edm output root files, 68 DQM output files
  • TriggerResults: found differences in 4 / 66 workflows

@slava77
Copy link
Copy Markdown
Contributor

slava77 commented May 4, 2026

Failed Tests: HLTP2Timing

@mmusich @fwyzard

I tried to understand the error, but the log looks OK:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ff23d0/53037/hlt-p2-timing.log
Did I miss something?
Please check.
Thank you.

@fwyzard
Copy link
Copy Markdown
Contributor

fwyzard commented May 4, 2026

Indeed, I don't see any errors in the log 🤷🏻‍♂️

@smuzaffar
Copy link
Copy Markdown
Contributor

smuzaffar commented May 4, 2026

the error is

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02940/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_17_0_X_2026-05-03-2300/src/HLTrigger/Configuration/python/HLT_75e33/test/runHLTTiming.sh: line 151: 1590853 Terminated              tail -f "$TMP_LOG_FILE"

#50867 should fix it.

@smuzaffar
Copy link
Copy Markdown
Contributor

please test with #50867

#50867 should fix the hlt p2 timing tests

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 4, 2026

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ff23d0/53047/summary.html
COMMIT: 327176e
CMSSW: CMSSW_17_0_X_2026-05-03-2300/el8_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/50856/53047/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

  • You potentially added 4 lines to the logs
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 68
  • DQMHistoTests: Total histograms compared: 4803421
  • DQMHistoTests: Total failures: 1681
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4801720
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 67 files compared)
  • Checked 284 log files, 246 edm output root files, 68 DQM output files
  • TriggerResults: found differences in 5 / 66 workflows

@jfernan2
Copy link
Copy Markdown
Contributor

jfernan2 commented May 4, 2026

+1

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented May 4, 2026

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @ftenchini, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen
Copy link
Copy Markdown
Contributor

+1

@cmsbuild cmsbuild merged commit 6333b5e into cms-sw:master May 5, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants