Skip to content

Revision of the PSimHit type storage, revert trackId to its basic meaning#49969

Merged
cmsbuild merged 3 commits into
cms-sw:masterfrom
fabiocos:fc-psimhit-20260122
Apr 30, 2026
Merged

Revision of the PSimHit type storage, revert trackId to its basic meaning#49969
cmsbuild merged 3 commits into
cms-sw:masterfrom
fabiocos:fc-psimhit-20260122

Conversation

@fabiocos
Copy link
Copy Markdown
Contributor

PR description:

Following #49732 and the discussion in the Simulation meeting https://indico.cern.ch/event/1634520/contributions/6878108/attachments/3199713/5699141/SIM_20260116.pdf , this PR proposes a revision of the mechanism used to store and propagate the hit type classification, so far used only by MTD for Phase2, but in principle applicable to any interested sub-detector.

trackId is reverted to the pure Geant4 id, without any offset and limitation on the maximum available (beyond the uint32_t capability), so as not to interfere with exceptionally populated events, and software developments for GPU.

The hit type is moved as a 7-bits subfield in the processType member of PSimHit, where the maximum code is at present 403, and this is a used-defined integer, not a pseudo-random variable. The interface is adapted, the assignment of hit type in MTD is adjusted accordingly, and all the dependencies in the code (truth accumulator, validation) are adapted.

PR validation:

Tests on 100 single pions (wf 34506.0) are successful, the detailed Geant4 debug printout shows the desired behaviour, when activated, and the usual DQM histogram test shows perfect comparison between histograms.

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented Jan 28, 2026

cms-bot internal usage

@cmsbuild
Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49969/47752

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Copy Markdown
Contributor

@cmsbuild
Copy link
Copy Markdown
Contributor

A new Pull Request was created by @fabiocos for master.

It involves the following packages:

  • RecoMTD/TimingIDTools (reconstruction)
  • SimDataFormats/CaloAnalysis (simulation)
  • SimDataFormats/TrackingHit (simulation)
  • SimG4CMS/Forward (simulation)
  • SimG4Core/Application (simulation)
  • SimG4Core/Notification (simulation)
  • SimGeneral/CaloAnalysis (simulation)
  • Validation/MtdValidation (dqm)

@Moanwar, @civanch, @cmsbuild, @ctarricone, @gabrielmscampos, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @nothingface0, @rseidita, @srimanob can you please review it and eventually sign? Thanks.
@ReyerBand, @VinInn, @VourMa, @apsallid, @bsunanda, @denizsun, @elusian, @felicepantaleo, @makortel, @martinamalberti, @missirol, @mmasciov, @mmusich, @mtosi, @rovere, @salimcerci, @slomeo, @thomreis, @wang0jin this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@fabiocos
Copy link
Copy Markdown
Contributor Author

@civanch @kpedro88 this is a possible solution to the problem of avoiding limits to trackId, that preserves the existing functionalities. Please comment.

@fabiocos
Copy link
Copy Markdown
Contributor Author

please test

Comment thread SimDataFormats/TrackingHit/interface/SimHitCategory.h Outdated
Comment thread SimDataFormats/TrackingHit/interface/SimHitCategory.h
Comment thread SimGeneral/CaloAnalysis/plugins/MtdTruthAccumulator.cc Outdated
@cmsbuild
Copy link
Copy Markdown
Contributor

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Size: This PR adds an extra 116KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/50974/summary.html
COMMIT: 5606fe1
CMSSW: CMSSW_16_1_X_2026-01-28-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49969/50974/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed Unit Tests

I found 1 errors in the following unit tests:

---> test testPhase2PixelNtuple had ERRORS

Failed RelVals

  • 25202.0A fatal system signal has occurred: segmentation violation
  • 14234.0A fatal system signal has occurred: segmentation violation
  • 312.0A fatal system signal has occurred: segmentation violation
Expand to see more relval errors ...

Failed RelVals-INPUT

  • 11024.211024.2_TTbar_13UP18HEfailINPUT/step2_TTbar_13UP18HEfailINPUT.log

@fabiocos
Copy link
Copy Markdown
Contributor Author

There si apparently a problem in MixingModule, likely reading old PSimHit. The backward compatibility mechanism needs to be verified.

@fabiocos
Copy link
Copy Markdown
Contributor Author

The unit test failure seems to occasionally appear also in the regular IB, and it looks independent on this PR. Here the only differences seen are related to this PR, and they are understood. So, all the other differences appearing in the previous tests with the alternative ROOT6 build are due to the different version of that, without or with the fix by P. Canal.

@cms-sw/orp-l2 @makortel @civanch @kpedro88 please suggest how to move forward with the integration.

@fabiocos
Copy link
Copy Markdown
Contributor Author

unhold

@cmsbuild
Copy link
Copy Markdown
Contributor

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/52599/summary.html
COMMIT: a9fdfc5
CMSSW: CMSSW_17_0_X_2026-04-10-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49969/52599/install.sh to create a dev area with all the needed externals and cmssw changes.

  • Static analyzer reported errors, please check

Failed Unit Tests

I found 1 errors in the following unit tests:

---> test testPhase2PixelNtuple had ERRORS

Failed RelVals

  • 25202.0A fatal system signal has occurred: segmentation violation
  • 14234.0A fatal system signal has occurred: segmentation violation
  • 312.0A fatal system signal has occurred: segmentation violation
Expand to see more relval errors ...

Failed RelVals-INPUT

  • 11024.211024.2_TTbar_13UP18HEfailINPUT/step2_TTbar_13UP18HEfailINPUT.log

@makortel
Copy link
Copy Markdown
Contributor

Is the conclusion that with cms-sw/root#231 the read rule would work? (sorry, I lost track)

@fabiocos
Copy link
Copy Markdown
Contributor Author

I would say yes, the ioread rules do not cause any more crashes, and the MTD histograms show the expected behaviour. Anyway the new ROOT branch introduces many changes everywhere, whose real importance needs to be assessed, and their number is increased by about one order of magnitude if the fix is applied.

Randomly browsing plots here and there, differences are hardly visible, but when you get tens thousand differences it is quite difficult to make a solid statement.

Let me relaunch the test in the "golden" working scenario

@fabiocos
Copy link
Copy Markdown
Contributor Author

please test with cms-sw/root#231 using full cmssw for CMSSW_17_0_ROOT6_X/el8_amd64_gcc13

@cmsbuild
Copy link
Copy Markdown
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/52626/summary.html
COMMIT: a9fdfc5
CMSSW: CMSSW_17_0_ROOT6_X_2026-04-10-2300/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49969/52626/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/52626/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/52626/git-merge-result

Failed Unit Tests

I found 2 errors in the following unit tests:

---> test TestFWCoreReflectionClassVersionUpdate had ERRORS
---> test TestIOPoolInputNoParentDictionary had ERRORS

Comparison Summary

Summary:

  • You potentially added 75 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 23220 differences found in the comparisons
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4181709
  • DQMHistoTests: Total failures: 158539
  • DQMHistoTests: Total nulls: 22
  • DQMHistoTests: Total successes: 4023128
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 4.843 KiB( 52 files compared)
  • DQMHistoSizes: changed ( 10224.0 ): 0.234 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 13034.0 ): 1.113 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 17034.0 ): 3.677 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 18634.0 ): -1.460 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 250202.181 ): 1.938 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 25202.0 ): -0.659 KiB SiStrip/MechanicalView
  • Checked 227 log files, 197 edm output root files, 53 DQM output files
  • TriggerResults: found differences in 6 / 51 workflows

@makortel
Copy link
Copy Markdown
Contributor

Would we then be ready for root-project/root#21669 to be merged in ROOT master (after their review completes) and a backport to 6.36?

Randomly browsing plots here and there, differences are hardly visible, but when you get tens thousand differences it is quite difficult to make a solid statement.

That volume of "hopefully numerical differences" occur pretty much every time between ROOT major versions.

@fabiocos
Copy link
Copy Markdown
Contributor Author

@makortel on my side I believe that, with the fix, this PR seems to be doing what we want. The unit test failures should be addressed in some way, but they seem unrelated to this specific PR, in my understanding.

@makortel
Copy link
Copy Markdown
Contributor

The unit test failures should be addressed in some way, but they seem unrelated to this specific PR, in my understanding.

Correct, those failures are related on "updating ROOT". Those tests by construction lead to such failures (and would be hard to improve). In cmsdist PRs that update ROOT we just ignore those failures.

@pcanal @vepadulano In order to make progress here we'd need root-project/root#21669 merged in ROOT master and backported to 6.36.

@fabiocos
Copy link
Copy Markdown
Contributor Author

hold

given that the ROOT fix needed to get this PR working (thanks a lot to @pcanal for his effort) is not going to be integrated into ROOT 6.36, I agreed with @makortel to hold its integration until CMSSW_20_0_X.

At that point most likely I will remove the ioread rule, as anyway backward compatibility will be broken.

@cmsbuild
Copy link
Copy Markdown
Contributor

Pull request has been put on hold by @fabiocos
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

@civanch
Copy link
Copy Markdown
Contributor

civanch commented Apr 25, 2026

@fabiocos , I am sorry, I had this concern in past but we my (wrong9 impression was that it is possible to converge in 17_0. If we postpone this PR, I would propose to make an intermediate step: PSimHits without fine bit structure but use only integer numbers. We need this in order to continue GPU development. It would be good to have some variant of hits, which is free from track ID overflow.

@cmsbuild
Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49969/49140

@cmsbuild
Copy link
Copy Markdown
Contributor

Pull request #49969 was updated. @Moanwar, @civanch, @cmsbuild, @ctarricone, @gabrielmscampos, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @nothingface0, @rseidita, @srimanob can you please check and sign again.

@fabiocos
Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild
Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cea6a3/52882/summary.html
COMMIT: 1e375b1
CMSSW: CMSSW_17_0_X_2026-04-26-2300/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49969/52882/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 2 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4186963
  • DQMHistoTests: Total failures: 162
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4186781
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 52 files compared)
  • Checked 227 log files, 197 edm output root files, 53 DQM output files
  • TriggerResults: no differences found

@civanch
Copy link
Copy Markdown
Contributor

civanch commented Apr 27, 2026

+1

1 similar comment
@Moanwar
Copy link
Copy Markdown
Contributor

Moanwar commented Apr 27, 2026

+1

@fabiocos
Copy link
Copy Markdown
Contributor Author

unhold

@fabiocos
Copy link
Copy Markdown
Contributor Author

I do not see how the Tracker plots failures in wf 34434.0 could be related to this PR, I let other expert to comment in case.

@makortel
Copy link
Copy Markdown
Contributor

I do not see how the Tracker plots failures in wf 34434.0 could be related to this PR, I let other expert to comment in case.

L1 track trigger has a non-reproducibility #47071

@fabiocos
Copy link
Copy Markdown
Contributor Author

@makortel ok, thanks for confirming

@fabiocos
Copy link
Copy Markdown
Contributor Author

@cms-sw/dqm-l2 any comment?

@gabrielmscampos
Copy link
Copy Markdown
Member

@cms-sw/dqm-l2 any comment?

Sorry, I missed this one. lgtm

@gabrielmscampos
Copy link
Copy Markdown
Member

+dqm

@cmsbuild
Copy link
Copy Markdown
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen
Copy link
Copy Markdown
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.