Skip to content

[NGT] Add workflow to produce Phase2 NANOAOD with L1 and HLT objects #51004

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
elenavernazza wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[NGT] Add workflow to produce Phase2 NANOAOD with L1 and HLT objects #51004

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0d5b238
Add L1 + HLT NanoAOD workflow
elenavernazza May 20, 2026
b6de68e
Apply procModifier to enable L1 + HLT NanoAOD
elenavernazza May 20, 2026
74ef3bd
Protect track associators from missing collection
elenavernazza May 20, 2026
31b674f
Add .774 workflow to ph2_hlt test matrix
elenavernazza May 20, 2026
2cbc11f
Clarify comment
elenavernazza May 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions Configuration/ProcessModifiers/python/nano_l1_hlt_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import FWCore.ParameterSet.Config as cms

# This modifier is for running NANOAOD production from L1 and HLT steps,
# whitout running RECO and PAT steps.
# It excludes all GEN sequences that depend on PAT collections.

nano_l1_hlt = cms.Modifier()
1 change: 1 addition & 0 deletions Configuration/PyReleaseValidation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ The offsets currently in use are:
* 0.771: HLT phase-2 NGT Scouting menu, Alpaka, TICL-v5, TICL-Barrel
* 0.772: HLT phase-2 NGT Scouting menu, with NANO:@NGTScouting
* 0.773: HLT phase-2 NGT Scouting menu, with NANO:@NGTScoutingVal
* 0.774: HLT phase-2 NGT Scouting menu, with NANO:@NGTScoutingVal+@Phase2L1DPGwithGen (L1+HLT objects)
* 0.775: HLT phase-2 NGT Scouting menu with Pixeltracks CA Extension + LST T5s as GeneralTracks
* 0.778 L3 Tracker Muons reconstruction Outside-In first, HLT Muon NanoAOD
* 0.78: Complete L1 workflow
Expand Down
1 change: 1 addition & 0 deletions Configuration/PyReleaseValidation/python/relval_Run4.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@
numWFIB.extend([prefixDet+34.771]) # NGTScouting + alpaka + TICL-v5 + TICL-Barrel
numWFIB.extend([prefixDet+34.772]) # NGTScouting + NANO
numWFIB.extend([prefixDet+34.773]) # NGTScouting + NANO (including validation)
numWFIB.extend([prefixDet+34.774]) # NGTScouting + NANO containing both L1 and HLT objects (including validation)
numWFIB.extend([prefixDet+34.775]) # NGTScouting + Phase2CAExtension&LSTT5 as GeneralTracks

for numWF in numWFIB:
Expand Down
12 changes: 11 additions & 1 deletion Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2205,7 +2205,7 @@ def condition(self, fragment, stepList, key, hasHarvest):
}

upgradeWFs['NGTScoutingWithNanoValid'] = deepcopy(upgradeWFs['HLTPhase2WithNano'])
upgradeWFs['NGTScoutingWithNanoValid'].suffix = '_NGTScoutingWithNanoVal'
upgradeWFs['NGTScoutingWithNanoValid'].suffix = '_NGTScoutingWithNanoValid'
upgradeWFs['NGTScoutingWithNanoValid'].offset = 0.773
upgradeWFs['NGTScoutingWithNanoValid'].step2 = {
'-s':'DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW,HLT:NGTScouting,VALIDATION:@hltValidation,NANO:@NGTScoutingVal',
Expand All @@ -2214,6 +2214,16 @@ def condition(self, fragment, stepList, key, hasHarvest):
'--eventcontent':'FEVTDEBUGHLT,NANOAODSIM'
}

upgradeWFs['L1NGTScoutingWithNano'] = deepcopy(upgradeWFs['HLTPhase2WithNano'])
upgradeWFs['L1NGTScoutingWithNano'].suffix = '_L1NGTScoutingWithNanoValid'
upgradeWFs['L1NGTScoutingWithNano'].offset = 0.774
upgradeWFs['L1NGTScoutingWithNano'].step2 = {
'-s':'DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW,HLT:NGTScouting,VALIDATION:@hltValidation,NANO:@NGTScoutingVal+@Phase2L1DPGwithGen',
'--datatier':'GEN-SIM-DIGI-RAW,NANOAODSIM',
'--procModifiers': 'ngtScouting,nano_l1_hlt',
'--eventcontent':'FEVTDEBUGHLT,NANOAODSIM'
}
Copy link
Copy Markdown
Contributor

@bfonta bfonta May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about manual configuration, will this step run if the user forgets to specify @Phase2L1DPGwithGen, despite having nano_l1_hlt active?

Copy link
Copy Markdown
Contributor

@mmusich mmusich May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would a user do that? Following wrong recipes results typically in wrong results, no?

Copy link
Copy Markdown
Contributor

@bfonta bfonta May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure; I was just suggesting that adding some protection might help.

Copy link
Copy Markdown
Contributor Author

@elenavernazza elenavernazza May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NANO:@NGTScoutingVal+@Phase2L1DPGwithGen step should be the only ingredient required to produce the L1+HLT NanoAOD sample.
The nano_l1_hlt procModifier is introduced as a protection against missing PAT collections that are required in some of the L1Nano sequences. The alternative would have been to reorganize the L1Nano configuration itself; however, this would have modified the standard L1Nano format used by existing L1 workflows and users, which I wanted to avoid.

Copy link
Copy Markdown
Contributor

@bfonta bfonta May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elenavernazza It is clear what nano_l1_hlt achieves. My comment was rather addressing future users that might use this new process modifier without fully knowing the required inputs. I don't know what happens if the user specifies only NANO:@NGTScoutingVal, but the potential crash might be cryptic. I was suggesting you could add some checks / protections that throw a meaningful error message if the user forgets to add @Phase2L1DPGwithGen.

Copy link
Copy Markdown
Contributor

@mmusich mmusich May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, why should it?

Copy link
Copy Markdown
Contributor

@bfonta bfonta May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comment.

Copy link
Copy Markdown
Contributor

@mmusich mmusich May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see #51004 (comment).

by this token, should you test all possible workflow combinations in the matrix ?

Copy link
Copy Markdown
Contributor

@bfonta bfonta May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never suggested this.
I tested NANO:@NGTScoutingVal with nano_l1_hlt, since the bot does not test it, and since I thought that it could lead to a crash, due to the absence of @Phase2L1DPG.
There is no crash, so all is well.

Copy link
Copy Markdown
Contributor

@mmusich mmusich May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the bot does not test it,

It is straightforward to see from the code changes that this should not lead to a runtime crash.

The only modifications affecting NANO:@NGTScoutingVal are in HLTrigger/NGTScouting/python/HLTNanoProducer_cff.py and are limited to:

  • removing the dstTriggerAcceptFilter filter for the production of the HLT tables
  • removing the event selection in the output

As already stated in the PR description, these changes simply allow all events to be written to the output NanoAOD, independently of the L1/HLT decision. No producer dependencies or runtime execution paths are modified in a way that could plausibly introduce a crash.

Because of that, I am a bit puzzled that this specific combination of steps/process modifiers was singled out in the review, while there are already many other untested combinations present in CMSSW that would be equally (or more) likely to expose runtime issues.


class UpgradeWorkflow_HLTwDIGI75e33(UpgradeWorkflow):
def setup_(self, step, stepName, stepDict, k, properties):
if 'DigiTrigger' in step:
Expand Down
1 change: 1 addition & 0 deletions Configuration/PyReleaseValidation/scripts/runTheMatrix.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ def runSelected(opt):
prefixDet+34.771, # HLT phase-2 NGT Scouting menu, Alpaka, TICL-v5, TICL-Barrel
prefixDet+34.772, # HLT phase-2 NGT Scouting menu, with NANO:@NGTScouting
prefixDet+34.773, # HLT phase-2 NGT Scouting menu, with NANO:@NGTScoutingVal
prefixDet+34.774, # HLT phase-2 NGT Scouting menu, with NANO:@NGTScoutingVal+@Phase2L1DPGwithGen
prefixDet+34.775], # HLT phase-2 NGT Scouting menu, Phase2CAExtension&LSTT5 as GeneralTracks
}

Expand Down
29 changes: 25 additions & 4 deletions DPGAnalysis/Phase2L1TNanoAOD/python/l1tPh2Nano_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ def addPh2GTObjects(process):
from DPGAnalysis.Phase2L1TNanoAOD.l1tPh2Nanotables_cff import *
def addPh2L1Objects(process):
process.l1tPh2NanoTask.add(p2L1TablesTask)

# This modifier excludes the hpsTauTable, which is based on the l1tHPSPFTauProducerPuppi collection, no longer available in the L1 menu.
# This allows to run the NANOAOD production from L1 and HLT steps, whitout requiring old inputs from the Spring24 datasets.
from Configuration.ProcessModifiers.nano_l1_hlt_cff import nano_l1_hlt
nano_l1_hlt.toReplaceWith(p2L1TablesTask, p2L1TablesTask.copyAndExclude([hpsTauTable]))

return process

#### GENERATOR INFO
Expand All @@ -25,6 +31,8 @@ def addPh2L1Objects(process):
from PhysicsTools.NanoAOD.taus_cff import * ## for Gen taus
def addGenObjects(process):

process.genNanoTask = cms.Task()

## add more GenVariables
# from L1Ntuple Gen: https://github.com/artlbv/cmssw/blob/94a5ec13b8ce76afb8ea4f157bb92fb547fadee2/L1Trigger/L1TNtuples/plugins/L1GenTreeProducer.cc#L203
genParticleTable.variables.vertX = Var("vertex.x", float, "vertex X")
Expand All @@ -45,22 +53,35 @@ def addGenObjects(process):
process.prunedGenParticleTable = genParticleTable.clone()
process.prunedGenParticleTable.src = "prunedGenParticles"
process.prunedGenParticleTable.name = "prunedGenPart"
process.l1tPh2NanoTask.add(process.prunedGenParticleTable)
process.genNanoTask.add(process.prunedGenParticleTable)

# lower genVisTau pt threshold
process.genVisTauTable.cut = "pt > 1"
# lower AK8 gen jet threshold
process.genJetAK8Table.cut = "pt > 10"

process.l1tPh2NanoTask.add(
process.genNanoTask.add(
puTable, metMCTable,
genParticleTask, genParticleTablesTask,
genTauTask,
)

# add all GenJets: AK4 and AK8
process.l1tPh2NanoTask.add(genJetTable,patJetPartonsNano,genJetFlavourTable)
process.l1tPh2NanoTask.add(genJetAK8Table,genJetAK8FlavourAssociation,genJetAK8FlavourTable)
process.genNanoTask.add(genJetTable,patJetPartonsNano,genJetFlavourTable)
process.genNanoTask.add(genJetAK8Table,genJetAK8FlavourAssociation,genJetAK8FlavourTable)

process.l1tPh2NanoTask.add(process.genNanoTask)

# This modifier excludes all GEN sequences that depend on PAT collections.
# This allows to run the NANOAOD production from L1 and HLT steps, whitout requiring PAT inputs.
from Configuration.ProcessModifiers.nano_l1_hlt_cff import nano_l1_hlt
nano_l1_hlt.toReplaceWith(process.genNanoTask,
process.genNanoTask.copyAndExclude(
[puTable,
metMCTable,
genJetAK8Table,
genJetAK8FlavourAssociation,
genJetAK8FlavourTable]))

return process

Expand Down
9 changes: 9 additions & 0 deletions HLTrigger/NGTScouting/python/HLTNanoProducer_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,9 @@
+ NanoPixelTables
)

from Configuration.ProcessModifiers.nano_l1_hlt_cff import nano_l1_hlt
nano_l1_hlt.toReplaceWith(dstValidationNanoFlavour, dstValidationNanoFlavour.copyAndExclude([dstTriggerAcceptFilter]))

######################################
# Customization
######################################
Expand All @@ -151,4 +154,10 @@ def hltNanoCustomize(process):
)
)

# disable SelectEvents for nano_l1_hlt
nano_l1_hlt.toModify(
process.NANOAODSIMoutput,
SelectEvents = cms.untracked.PSet()
)

return process
4 changes: 4 additions & 0 deletions PhysicsTools/NanoAOD/plugins/TrackingAssocValueMapsProducer.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ void TrackingAssocValueMapsProducer::produce(edm::Event& iEvent, const edm::Even
iEvent.getByToken(trackAssociatorToken_, associatorH);
inputsValid = inputsValid && associatorH.isValid();
}
} else {
LogDebug(metname) << "Invalid handles: Track handle valid: " << tracksH.isValid()
<< ", TP handle valid: " << tpH.isValid();
return;
}

const size_t nTracks = tracksH->size();
Expand Down