mayConsume runs prefetching

[silviodonato]

Dear @cms-sw/core-l2 ,
the scouting DQM currently runs only at T0 in the ScoutingPFMonitor dataset, which contains the scouting objects in all events.
I would like to run the scouting DQM in the Run-3 RelVals.

The problem is that in RelVals we might not have scouting objects in all events and this causes a crash:

cmsRun onlyJMEDQM_DQM.py
12-Nov-2025 15:05:48 CET  Initiating request to open file file:step2.root
12-Nov-2025 15:05:51 CET  Successfully opened file file:step2.root
Begin processing the 1st record. Run 391668, Event 93152, LumiSection 1 on stream 0 at 12-Nov-2025 15:05:54.819 CET
----- Begin Fatal Exception 12-Nov-2025 15:05:54 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 391668 lumi: 1 event: 93152 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module JetAnalyzer/'jetDQMAnalyzerAk4ScoutingUncleaned'
   [3] Prefetching for module ChainedJetCorrectorProducer/'ak4PFScoutingL1FastL2L3ResidualCorrector'
   [4] Calling method for module L1FastjetCorrectorProducer/'ak4PFScoutingL1FastjetCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: double
Looking for module label: hltScoutingPFPacker
Looking for productInstanceName: rho

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------
12-Nov-2025 15:05:54 CET  Closed file file:step2.root

The cause of this crash is that jetDQMAnalyzerAk4ScoutingUncleaned (JetAnalyzer) consumes
ak4PFScoutingL1FastL2L3ResidualCorrector (ChainedJetCorrectorProducer) that consumes
ak4PFScoutingL1FastjetCorrector (L1FastjetCorrectorProducer) that consumes
hltScoutingPFPacker rho

I tried to replace all consumes with mayConsume, but I'm still getting the error in the prefetching.

The desired behaviour is to have no prefetching for module with mayConsume to avoid the crash.
In this way we can avoid to run getByToken if the scouting objects is missing (here the code would be already ok).

I copied here a simplified configuration that you can use to reproduce the error in CMSSW_16_0_0_pre2.

From SWGuideEDMGetDataFromEvent I read

NOTE: At this time mayConsume() is functionally equivalent to consumes(). Properly functioning "may consume" would require further development and a slightly different API.

so I don't know if this is a planned feature that is not ready yet, or if it is something new.

Thank you,
Silvio.

fyi @cms-sw/hlt-l2 @patinkaew

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @silviodonato.

@Dr15Jones, @ftenchini, @makortel, @mandrenguyen, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[Dr15Jones]

mayConsumes is pretty much an anachronism from pre-threading days. The framework requires ALL requests to be prefetched. There is no plan for making mayConsumes change behavior (in fact we should remove it all together).

[mmusich]

stupid suggestion: can't we check the handle validity here:

cmssw/JetMETCorrections/Algorithms/src/L1FastjetCorrectorImpl.cc

Line 43 in 7f36f1b

fEvent.getByToken(rhoToken_, hRho);

and return a default in case it's not valid? Likely consumers need to be adapted.

[Dr15Jones]

There are a number of ways you could accomplish what you want

• You could make the getting of the data product controlled by a configuration parameter.
• You could ask for the result but then check the edm::Handle to see if it is valid and if it is not valid do not dereference it.
• You could also use the callWhenNewProductsRegistered method which gets called a new data product is registered. You then look to see if the scouting data was registered and if so, only then do consumes for it. See https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideEDMGetDataFromEvent#Consumes_and_Helpers

[makortel]

assign dqm

[cmsbuild]

New categories assigned: dqm

@ctarricone,@gabrielmscampos,@nothingface0,@rseidita you have been requested to review this Pull request/Issue and eventually sign? Thanks

[Dr15Jones]

Also, why are you running a configuration on a file that doesn't contain the data you need?

[silviodonato]

Also, why are you running a configuration on a file that doesn't contain the data you need?

the idea is to have the scouting DQM as part of DQM step. So the scouting objects will be produced and saved by the previous step HLT. The problem is that in some configuration it can happens that we don't reconstruct the scouting objects in all events (eg. because some of them were triggered by Randoms or because we are running a fake HLT menu).
(see #49358)

• You could make the getting of the data product controlled by a configuration parameter.

Only some events do not have scouting objects, while with a configuration parameter I would activate or disactivate the getting of the data product for all events, if I understand correctly

• You could ask for the result but then check the edm::Handle to see if it is valid and if it is not valid do not dereference it.

Do you mean to use iEvent.getHandle(scoutingPfJetsToken_) in JetAnalyzer::analyze to check if we have scouting jets? and if we don't have them avoid to run iEvent.getByToken(jetCorrectorToken_, jetCorr);?
The problem is inside the constructor JetAnalyzer::JetAnalyzer , specifically in the line consumes<reco::JetCorrector>(jetCorrectorTag_). I can even remove completely iEvent.getByToken(jetCorrectorToken_, jetCorr); from the analyze and I still get the crash, because the crash comes from the constructor.

• You could also use the callWhenNewProductsRegistered method which gets called a new data product is registered. You then look to see if the scouting data was registered and if so, only then do consumes for it. See https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideEDMGetDataFromEvent#Consumes_and_Helpers

If callWhenNewProductsRegistered checks also the data already existing in the input file it should work.

If it helps , the collection double_hltScoutingPFPacker_rho_reHLT exists in step2.root but double_hltScoutingPFPacker_rho_reHLT.isPresent() is 0.

[makortel]

and return a default in case it's not valid? Likely consumers need to be adapted.

An alternative would be to define an EDFilter that accepts only Events where the scouting data products are available, put that filter as a first module in a Sequence Path, and place all of the scouting-depending modules in that Sequence Path behind the filter.