Comments (14)
@nclopezo did you look into this?
from cms-bot.
Hi,
It seems that it was a glitch (cms-sw/cmssw#7565 (comment)). If this is happening again I can check it.
from cms-bot.
Hi David,
I'm not sure what was a glitch.
The state of RelMon is still the same, at least as of 2 days ago.
Since DAS is now a good generator of large differences in 140.53
I can use it as an example of missing differences in RelMon in 8752
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7174/
RelMon result has essentially no differences
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7174/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/
compare this to the slew of changes in the log file of RelMon (180MB log
file, you can dare to open)
vs
e.g. fwlite pt of tracks
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7174/validateJR/all_OldVSNew_RunHI2011wf140p53/all_OldVSNew_RunHI2011wf140p53c_log10recoTracks_hiGeneralTracks__reRECO_obj_pt.png
--slava
On 4/17/15 3:16 AM, David Mendez wrote:
Hi,
It seems that it was a glitch (cms-sw/cmssw#7565 (comment)
cms-sw/cmssw#7565 (comment)). If
this is happening again I can check it.—
Reply to this email directly or view it on GitHub
#313 (comment).
from cms-bot.
(I can't reopen an issue closed by an admin)
from cms-bot.
Hi Slava,
I modified the threshold to be 0.05, here are the results:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7242/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/
Are these results better? I can try with other values if you want.
from cms-bot.
Hi David,
As a check, could you please generate a diff with 0.005 as a threshold.
In majority of the cases of tests in jenkins we expect no changes at all,
since we are usually rerunning on the same events.
In this test case, where files are different, the number of failed
comparisons is still fairly small.
So, a looser selection would make sense.
(recall that for the alternative-comparisons we applied some logic of
changing
thresholds in diffs depending on how many are found; it's unclear if the
same can be applied here as well)
On 4/21/15 6:21 AM, David Mendez wrote:
Hi Slava,
I modified the threshold to be 0.05, here are the results:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7242/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/
Are these results better? I can try with other values if you want.—
Reply to this email directly or view it on GitHub
#313 (comment).
from cms-bot.
Here are the results with 0.005
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/20/
If I modify it to run first the JR comparison, the thresholds could be assigned in a similar way as for example: https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7174/alternative-comparisons/assignedParameters.log
from cms-bot.
On 4/21/15 8:23 AM, David Mendez wrote:
Here are the results with 0.005
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/20/
Sorry, I misunderstood the direction of the threshold.
It's actually showing differences less probable that the threshold.
So, lets try 0.1 for this case with many differences
(this should be an equivalent of mod 2 in the alternative-comparisons).
The mapping to the choices done in alternative comparisons should be
- 0.999 for mod=3
- 0.1 for mod=2
- 0.999999999999 for mod=1
I'm not sure if there is a switch for mod=0 (any bin difference) in RelMon
If I modify it to run first the JR comparison, the thresholds could be
assigned in a similar way as for example:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/7174/alternative-comparisons/assignedParameters.log—
Reply to this email directly or view it on GitHub
#313 (comment).
from cms-bot.
Right now it is like this: https://github.com/cms-sw/cms-bot/blob/master/comparisons/analyzeFWComparison.py#L65
I can do the same with the thresholds for RelMon comparisons
from cms-bot.
David,
I think the same logic should work for RelMon.
Thank you.
from cms-bot.
Hi Slava,
Here you can see the results with the dynamic assignment of thresholds:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_7_4_X_2015-04-15-1100+8752/38/
You can see in the file RelMonAssignedParameters.log how the parameters were assigned.
And in this RelMonComparisonParams.txt a summary of the thresholds which is used by the script when it launches the RelMon comparison.
from cms-bot.
If this is ok I will merge this new feature to the cms-bot master branch.
from cms-bot.
I merged it and it should now be available for the next comparisons.
from cms-bot.
thank you.
Sorry, I couldn't check earlier.
Looking at the RelMonAssignedParameters.log , the chosen params make sense
from cms-bot.
Related Issues (20)
- parse and monitor JobReport.xml HOT 3
- Security Issue? HOT 12
- bot / github UI get confused if the same branch is used for multiple PRs HOT 3
- Add battibass as muon-dpg L2 HOT 4
- add step5 / NANO for reco release profiling HOT 8
- Test UBI8 and AlmaLinux8 HOT 7
- PF as a PR type HOT 3
- patching command HOT 2
- Integrate part of NANO test in cmsbot commands HOT 2
- Use cvmfs parallel publishers/gateway to deploy CMSSW IBs HOT 22
- Workflow `39634.21` undefined in `igprof` 12_6_X HOT 7
- add automatic labeling of PR types from package lists HOT 12
- Empty `Igprof Comparison cpu usage RECO produce methods` HOT 11
- Include CMSSW non-blocking dpg/pog labels to release notes/summary HOT 3
- Change default wfs for PR tests on GPU in `CMSSW_13_X_Y` HOT 6
- signature delegation on specific packages HOT 4
- Change default action to squash merge in cmssw repo. HOT 45
- [RFC] Code-style for cms-bot scripts HOT 5
- forcing merge of externals once cmssw PR is merged HOT 19
- New PR label for HLT integration HOT 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cms-bot.