We need to add some example likelihoods so users know how to build theirs when they want to use the blinding infrastructure.

Ideally we should implement an example from:

• Cosmic Shear alone
• 3x2pt
• SNe Ia
• Strong Lensing
• Clusters

Anything else?

Hello there! While attempting to install using the environment.yml file, I encountered the error message "Solving environment: / Killed". Trying to understand what was causing this issue, I found that fitsio and pyccl dependencies in the .yml file were necessary. After adding these dependencies, the installation proceeded successfully.

A basic example of how to run and check the shifts.

The project toml and other setup files from legacy blinding interfere with building the new package.

We will move them into a new repository alongside Legacy Blinding related issues.

To Do:

• Create new legacy blinding repository
• Move code to the new repository
• Move legacy related issues to the new repository
• Remove legacy blinding files and scripts from this repository

Change the namespace and the repository name.

Ensure name changes in

• installation files
• examples
• documentation
• example notebooks
• unit tests
• README.md

A mixed DM/CMB/LSST group spent the final afternoon at the KIPAC workshop trying to summarize the workshop proceedings in a shared doc here. These notes can act as a starting point for us: I guess the Note we write for CX8 will be more extensive, and LSST-focused. Still, comments welcome in place, or in this thread!

Talk to TXPipe devs prior to this.

This issue will track the implementation of Jessie's Muir legacy cosmosis blinding scripts.

The scripts will be adapted to Cosmosis v2 and interfaced with SACC instead of the standard cosmosis fits outputs.

Current Tasks:

• adapt Jessie's script to use Cosmosis v2 and remove DES Lib dependencies
• Generate a fiducial test data:
  • in fits format
  • in SACC format
• re-plot the fiducial/blinded with the new fiducial datavector
• Re-package the legacy scripts
• Make the legacy scripts pip installable
• Re-package the plotting scripts interfacing it with SACC
• Documentation
• Tests:
  • Unit
  • Regression

Discussion points:

• How do we want to deal with the legacy scripts as we move towards the package that will interface with firecrown and TXPipe? Do we keep them as a subpackage in this repo or do we keep it in a "legacy" branch (not a fan of multiple branches with different implementations!)

Cc: @jablazek and @jessmuir

Allow for SACC files to be used and saved in the legacy blinding.

Following this example: https://github.com/LSSTDESC/sacc/blob/master/examples/Convert_DES_Sacc.ipynb

Discussions in the collaboration raised the point that the term "blinding" can be perceived as problematic and DESC should referr to this practice by another term instead.

Discussion in telecons and slack suggested terms like:

• conceal
• vail (also pointed out as sensitive)
• cloak
• etc

The package also needs a new naming.

Please feel free to discuss these terms and suggestions.

Notice that there are two terms in discussion here:

1. A higher level discussion based on what we will call the act of "blinding" our data and analysis
2. Just the name of this package.

Need to implement proper documentation in readthedocs for this library

Save the blinding data-vector as a sacc file

I was interested in whether the specific question of how to do a blind analysis with open data and (well documented, open) software (ie. where at any stage, a different person could hopefully work out the un-blinded results and therefore un-blind it) was discussed. Is there a clear method of doing this, or is there an assumption that no person would do this and make these common knowledge?

This issue will track comparisons between the legacy Cosmosis and the current TXPipe blinding implementations.

Questions

• Can TXPipe blind a data-vector directly or only blind a data-vector from applying estimators to a catalogue?
• Can cosmosis output in SACC format? Relevant to #6 too.

Tasks

• Output a fiducial data-vector from DESC pipelines
• Compare fiducial cosmology data-vectors between TXPipe (maybe firecrown?) and Cosmosis v2
• Compare blinding ratios between both approaches
• Investigate discrepancies

Cc: @jablazek @jessmuir

I guess we'll want this to be latex (so we can cite a lot of literature properly), with PDF that builds when master is updated. I'll set something up, so that we can make a good start here on the last day of the KIPAC blinding workshop

We have the flat shift implemented but not gaussian.

We need a base sacc object to be able to save the blinded data-vector.

This means we cannot make sacc_data an optional parameter for the main blinding module.

Need to update:

• Blinding Class
• Examples
• Unit tests

Current shift types are contextual and could leave to issues:

Ex:

• {'Omega_c': 0.3} is a determinist shift
• {'Omega_c': (0.2, 0.4)} is a range
• {'Omega_c': (-0.1, 0.1)} is a delta shift

This can be problematic for parameters like w!

Github actions is failing the tests as it cannot properly install Cosmosis.

Instructions here may help to fix that: https://github.com/joezuntz/cosmosis/blob/main/.github/workflows/ci.yml

New version of Firecrown makes it much easier to get a dictionary of systematic parameters from the likelihood (which then can easily get it from the SACC file).

To Do:

• Update to Firecrown 1.8
• Test implementing these capabilities in the blinding infrastructure
• Re-write the examples to match these changes
• Update the unit tests

More information here: https://firecrown.readthedocs.io/en/latest/_static/two_point_generators.html

The main blinding class needs to be adapted to receive either a sacc file path or a sacc object which is then built with the likelihood.

We can support either a dictionary or a "NamedParameter" type.

To call things with cosmosis and cobaya, @tilmantroester says we should pass this as a NamedParameter.
If that is the case, this should be the standard for this library too.

There are a few models for 3x2pt data vector lying around in Augur, TXPipe, Firecrown, etc... we need to determine which and how it will be implemented in the Blinding Library.

Examples:

• TXPipe
• Augur
• Firecrown - DES 3x2pt (Unclear Firecrown already has an official 3x2pt model as it seem to have for clusters... )

Ideally, we should use the one that is officially packaged from firecrown.

This is used inside the main module of smokescreen but also in the utils.

The reason I am not updating now is that I am not able to build CCL from source and v3.0.1 is not yet on conda-forge.

As soon as it is, I can update the code to use this new functionality.

Or some way to run it with debug mode.

Hi! I made a repo to host discussion, literature review, documentation, test code etc for key project CX8, "Blind Analysis Strategies". There's a few of us together here at KIPAC this week for a workshop on blind analysis, so hopefully we can start writing something on our preliminary thinking in this area. Issues we can anticipate, things that are likely to work well (and badly), and so on. I made a team with write access to this repo, @LSSTDESC/blind-analysts and made @elikrause a team maintainer - lets add a few more people, who we know are interested - others, please do either watch this repo for notifications, or let us know if you want write access so you don't have to contribute text from a fork.

Implement a main function that can be called from the terminal and blinds a SACC file provided by the user.

Code coverage will be important to keep track of the unit tests implemented

If the user gives a single value in the shifts dictionary, do a deterministic shift instead.