moveonorg / email2hash Goto Github PK

We are currently sorting the list of hashes and then outputting the list to a file. We outline the reasons for doing so in the spec:

    The script computes the HMAC (see 4 below) for each email address in the
    CSV file and saves it to a list. The list is then sorted to make it easier
    to check for overlap and to hide the information about member activity
    before being written to disk.

Instead of doing this, we should preserve the order in which the email addresses were hashed (as we were doing earlier) and append the line number in front of them, like this:

1,<hash-1>
2,<hash-2>

The benefits of this:

1. The comparisons will be easier as the hashed list can be compared with the raw list by comparing the index. So for example if the raw list was:

1,[email protected]
2,[email protected]

and the hashed list is:

1,<hash-1>
2,<hash-2>

It's easy to compare the overlap by checking the index and matching it to find the corresponding email address.

2. We no longer need to deal with (the memory overhead) of a list that will have millions of email addresses; instead we write the output as we process directly to the file.

The timer should start when the hashing starts, and not when the script starts running.

We shouldn't be supporting any hashing algorithm older than SHA3. Perhaps we could support BLAKE2 as well but not required currently. Let's make SHA3 256 default for now.

We currently ask the user to input a secret key. We should automatically do that for them if they don't want to choose their own.

Dependabot couldn't find a requirements.txt for this project.

Dependabot requires a requirements.txt to evaluate your project's current Python dependencies. It had expected to find one at the path: /requirements.txt.

If this isn't a Python project, or if it is a library, you may wish to disable updates for it from within Dependabot.

You can mention @dependabot in the comments below to contact the Dependabot team.