Handle un-scannable files more gracefully about detect-secrets HOT 5 CLOSED

Thanks @dgzlopes! 👍 🎈 🎉 🎂

from detect-secrets.

Hello! I don't understand why the IGNORED_FILE_EXTENSIONS structure should change from a tuple to a dict. Could you offer some more insight on this?

On the other side, I made this quick fix dgzlopes@3828be9 on my fork (handling the check inside scan_file() instead of _extract_secrets_from_file()) and seems to work ok. Any idea on where to handle it more gently?

from detect-secrets.

The reason why I would like to change it to a dict is because the only use is an in check, which is faster in a hash table (dictionary). It's less for performance reasons and more for knowing how it is used, from seeing the type.

from detect-secrets.

Oh! I see. On the other hand (and sorry if this question is stupid) why we can't use a Set()?

They have both the same time complexity (O(1) for membership checking on the average case [0]) and both are implemented with a Hashtable. Also our file extensions values fit in one dimension [1], are unique and can be unordered. Also the Python docs [3] states that they are a good option for membership testing.

[0] https://wiki.python.org/moin/TimeComplexity#set
[1] https://stackoverflow.com/questions/19454970/is-there-a-python-dict-without-values
[2] https://docs.python.org/3/library/stdtypes.html#set

from detect-secrets.

Oh, I wasn't clear, I did mean set.
e.g. replacing the ( with { and ) with }

A dictionary without keys is a set, so I kind of used them interchangeably.

from detect-secrets.