zippapp/ macOS: .DS_Store files should be excluded from the generated archives about cpython HOT 6 OPEN

Quietly ignoring a specific file on MacOS does not seem like the desirable behavior to me. Currently the bahavior of zipapp is "pack everything", which is easy to understand. Changing it to "pack everything except for files with name .DS_Store" is kind of weird. There are other commonly generated files, __pycache__ or .coverage for example. Who should be the person to decide what's on the list and what's not?

To achieve your goal, the Python API zipapp.create_archive provides a way to do it. You can use the argument filter to filter the paths. It should be easy to create your own script to generate the zipapp file with customized filtering. I don't believe changing the default behavior is the correct direction.

I can, somehow, imagine a new option to ignore all hidden files, which is probably more generic to users.

And this is definitely not a bug, it's a feature request, so I'll change the lables.

from cpython.

If we plan to add a command line option to filter, I think we should avoid the name filter, unless we can make it comparable to the filter argument for create_archive. The filter argument is a function that returns a boolean indicating whether the path should be accepted. The only way I can think of to make the option filter work is for it to pass in a regular expression that provides a pattern for all accepted paths (not declined, or it will have opposite meaning to the argument).

A better way might be --include and --exclude which is also commonly used to provide both allow-list and deny-list.

I'm a bit surprised that Finder will generate the file in the background (when it's not opened in the OS) though.

from cpython.

I do not agree. I think by default they should be excluded but there should be an option to include them.

from cpython.

Quietly ignoring a specific file on MacOS does not seem like the desirable behavior to me. Currently the bahavior of zipapp is "pack everything", which is easy to understand. Changing it to "pack everything except for files with name .DS_Store" is kind of weird. There are other commonly generated files, __pycache__ or .coverage for example. Who should be the person to decide what's on the list and what's not?

To achieve your goal, the Python API zipapp.create_archive provides a way to do it. You can use the argument filter to filter the paths. It should be easy to create your own script to generate the zipapp file with customized filtering. I don't believe changing the default behavior is the correct direction.

I can, somehow, imagine a new option to ignore all hidden files, which is probably more generic to users.

And this is definitely not a bug, it's a feature request, so I'll change the lables.

agree 100%

from cpython.

Quietly ignoring a specific file on MacOS does not seem like the desirable behavior to me. Currently the bahavior of zipapp is "pack everything", which is easy to understand. Changing it to "pack everything except for files with name .DS_Store" is kind of weird. There are other commonly generated files, __pycache__ or .coverage for example. Who should be the person to decide what's on the list and what's not?

Adding a filter option to the zipapp command line interface could be useful, but as you write the hard bit is which name should be included in the default filter.

Two options (which could both be added)

• python -m zipapp --filter GLOB ...: Multi-use argument for specifying which names should be excluded (user needs to determine useful filters themselves)
• python -m zipapp --filter-junk ..: Filter some commonly seen files that should generally not be included, such as .DS_Store, __pycache__ and backup files (*~, ._*.swp, ...).

The second option is a nicer user interface for users, but is harder to get right (both naming and which filters to include).

Currently the only API implements something similar to the second option is dircmp, other APIs try to avoid the temptation to guess ;-).

from cpython.

The problem with the .DS_Store files (unlike other temporary junk mentioned here) is that they can come back randomly and quickly during the time the zippapp is created. E.g. I do run a cleanup before already after I noticed the issue with .DS_Store files ballooning the sizes of the generated archives:

find zipapptempdir -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete -o -type f -name '.DS_Store' -delete && \
  python3.12 -m zipapp --output s3cmd.pyz --compress --python /usr/local/bin/python3.12 zipapptempdir

And in one case a .DS_Store file still managed to sneak into the archive. It's very hard to make this work reliable from a single user account. Maybe killing off Finder before, and then relaunching it afterwards or some other awkward kludge?

I'm reporting about the the zipapp app/module itself (as used in python3 -m zipapp from a user perspective) and I'm perfectly fine with some --filter REGEX or --reject GLOB option, if that's more suitable than just default filtering out the .DS_Store files. 👍

I would consider those files getting added as a privacy/security/ information leak issue (hence a bug report), e.g. when I plan to distribute generated zipapps to other users: they really don't need my .DS_Store files.

from cpython.