mkfrogfs.py directory filters are broken about frogfs HOT 14 CLOSED

Oh interesting. Okay. I will work on this later tonight. Sorry I didn't do any local testing here. I've just experienced these things first hand when testing on my own during development. Thanks for your help in bringing these issues to my attention!

from frogfs.

This is normal behavior for Python 3. It generates optimized bytecode along side source files and stashes it in there. I confirmed that __pycache__/ is in the .gitignore, so it shouldn't get pushed.

from frogfs.

Web Root: webroot.zip
And the config file:

filters:
  '/.test-data*':
    - discard
  '*.json':
    - compress deflate
  '*.js':
    - compress deflate
  '*.css':
    - compress deflate
  '*.woff*':
    - no compress
  '*':
    - compress heatshrink:
        window: 11
        lookahead: 4

from frogfs.

Ah, try without the leading slash. It might seem ambiguous, but without a glob at the beginning, the filter starts with the root.

from frogfs.

It still won't work :)

       - Stage 1
         - index.html
           - compress heatshrink... done (41.0%)
       - Stage 2
Traceback (most recent call last):
  File "E:\Espressif\workspace\test-proj\components\frogfs\tools\mkfrogfs.py", line 528, in <module>
    generate_file_header(entry)
  File "E:\Espressif\workspace\test-proj\components\frogfs\tools\mkfrogfs.py", line 386, in generate_file_header
    data_sz = os.path.getsize(os.path.join(g_cache_dir, ent['path']))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen genericpath>", line 50, in getsize
FileNotFoundError: [WinError 3] Системе не удается найти указанный путь: 'E:/Espressif/workspace/test-proj/build/CMakeFiles\\frogfs-cache\\.test-data/state.json'

from frogfs.

Delete your frogfs cache json file in the CMakeFiles directory. Yep. Another bug. Hahaha.

from frogfs.

The whole caching thing, and keeping track of the state was (and still is) very difficult to get right.

from frogfs.

I'm actually did full clean (delete build dir) after changing yaml and before the build.

from frogfs.

The discard verb should now work in the next branch. I've made a number of improvements to mkfrogfs.py. This issue is still unresolved as the cache verb does not work recursively on directories yet. The yaml format changed a bit, the filter glob patterns are a list now instead of a dictionary. You can now have duplicate filters, and they are evaluated first-come first serve. I need to update the documentation still, and I will do this once I fix the cache verb. Please refer to default_config.yaml

from frogfs.

Great! I'll try to find a time to check the changes next week.
About cache keyword. Mmmm. I have mixed feelings about this. From one hand it's interesting to have FS which will provide data about every file and how to handle that file. But, from another hand, maybe it's better to stick with simplicity? It looks like FrogFs is based on (or takes it roots from) EspFs invented by SpriteTm. So, maybe it's better to go this way and just provide a stable and working read-only FS for ESP-IDF and without any "bells and whistles"?

from frogfs.

Oh, the cache is a build time thing. It means that it doesn't watch the mtime of the file. It might need a better verb. It has nothing to do with runtime. Caching is enabled by default. "no cache" means that every time mkfrogfs.py runs, it will preprocess the file and run stage 2 every time. It is useful in debugging and developing transforms.

from frogfs.

Oh, the cache is a build time thing. It means that it doesn't watch the mtime of the file. It might need a better verb. It has nothing to do with runtime. Caching is enabled by default. "no cache" means that every time mkfrogfs.py runs, it will preprocess the file and run stage 2 every time. It is useful in debugging and developing transforms.

Oh, I had wrong understanding of "cache". Looks like it's very useful feature.

from frogfs.

The discard verb should now work in the next branch. I've made a number of improvements to mkfrogfs.py. This issue is still unresolved as the cache verb does not work recursively on directories yet. The yaml format changed a bit, the filter glob patterns are a list now instead of a dictionary. You can now have duplicate filters, and they are evaluated first-come first serve. I need to update the documentation still, and I will do this once I fix the cache verb. Please refer to default_config.yaml

I checked new changes and looks like all is fine.
Noticed that now there is __pycache__ dir created in frogfs/tools directory. Maybe I'm wrong, but I think earlier this directory was created somewhere inside project build directory (but I'm not sure :D )

from frogfs.

With the latest iteration of mkfrogfs, this is irrelevant. You can now build a filesystem from multiple sources, piecing it together. I developed it for a project using Lua, and it works really well.

from frogfs.