use with libesphttpd about frogfs HOT 13 CLOSED

@phatpaul which external git module are you proposing to use? Or are you saying you are using libesphttpd/espfs in your project and you are wondering if that folder should be broken out into its own repository?

from frogfs.

@phatpaul this repo is designed to be a IDF component, not a bare submodule. Might be able to make it serve both roles though. I haven't tested it with chmorgan's libesphttpd fork, I think it would work though.

from frogfs.

Why don’t we ask the user to use this component rather than including it as a sub module in libesphttpd? That’s what other components have done, when there are separate repositories available.

from frogfs.

@chmorgan I think I understand what you're saying.
So we would remove the ESPFS stuff from libesphttpd repo.
The user's project (i.e. the example project esphttpd-freertos) would include a espfs submodule which libesphttpd could use.
But we would need to resolve the dependency of libesphttpd on espfs in the case that the user didn't include espfs module.

Like this?
/main/ <-- user's code
/components/libesphttpd/ <-- git submodule
/components/espfs/ <-- git submodule

from frogfs.

Yep, like David antliffs one wire ds18b20 component requires his other lower level component.

https://github.com/DavidAntliff/esp32-ds18b20/blob/master/README.md

from frogfs.

We would mention needing the other component in the readme. We should be able to add a dependency on that module in the components file right? Like any other required component?

from frogfs.

OK thanks.
I will play around with this arrangement since I'm needing to share espfs between httpd and another (bluetooth) module in my project. And I want to access espfs via VFS in my bluetooth module.

I assume we want to continue the raw espfs api access (not via VFS) in libesphttpd. So those need to be exposed in the submodule.

I don't want to break ESP8266 support, but I don't have any experience with that platform. What should I watch out for?

from frogfs.

@phatpaul what would be the tradeoff between raw vs. vfs?

On ESP8266 support I think there is most development effort around esp32 these days, I think we would want help from esp8266 users to fix that target. It would be nice not to break it but we shouldn't hold up development on esp32 features to avoid that.

from frogfs.

It looks like VFS just adds a layer of abstraction. It adds the code here: https://github.com/jkent/esp32-espfs/blob/master/esp_espfs.c - some indirection and an info struct in RAM.

I don't think it would benefit libesphttpd to remove the raw espfs access, except maybe cleaning-up/consolidating the code.

Anyway libesphttpd already has a VFS cgi example (in addition to espfs), so it's up to the user to choose how to access espfs (directly or via vfs).

from frogfs.

So turns out there is a good reason to keep the raw espfs access for httpd. It provides the gzip flag which tells httpd to send a gzip header to the client.

I just tried using a VFS to espfs connector for httpd and I'm just seeing garbled text instead of index.html. I think it's because VFS doesn't know that index.html has been gzip compressed, so no header is sent via httpd.
https://github.com/chmorgan/libesphttpd/blob/master/core/httpdespfs.c#L135

So if using VFS then logic in httpd is looking for a file with .gz extension. https://github.com/chmorgan/libesphttpd/blob/master/util/esp32_httpd_vfs.c#L90

I guess mkespfsimage should have an option to append .gz to filename??

from frogfs.

I checked for the gzip magic bytes in my fork of libesphttpd:
https://github.com/jkent/esp32-esphttpd/blob/master/util/esp32_httpd_vfs.c#L93

Perhaps this should be done along with a file extension check? I think urls with .gz are kinda ugly.

from frogfs.

@jkent, @phatpaul do Nginx and Apache use the magic bytes? That could be helpful info.

Imo we could easily use magic bytes. @jkent is that something you could create a PR for to merge back into the main repo?

from frogfs.

@chmorgan with mime magic I think apache does, but it probably checks if its text or binary first.

Yeah, I think extension + magic bytes would the best way to go. mkespfsimage only gzip compresses if the file extension is in "html,css,js,svg" by default. I'll work on this now.

from frogfs.