Docs do not build on docs.rs about esp-idf-sys HOT 9 CLOSED

So I'm not very knowledgable about cross-compiling or embedded Rust, but I can try to take a crack at this. I do think it's worthwhile to investigate this. Even though the textual documentation on this crate and esp-idf-svc are minimal, I constantly use rustdoc generated docs to see how different types slot together. It's probably also easier to justify writing good docs if we know that those docs can actually be read by someone.

Documentation on docs.rs confirms that networking is not available for documentation builds (https://docs.rs/about/builds#hitting-resource-limits). I think to get around this, we would need to vendor the esp-idf C library as a git submodule or something.

Another alternative is to build the docs ourselves and publish them to github pages linked in the readme. This is less work today, but it does require more effort each time a new version is published.

I am able to build the docs locally with the esp compiler fork and the appropriate target, FWIW. @ivmarkov do you have any thoughts on which method we should take to resolving?

from esp-idf-sys.

It is not just the ESP-IDF C library, right, it is also the whole toolchain that is necessary to compile the thing (as in the GCC toolchain for that target).

I'm thinking of an alternative: perhaps I can implement a build option (using a feature or something - but really have to see how much flexibility is there in the docs.rs build system) - where instead of compiling the ESP-IDF toolchain, we are actually using a pre-generated bindings.rs file. Since none of the crates in question (esp-idf-svc, esp-idf-hal, and most importantly - esp-idf-sys - from where the trouble comes) really link against the C library (they just use the autogenerated unsafe APIs from the bindings.rs file to build higher level abstractions; (OK esp-idf-sys is also emitting link instructions for the binary crate if there is one around - but that's irrelevant for documentation)) - that approach might work and circumvent the whole vendoring thing. Now, how exactly this pre-generated bindings.rs file will be generated and maintained up to date - is an interesting question by itself. Perhaps a GitHub action would do, as these are capable of doing networking.

from esp-idf-sys.

One problem is also that the generated bindings depend on the generated sdkconfig.h by the esp-idf and on the selected mcu. So it isn't really clear which bindings should be displayed on docs.rs.

Maybe one approach would be to make the bindings for every mcu available in different modules, and just set sane defaults for the sdkconfig. Or, like @lily-mara already mentioned, host the docs ourselves, but separate docs for every mcu.

Also, that's the reason we can't/shouldn't vendor the bindings in the repository, again because they only correspond to one mcu and sdkconfig.

from esp-idf-sys.

Maybe one approach would be to make the bindings for every mcu available in different modules, and just set sane defaults for the sdkconfig. Or, like @lily-mara already mentioned, host the docs ourselves, but separate docs for every mcu.

Having bindings per-MCU is not really solving the problem completely but might be part of the solution. While indeed there are differences from MCU to MCU (pin number and layout as well as number of SPI devices in esp-idf-hal immediately come to mind), the problem is exacerbated by the gazillion of config options in the kconfig menu system of the ESP-IDF which are mostly orthogonal to the MCU selection and lead to a combinatorial explosion of options against which you can compile the crates. In other words - if you don't explicitly enable the BT via menuconfig, you don't get unsafe BT bindings in bindings.rs and - by extension - you don't get safe abstraction on top of these in esp-idf-svc (not that we have safe abstractions for BT yet). Ditto for HTTPS, NAPT, and the list will keep growing.

So what we probably need is an auto-generated bindings.rs with all config options enabled (somehow), which is - optionally - per MCU.

Also, that's the reason we can't/shouldn't vendor the bindings in the repository, again because they only correspond to one mcu and sdkconfig.

Indeed. The bindings.rs file used to be generated manually in the past and committed to the repo, but that was only usable for toy projects and just for one MCU.

Yet, (auto-)generating bindings.rs (per MCU?) with all kconfig options enabled, committing it to a special place in the repo and then using it only when compiling the crates in a "please generate docs" mode (if that's even possible) seems like the most plausible solution to me. With the benefit that we get the docs in crates.io, where they belong.

from esp-idf-sys.

Yet, (auto-)generating bindings.rs (per MCU?) with all kconfig options enabled, committing it to a special place in the repo and then using it only when compiling the crates in a "please generate docs" mode (if that's even possible) seems like the most plausible solution to me. With the benefit that we get the docs in crates.io, where they belong.

I don't even think we need to commit these bindings to the repository, instead, when publishing the crate we have to just publish the vendored bindings with it (and maybe automate this with a github action). But maybe we want to commit them so that someone that clones the repository is able to build the documentation without having LLVM installed (and without building the whole esp-idf)?

At any case, I think the way forward with the docs.rs approach would be to add modules for the bindings of every mcu, and then export the bindings of esp32c3 (ex. pub use bindings_esp32c3::*) as that's the only one supported with upstream rust atm. But only do this in the special case when building documentation¹. In the default case, it should use the locally generated bindings as we do currently (with include!(...)).

For this, we also need to provide sdkconfigs for every mcu that enable all settings (or something to that effect) which are used to generate the bindings.

Edit to clarify:
¹ We still would want locally generated documentation to be specific to the mcu and sdkconfig used. So when building documentation locally it should default to generating the bindings.

from esp-idf-sys.

I don't even think we need to commit these bindings to the repository, instead, when publishing the crate we have to just publish the vendored bindings with it (and maybe automate this with a github action). But maybe we want to commit them so that someone that clones the repository is able to build the documentation without having LLVM installed (and without building the whole esp-idf)?

Yes, that reason and also if someone wants to browse the esp-idf-sys/../bindings.rs source code in GitHub.
Not strictly necessary, nice to have maybe.

At any case, I think the way forward with the docs.rs approach would be to add modules for the bindings of every mcu, and then export the bindings of esp32c3 (ex. pub use bindings_esp32c3::*) as that's the only one supported with upstream rust atm. But only do this in the special case when building documentation. In the default case, it should use the locally generated bindings as we do currently (with include!(...)).

For this, we also need to provide sdkconfigs for every mcu that enable all settings (or something to that effect) which are used to generate the bindings.

Agreed on all points.

from esp-idf-sys.

Unfortunately I don't think I know enough about embuild or cross-compiling to create the per-chip setup that you're talking about. I think I'd need to punt this to someone else.

from esp-idf-sys.

Can this be closed now? Whilst it still isn't being built on docs.rs, we have links from docs.rs & crates.io to our hosted documentation here.

from esp-idf-sys.

Yes, thanks for adding the self-hosted docs!

from esp-idf-sys.