(Migrated from espressif/rust repo)
@igrr Patches in https://github.com/ivmarkov/rust/commits/stable provide an std-capable environment on top of ESP-IDF.
Some of these patches work around the fact that ESP-IDF is not completely POSIX-compatible.
Here is a list of things that can be fixed in ESP-IDF to reduce the number of changes we need to make in rust:
Easy changes
- add posix_memalign implementation — should be easy, ESP-IDF already has
memalign function implemented
- readv, writev — simple version can be wrappers around read/write
- BSD Socket APIs: accept, bind, shutdown, getpeername, getsockname, getsockopt, setsockopt, closesocket, connect, listen, recvmsg, recv, recvfrom, sendmsg, send, sendto, socket, ioctlsocket, gethostbyname, gethostbyname_r, getaddrinfo, freeaddrinfo. These functions exist in IDF, but only as static inline functions. They need to be exported functions to be found by the linker.
It can be done pretty easily, adding a C file with the following pattern:
// defines an alias symbol:
int accept_impl(int socket, struct sockaddr *restrict address, socklen_t *restrict address_len) \
__attribute__((alias("accept"));
// calls the "static inline function"
int accept_impl(int socket, struct sockaddr *restrict address, socklen_t *restrict address_len) {
return accept(socket, address, address_len);
}
- Define getcwd. Can simply return "/".
- Define chdir. Can return -1 and set errno to ENOSYS.
- Define
sysconf and _SC_PAGESIZE. Can be a very simple implementation, returning constant value.
- Process-related functions (
kill, signal, exec, wait, spawn, etc) — no meaningful implementation possible, but each can return -1 and set errno to ENOSYS.
nanosleep — can be a wrapper around usleep
More complex changes
- duplication of file descriptors — needs support in
vfs component, not sure how complex this would be, and the semantics required by libstd.
PTHREAD_COND_INITIALIZER definition in IDF is non-standard:
// The newlib definitions in libc for pthread_mutex_t and PTHREAD_MUTEX_INITIALIZER
// are way off compared to what we have in ESP-IDF:
// 1) ESP-IDF's pthread_mutex_t is only 4 bytes and not 40+ (because it is in fact a pointer to dynamically allocated structure)
// 2) While the above is just a waste of space, ESP-IDF PTHREAD_MUTEX_INITIALIZER is the real issue:
// in libc it is defined as a series of 0s, while in ESP-IDF it is equal to 0xffffffff
No changes
These changes in the current patch set do not seem to be necessary, need to find why they were needed.
getrandom — it should be already supported by ESP-IDF.
igrr:
@ivmarkov here is the summary after looking at the std-related patches. Most things can be solved in IDF pretty easily.
There are probably other things in the patchset that i haven't noticed yet, please feel free to add more items.
ivmarkov:
Hi @igrr,
Thanks for all the feedback so far.
Before commenting on details, let me state something generic, which is very important:
Using an API (function) implemented in ESP-IDF from Rust STD is really a two-step process:
- The function should (obviously) be available in ESP-IDF
- The function should be declared by the libc crate
The reason why we need (2) is because currently the whole Rust STD support is based around consuming "extern C" API definitions available in the libc crate from above, which are manually defined and maintained for the various libC libraries (msvcrt, glibc, newlib, musl etc.).
Currently, Rust STD does not know about our esp-idf-sys crate and therefore about all the Bindgen bindings generated by it. Letting STD know about esp-idf-sys might increase the STD patch size and its complexity, and decrease the chances of it being accepted so far.
Therefore, I think we should stick to consuming the libc crate instead of our own autogenerated bindings in the context of Rust STD (files, threads and networking/sockets).
That would imply that we might need to file patches against libc (the newlib sub-module specifically), so that it properly reflects the status quo on ESP-IDF.
That work has already started. For example, @reitermarkus, who started the STD-on-ESP32 effort originally was able to push these changes to the libc crate, which were key for having the initial ESP32 STD support for Rust.
I think we should just continue this for riscv, as well as for all "deviations" ESP-IDF has from the standard newlib - particularly in the pthread area.
Of course not everything needs to be in the libc crate so as to be usable from Rust STD. You can always put an "inline" "extern C" definition, as I did e.g. here. It is just that these should be a minority, I feel.
One more thing: I'm not implying that there will be a huge amount of changes to the libc crate. Just that there will be some, and whenever e.g. there is an API in ESP-IDF (like posix_memalign) that does not necessarily means that it is exposed in libc/newlib.
As for the types of changes, see the riscv support here. Currently, a bit of a hack as it delegates to the "xtensa" support, even though I don't expect these to be different.
igrr:
- Regarding your assessment of the "easy" changes: I agree completely. Before starting with those, I'm wondering, perhaps we should approach the Rust libstd maintainers with the current patchset, to get early feedback / directions. Shall I do this, or will Espressif drive it?
- Regarding the "more complex ones":
- FDs duplication, sematics: aren't they just expecting the POSIX semantics, whatever it is?
- pthread*_t sizes being incorrect, and the PTHREAD_MUTEX_INITIALIZER of ESP-IDF not having the value defined in the libc crate for newlib - that might be something we have to push for fixing in the libc crate itself
igrr:
@ivmarkov Thanks for the explanation, I didn't know about newlib-specific things present in libc create, it makes a lot more sense now. I would appreciate if you could start the discussion with libstd maintainers regarding this existing patch set.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent