Comments (12)
mimalloc
also claims to work with wasm32
. I suppose it's possible to vendor mimalloc
in this tree and have some degree of testing, and there's preliminary work on allowing building without dlmalloc
.
from wasi-libc.
@bjorn3 Unfortunately, IANAL also :) The patents may very well be expired and even dlmalloc be fine. However, from the Fedora PoV, this is still inclusion of new (IOW not previously included in Fedora) code with a problematic license and would need a review and exception from people that actually are lawyers.
Given the age of the code, I would probably get the exception – note however that I would not know about the expiration possibility if I did not open this issue :) Now, with the emmalloc porter, this discussion is probably just academic – I can (and will) use the malloc with non-problematic license.
With that, I'm now considering my issue to be addressed, so feel free to mark it resolved. Thanks for taking the time to do the port!
from wasi-libc.
IANAL, but dlmalloc is more than 20 years old (earliest version I could find is v2.5 from 1993, or in other words 29 year old).
There have been algorithm changes in dlmalloc much later than that.
The issue is a formal matter, we not actually expect any patent enforcement action from this particular upstream. But we don't really to divide upstreams into those who can use slightly out-of-policy licenses, and those who cannot.
from wasi-libc.
The mmap emulation code works by calling malloc, so it unfortunately won't work for implementing a malloc.
As @TerrorJack mentions, there is a build parameter for changing the malloc implementation, though the current options are dlmalloc
or none
.
mimalloc
is a possibility. We could certainly add that to the wasi-libc tree as an option, if someone wanted to port it. It'd also be something we could consider making the default allocator, though we'd want code size and performance comparisions. Also, I am concerned about the fact that mimalloc uses spin-wait loops for some things, though I haven't studied it in depth.
Another option would be to ask Fedora for an exception for dlmalloc. It's used in a lot of places.
from wasi-libc.
Another option is emmalloc. Like dlmalloc it uses sbrk
, and it's used in wasm today in emscripten as a drop-in dlmalloc replacement. It should just work in wasi-libc (as long as you don't #define
the emscripten verbose logging code). License is MIT (compatible with wasi-libc).
emmalloc is, like wee_alloc, more focused on size than speed, so it's not as fast as dlmalloc on heavy malloc benchmarks. But in general usage it works very well in our experience, and it is significantly smaller than dlmalloc, about 1/3 the size (which is often significant in small programs).
As an additional benefit this would be a nice step towards convergence of wasi-libc and emscripten's libc code.
from wasi-libc.
Older versions of dlmalloc
also come with an actual PD dedication.
from wasi-libc.
Another option would be to ask Fedora for an exception for dlmalloc. It's used in a lot of places.
I can certainly do that, but that assumes I asked the project their opinions about changing the implementation and got rejected, told it's a WIP etc. It was one of the reasons for opening this issue ;) However, seeing now that there are options, I would like to explore them more.
Another option is emmalloc. Like dlmalloc it uses sbrk, and it's used in wasm today in emscripten as a drop-in dlmalloc replacement.
As an additional benefit this would be a nice step towards convergence of wasi-libc and emscripten's libc code.
Interesting, and I'll try to import this to my fork and play around with it. No experience with benchmarking mallocs etc., so an actual PR may be a bit off, but let's try it.
Older versions of dlmalloc also come with an actual PD dedication.
Also an option, but I would rather not downgrade… I tried to reach Doug Lea with the problem description in order to see if it can be re-dedicated, but got no answer so far.
from wasi-libc.
emmalloc is, like wee_alloc, more focused on size than speed, so it's not as fast as dlmalloc on heavy malloc benchmarks. But in general usage it works very well in our experience, and it is significantly smaller than dlmalloc, about 1/3 the size (which is often significant in small programs).
wee_alloc has memory fragmentation issues and has a relatively high lower bound on memory usage as I understand it. See for example rustwasm/wee_alloc#85.
Edit: you were suggesting emmalloc, not wee_alloc. My bad.
from wasi-libc.
@bjorn3 That looks like a specific bug in wee_alloc
, yeah.
We're not aware of any significant bugs on emmalloc
atm. It is used in production in both large and small applications, using large and small amounts of memory.
But it is true that emmalloc
(and other small allocators like wee_alloc
) is definitely not as fast as dlmalloc
on malloc-intensive benchmarks - dlmalloc
has a lot of nice optimizations particularly for allocating huge numbers of small objects of odd sizes. Those situations are somewhat rare, though, in real-world code, in my experience, likely since developers generally try to avoid tons of malloc calls.
from wasi-libc.
As an update here, emmalloc is now in wasi-libc (#340). It currently requires a wasi-libc build-time option, MALLOC_IMPL=emmalloc
.
from wasi-libc.
We switched to emmalloc
as the default allocator in zig cc
. The dlmalloc
code is not shipped any more.
from wasi-libc.
@khardix IANAL, but dlmalloc is more than 20 years old (earliest version I could find is v2.5 from 1993, or in other words 29 year old). This means that any patents that could apply to dlmalloc have since expired and any patents filled after that could be invalidated using dlmalloc as prior art, right? Or am I misunderstanding how patent law works? Now there have been releases that are less than 20 years old, but using any 20+ year old release should be fine with respect to patents, right?
from wasi-libc.
Related Issues (20)
- `errno` is broken on `wasm32-wasi-threads` HOT 15
- undefined symbol: alarm (and maybe dup) HOT 2
- Please tag wasi-sdk-21 HOT 2
- is there any plan to support mqueue/rwlock api? HOT 1
- Support sigaction (for LLVM) HOT 1
- how does preview2 deal with wasm64? HOT 2
- locking issues in chdir.c
- O_RESOLVE_BENEATH is forced HOT 4
- both `/` and `.` preopens "normlaize" to empty string meaning it is not possible to preopen root and current dir simultaneously HOT 6
- main in a library regression HOT 2
- Enhancement request: interface to get the wasi-libc version at runtime HOT 2
- Support `mprotect` in wasi-emulated-mman HOT 6
- Consider including a pthread.h for all targets HOT 5
- httpclient module for API request HOT 1
- Dereferencing nullptr in __sec_to_zone function HOT 2
- wasi-libc should probably not define `_POSIX_CPUTIME` and `_POSIX_THREAD_CPUTIME`...
- Command line length limits are easily exceeded on Windows HOT 1
- Keep/separate out a list of symbols that are only stubs
- sysroot for wasm64
- FTS Support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wasi-libc.