Recycler in 'Memory Management for Big Data' about blog HOT 4 OPEN

Eh, it's a bit hard to comment without knowing the folks. My initial inclination was to go vicious, but I got that under control. ;)

Generally, the more you need tools to work around, e.g. the allocator, the better the language. You don't need to work around the allocator in a slow interpreted language, because it isn't the bottleneck. Both jemalloc and the system allocator (what Rust uses) are decent bits of engineering, and it speaks to Rust's performance that you can still get 2x by going around them. If one's program doesn't need that because it does enough other things, no worries, but lean enough computations do benefit.

I'm not sure what the age of the code has to do with anything. I don't think it slows down each year or anything like that. It could probably be spruced up with some more types, but .. it still does today what it did years back.

Not sure if this clears anything up, or just causes trouble by sounding sassy. :)

from blog.

Ha. Sorry, I probably shouldn't cut and paste people's comments without any context... this was in the context of me saying (without having looked) that I thought that the recycler was 'a bit like a cursor'.

Is it right that the performance improvement with recycler is down to the reuse of allocated memory? Is there some subtlety around the allocated memory being owned? or is it just a case of pre-allocated is faster that allocating fresh? (sorry if that's a silly question, I am new to Rust).

[edit] it is a stupid question isn't it? it can be reused because its owned right? I'll get my coat.

from blog.

It is just about the re-use of memory, yes. And Rust is picky about the memory being owned, and being typed. In this case it was interesting to me that you can do this sort of memory pooling based on the structure of the type (pooling allocations deeper in the type signature, rather than just String or Vec<usize>).

It is also a case that de-allocation by stashing is faster than using the allocator. You'll often find that allocating is relatively cheap (most operating systems do "demand paging", in which they only do work as you start to use the memory, but when you hand them back lots of memory they seem to want to freak out and do a bunch of work (jemalloc, at least)).

from blog.

Ok that makes lots of sense. Thank you!

from blog.