dennis-van-gils / python-dvg-ringbuffer Goto Github PK

Hi @Dennis-van-Gils.

Amazing ring buffer !
I am studying your code and don't understand why do you have a special treatment for np.float64.

It is only in the __init__ and clear methods.

I would guess you are doing that to optimize your array creation (and reset) with np.nan which should be faster than setting the whole array to zeros.

But I failed to understand why it is only for dtype=np.float64.

Could you explain please ?

Hi again @Dennis-van-Gils .

I fully studied your code and it inspired me a lot. Thank you.

I rewrote the ring buffer.
The key idea I use is to use only one numpy array buffer, but of size 2 N and write input in two places, (in append it's at [self._idx_R] and [(self._idx_R + self._N) % self._2N]) and to fix the pointers like you do too (but a bit differently).

I also added the mixin NDArrayOperatorsMixin to have the extra operator special methods (link)

I tested my implementation manually and with your tests suit (some tests has to be change since I change some error classes and the representation) and it seems to be correct.

I added 2 new methods: pop_n(self, n: int) -> np.ndarray and popleft_n(self, n: int) -> np.ndarray which let you pop n items and return an array.

I tested the performance with your timeit test that I modified and those are the relative result on my machine:

Buffer Type	Speed Factor with capacity = 20500	Speed Factor with capacity = 80000	Comment
deque	1	1	baseline, always 1
numpy_ringbuffer	~19.7	~46.6	from Eric Wieser
dvg_ringbuffer	~25.3	~70.6	from Dennis-van-Gils
gs_ringbuffer	~29.7	~120.9	from Gil Shoshan

I modified your test to test also when the buffer is not fully filled.

My implementation is faster than your from what I tested.

I also tested with a very big capacity at 800000 and my implementation seems to time pretty constant while your get slower,
at this big capacity, my buffer was ~34 times faster than yours.

At small capacity (I tested 80), my implementation was a bit slower, yours was 1.1 times faster.

I might have some bugs (I hope not) so be sure to test it yourself if you want to use it.

Here the my implementation and the performance test files:
ringbuffer_v2.zip

You will find my implementation in the ringbuffer_v2.py file.

Hope it can be useful to you and thank you again for sharing your code.
I will take a look at https://github.com/Dennis-van-Gils/python-dvg-dump/blob/master/src/dvg_fftw_welchpowerspectrum.py as you suggested me.

Have a nice day.