Comments (4)
Thanks @sgorozco, this is in fact whats happening here, a BinaryWriter writes a lot of little chunks (e.g. int, byte, etc) for a OPC UA binary encoder. The Write implementation should only copy the small chunks into the buffer, keep two indexes (one for the actual buffer, one for the write pointer into the actual buffer). An overflow may split the write and allocates a new buffer or moves to the next buffer. Thats it. I would be interested to see why a division is necessary.
Update -- I checked:
--> GetBlockAndRelativeOffset uses a division.
I guess its a design decision to make write faster or the calculation of the position, or what to use as the source of truth.
My expectation were that the Write is almost as fast as the Write in the MemoryStream base class. Wishful thinking :-)
from microsoft.io.recyclablememorystream.
Yeah, the issue with lots of small writes is something that has come up a few times before. Unfortunately, it just wasn't optimized for that scenario. The workaround would be to operate directly on the buffer, but not all APIs will support that, and that does defeat the purpose of using a stream.
A way to fix this would be to change the source of truth for the current position
. Currently, we store a long position
, that we then convert to block/offset on every necessary operation, which involves, as you noted, two divisions. We could change this to track the block/offset instead, and then when we need the position
, do the multiply+add, which should be much faster. Alternatively, we could just track both values and keep them in sync, but I suspect this doesn't offer any additional benefit (we'd be doing the multiply+add anyway, or a lot of individual adds).
I'd have to think about any negative performance ramifications (e.g., calling Position is much more expensive), but perhaps it's not so bad. I'll ponder it.
from microsoft.io.recyclablememorystream.
from microsoft.io.recyclablememorystream.
As an additional sidenote, turns out that in my JSONEncoder Tests where writing uses StreamWriter in contrary to BinaryWriter above, RecyclableMemoryStream outperforms all other stream implementations... this is currently our main use case so technically I'm a happy camper with the current implementation.
from microsoft.io.recyclablememorystream.
Related Issues (20)
- Using memory stream with httpClient post HOT 3
- Is there a plan to contribute to the dotnet/runtime? HOT 4
- FeatureRequest: Add opt-in possibility to zeroed-out buffers HOT 10
- Continous benchmarking HOT 1
- Multiple StreamManagers in referenced libraries - how to find total memory usage HOT 1
- Large pool limit should apply to whole pool, not each slot HOT 1
- Humanitarian Organization Team mbrgi
- ResponseTime is almost doubled with RecyclableMemoryStream. HOT 5
- Question: GetStream copy existing buffer use case HOT 1
- How effective is this package for buffers that are usually smaller than 256 bytes? HOT 1
- Consider using ArrayPool<byte> as underlying storage mechanism HOT 4
- Breaking changes at 3.0.0 version break OfficeOpenXml NuGet package HOT 11
- Guideline about cryptography with RMS HOT 5
- Unclear Documentation for MaximumFreeSmallPoolBytes
- Upgrade blocker: Recent change of GetStream() to return RecyclableMemoryStream instead of MemoryStream HOT 1
- why just pool MemoryStream HOT 2
- FileStreamResult: ObjectDisposedException: Cannot access a disposed object. HOT 4
- ETW event source is not working HOT 2
- Use of RecyclableMemoryStreamManager with Confluent.Kafka HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from microsoft.io.recyclablememorystream.