Comments (14)
To be clear, I really like Netty, but ReactiveSocket should not require someone using Netty. There are plenty of other options out there. For example, if using Aeron as the transport, someone shouldn't have to bring along Netty for the ride just for the Buffer type. And if someone is using one of the many alternatives to Netty for TCP, including just standard JDK libraries, again, they shouldn't have to bring along Netty.
Even Agrona is a dependency I'd prefer not having as I strongly prefer zero-dependency libraries whenever possible.
from rsocket-java.
@stevegury Yes we need a ByteBuffer
class to represent data and metadata Payload
s and this issue couldn't have been better timed when I am fighting with potential bugs caused by using 3 different buffer types 😞
@yschimke you are not the only one who have issues with java.nio.ByteBuffer
API 😉 and thanks for asking the question! Apart from the annoying flip()
, rewind()
methods and the lack of writerIndex
there are complexities which aren't required from our usage point of view, eg: differentiation between direct and on-heap buffers.
I think the primary benefit of using java.nio.ByteBuffer
is that it is a part of the JDK and hence a standard (good or bad is debatable). So, the question is whether a standard here is helping us or hurting us as we know that for internal and many external code, we have to downcast to native buffer APIs like agrona. Whatever, it is, it certainly isn't great that we have 3 buffer APIs to deal with, each having subtle differences that cause hard to debug bugs (you pointed to the capacity()
and limit()
, remaining
differences between agrona and JDK buffer. From netty's buffer point of view, at some places we were using .nioBuffer()
which is a view of the underlying buffer (pooled in some cases) and hence can change over time.)
I would like to propose that we stop using JDK buffer API and define a restricted, usable buffer API for ReactiveSocket
This may sound extreme to some but it will provide the following benefits:
reactivesocket-core
doesn't have to deal with two buffer abstractions (JDK and agrona) as it does today.- Any buffer related optimization can reside in respective transport implementation. Aeron implementation can have its optimizations with agrona buffer and any implementation directly using that implementation will be aware of the optimization details and side-effect contracts. Netty can have its optimization related to pooling this new API buffer instance, instead of netty's buffers.
- There is a clear boundary between external buffer libraries and
reactivesocket
buffers.
Just to be clear, I don't believe we ever need to provide any implementation of our buffer API as we always read/write what is provided by the user/transport
I think there may be cases where we would trade some performance for clarity and maintainability of code but I think we should have strong data points that it indeed is the case. In absence of which, I would like to prefer clarity over micro-optimizations (assuming individual transports have done the optimizations required)
from rsocket-java.
I'll start on commons-buffers using runtime classloading tricks to choose an implementation at runtime :)
I kid... I kid...
from rsocket-java.
IIRC the reason why we stick to java.nio.ByteBuffer
has to do with the fact that we expose/leak the type via the Payload
interface.
@tmontgomery @benjchristensen or @NiteshKant can you confirm?
from rsocket-java.
This is pretty much what Finagle did https://github.com/twitter/util/blob/develop/util-core/src/main/scala/com/twitter/io/Buf.scala
Although personally, I'd be interested if we could survive with just Netty's buffers and the handover to agrona still be efficient.
from rsocket-java.
@yschimke we can chose netty's buffer as the library for ReactiveSocket but I think it is too broad an API for our use and has ref-counting as a core part of the API, which I am unsure of whether we need it or not.
from rsocket-java.
I'm not a fan of locking into to Netty as a dependency.
from rsocket-java.
@benjchristensen so looks like you are inclined towards defining our own buffer abstraction or you prefer using JDK buffers?
from rsocket-java.
inclined towards defining our own buffer abstraction or you prefer using JDK buffers
Whichever works better. Our own buffer abstraction as an interface may work best as it avoids the concrete implementation of JDK.
Can we have the interface behave similar to DuplexConnection where we interact with just the interface, and then the binding implementation is provided by whatever provides DuplexConnection?
In other words, a ReactiveSocket can't be created without a concrete DuplexConnection, and without a concrete Buffer factory?
from rsocket-java.
Can we have the interface behave similar to DuplexConnection where we interact with just the interface, and then the binding implementation is provided by whatever provides DuplexConnection?
The first part is what my thinking was that we don't have to instantiate a buffer ourselves which removes the necessity of providing a factory. Although, users would have to provide Payload
instances which would require a Buffer
implementation. We can provide buffer instances created using a byte[]
if a user does not have the buffer from a transport. This may just be "good enough".
from rsocket-java.
I'll start on commons-buffers using runtime classloading tricks to choose an implementation at runtime
@yschimke I think that is a good idea (#maniacalLaughter)
from rsocket-java.
@yschimke
I'd prefer to just to use the nio ByteBuffer only if we want one library. It's standard and everyone can use it. We don't have to write anything to make it work with other libraries - it just does. The api isn't the greatest but it's standard so people know how to use it. The ReactiveSocket code is already based on ByteBuffer internally. Everyone supports it so it's not opinionated.
from rsocket-java.
Also - we already chose ByteBuffer
#6
from rsocket-java.
This seems pretty good now
Agrona byte buffer usage seems limited to
- internal packages
- aeron transports
- a few factory methods e.g. Frame.from
from rsocket-java.
Related Issues (20)
- Ability to store session object inside RSocketRequester chain HOT 1
- About JWT authentication and authorization HOT 1
- How to set SETUP setupRoute
- SendUtils onDiscard handler class cast exception HOT 5
- https://maven.pkg.github.com/rsocket/rsocket-java (needs permission to access) HOT 1
- Tcp Example Server and Client in Seperate files doesn't work HOT 5
- NoSuchFieldError: rsocket-core overrides dependency for netty-buffer and therefore clashes with reactor-bom / reactor-netty HOT 2
- ClosedChannelException making RSocket request with invalid credentials HOT 1
- Bump reactor-bom to 2020.0.24 to fix CVE in reactor-netty HOT 2
- 1.1.4 Regression - RejectedSetupException on auth failure results in ClosedChannelException HOT 3
- Releasing memory in ZERO_COPY mode.
- Ongoing flux subscription is not getting cancelled even if the client is closed
- LoadbalanceRSocketClient lacks reconnection and retry functions HOT 1
- Send a message to an already established request-stream connection from the rsocket-server to all signed rscoket-clients except one
- How to keep RSocket connection alive forever. What should be configuration values for keepalive(interval, maxLifeTime) to achieve this? HOT 1
- Update "Supported Core Protocol Features Matrix" to add that Resumption is supported
- Server does not close and re-connect on no keep alive ack issue.
- Rsocket error "java.lang.IllegalStateException: Source has to be ASYNC fuseable" with Spring boot 3.2.x & JDK17 HOT 2
- RSocketRequesterTracingObservationHandler producing netty buffer LEAKs
- throw exception on errorConsumer, client will not finish
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rsocket-java.