Comments (7)
No, it is currently not possible.
Main reason being that each message may be treated differently by extensions and listeners that may be user-specific code.
Also, from your logs above, you received 3 messages, but only 2 were "application" messages, so CometD still need to do some processing.
However, at the very bottom, CometD does a for
loop and calls your code, which does not seem much different from you doing a for
loop and calling your code.
Have you evidence that the current model is inefficient, and by how much?
from cometd.
The log example was shorted by me and was just to help convey that long polling receives an entire Batch of messages and that's what I want to receive.
There is a very specific performance issue that I'm dealing with which is the reason for my questions.
The performance is not necessarily in cometd code but because of latency in downstream processes it causes issues with the cometd connections.
We have issues completing our downstream process for larger batches within the 40 seconds required before the Server drops by subscription. This causes the 403 connect errors to be raised . And, my APP has issues recovering from the 403 error and needs to be Killed and Restarted. [was trying to resolve for months without success]
I was hoping I could configure client to tell the Server to limit the batch size of messages. But, as I understand it I have no control from client side to control the BATCH size of the messages sent Server to Client. (I saw a separate thread of yours where I think you mentioned needing add Extensions on client and server in order to do it)
That is how I came up with the Question. Since Server is returning 1 response with multiple messages then I might be nice option to allow it to be received at Client as 1 response with multiple messages. Yes, I'm eventually going to parse it into individual events but I could be in control of where that happens.
Thanks for responding so quickly.
from cometd.
The performance is not necessarily in cometd code but because of latency in downstream processes it causes issues with the cometd connections.
Can't you just dispatch the processing to another thread (or otherwise asynchronously), so that the CometD listener returns immediately so that the CometD thread can continue the processing?
Also, you can just queue up on the client batches of N messages, with a timer to drain the queue for processing.
This also will free up the CometD thread to continue the processing.
Error 403
is typically related to protocol or security errors, you should not be getting it for lack of processing.
What CometD version are you using?
from cometd.
in my opinion the Async method is susceptible to receiving duplicate events or even missing events when my client has to reconnect from a Last ReplayId. I can't have 2 threads working on different batches because I have to maintain 1 ReplayId as last committed.
I'm not sure how to implement that Timer to drain queue. Are you trying to say I should try to set some other Timer that is less then the 40 seconds.
The 403 error occurs because the Server as dropped my Client's subscription when 40 seconds elapsed. Then after I finally complete the batch that I'm working on I'll encounter the {advice={reconnect=handshake, interval=0}, channel=/meta/connect, id=3140, error=403::Unknown client, successful=false}
from cometd.
in my opinion the Async method is susceptible to receiving duplicate events or even missing events when my client has to reconnect from a Last ReplayId. I can't have 2 threads working on different batches because I have to maintain 1 ReplayId as last committed.
This seems all logic outside CometD, which has no ReplayId, whatever that is.
Not sure what you mean by "receiving duplicate events", CometD does not do that, unless the application does that.
I'm not sure how to implement that Timer to drain queue. Are you trying to say I should try to set some other Timer that is less then the 40 seconds.
I'm saying that if you receive M messages over the network in a single /meta/connect
response, you can queue them (one by one) elsewhere, and then have a timer that every N seconds drains the queue.
Now you can process your messages in order, as a batch, in a different thread, without compromising the CometD processing.
403::Unknown client
is not an error reported by this project.
Is this Salesforce?
Otherwise, you still have not specified what CometD version.
from cometd.
Thanks again for engaging in conversation.
I am talking to a Salesforce Server. And, my client side code is using cometd version : 4.0.9
I guess 'ReplayId' must be unique to Salesforce implementation. But, every message is uniquely identified and used as a placeholder for where to begin the Subscription whenever starting , reconnecting, or resubscribing to a topic(channel)
I'm trying to queue them one by one elsewhere but that downstream process can sometimes take longer than this 40 second limit for the batch N messages received. And, I'm not able to configure anything to tell the server to send smaller Batch.
Now I find myself trying to queue them on another queueing system (like kafka or mq) which may be more responsive and try to beat the 40 second timer.
implementation 'org.cometd.java:bayeux-api:4.0.9'
implementation 'org.cometd.java:cometd-java-client:4.0.9'
implementation 'org.cometd.java:cometd-java-common:4.0.9'
from cometd.
I'm trying to queue them one by one elsewhere but that downstream process can sometimes take longer than this 40 second limit for the batch N messages received.
It should not matter. You have the CometD thread that just does the queueing, so it is super-quick and you can continue to receive messages.
On a different thread, you dequeue and it does not matter how long the processing takes, because now you're decoupled from CometD. The only problem could be that you receive messages faster than you can process, and you'll have a memory problem. That is outside of CometD too.
CometD 4.0.x is End-of-Life since a long while now, and CometD 5.0.x is going to be End of Community Support very soon.
from cometd.
Related Issues (20)
- NullPointerException in AbstractClientSession.getReleasableChannel HOT 5
- Clarification : Controlling the Inbound 'Data' messages to bayeux client HOT 1
- Orphaned sessions should not enqueue messages
- CometD client (BayeuxClient) state HOT 8
- Duplicate Seti messages when observing /**
- java.util.concurrent.TimeoutException: Network delay expired: 30000 HOT 3
- Consolidate notification code
- OORT long-polling 402::session_unknown HOT 3
- Monitoring cometd client HOT 1
- Verify whether jetty-io is a necessary dependency
- Make JettyWebSocketTransport non-blocking HOT 5
- Make sweep parallelizable
- How to handle Exception from WebsocketTransport in cometD5? HOT 2
- Typescript definition issue for Extension interface HOT 2
- [Client Side JS] Scheduler not working randomly when cometd object is reconfigured at runtime. HOT 12
- stuck tomcat http threads HOT 18
- How to obtain a reference to the BayeuxServer in the SpringBoot CometdApplication HOT 11
- Received message are still being processed after disconnect HOT 9
- When is JakartaEE 10 compatible version release planned for cometd ? HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cometd.