Listeners are running at lower concurrency than the configured concurrency about rqueue HOT 10 CLOSED

Hi Omar
Thanks for reporting this issue, 2 is the default concurrency, can you please try latest Rqueue version 2.7.0 and report this here if issue persist.

Also, you have set concurrency to 10-20, so it should scale this to 20 workers.

Also, you can add a debugger here to check why this is not polling upto 20 messages

The semaphore permits should be 20 and it should increase/decrease based on the listener message consumption pattern.

I just tried for one queue, with same concurrency for testing on latest version 2.7.0 see the screen recorder here.

screen-capture.webm.zip

from rqueue.

Hi Sonus,
I already use V 2.7.0 .I will try to debug the app from the point you mentioned and get back to you regarding it.
for now here is a copy of my pom.xml dependency

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-web</artifactId>
</dependency>

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-test</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>



<dependency>
	<groupId>com.oracle.database.jdbc</groupId>
	<artifactId>ojdbc8</artifactId>
	<scope>runtime</scope>
</dependency>

<dependency>
	<groupId>com.h2database</groupId>
	<artifactId>h2</artifactId>
	<scope>runtime</scope>
</dependency>
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

<dependency>
	<groupId>org.projectlombok</groupId>
	<artifactId>lombok</artifactId>
	<optional>true</optional>
</dependency>
<dependency>
	<groupId>com.github.sonus21</groupId>
	<artifactId>rqueue-spring-boot-starter</artifactId>
	<version>2.7.0-RELEASE</version>
</dependency>

and here is the output of java -showversion command

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

from rqueue.

from rqueue.

Hi Sonus,
I tried to remove the bounding and set it to 20 without a lower bound .
I still have only two working consumers (consumer 1 & 2) .
I tried to debug the semaphore state and queueThreads size. the state was set to 20 and the queueThreadSize is always 0 or 1 or 2 . I tried to trace it back to why this thing happens but wasn't able to pinout the problem.

in one case while debugging there was 20 running job but they were more like idle running jobs and the queue only proceeded 2 jobs per time. I wasn't able to reproduce this case to further debug it.
here is my project if you wish to check it out.
debuq-rqueue.zip

from rqueue.

Hi Omar
If I understand correctly, you observed Rqueue is processing only 2 jobs once and you can't replicate this issue as now it's processing 20 jobs at a time?

On my machine, its processing ~20 jobs always and I have enqueued 1 million jobs.

Btw what do you mean by this line?

in one case while debugging there was 20 running job but they were more like idle running jobs and the queue only proceeded 2 jobs per time.

from rqueue.

Anyway, I've a theory that could potentially explain why this happens, your job is taking sub milli seconds to process(since it does nothing, if you add some work like sleep or some http call that takes time you can see the effect), around similar time is taken by message poller to poll the message.

So what's happening now is,

the time message poller takes to poll a message ~ the time your listener takes to execute a message

So there's no advantage of increasing the listener count. Here we need to increase message poller count that's currently not supported.

from rqueue.

Hi @OmarBish
Would you like to give a try to a new release candidate? You can also deploy to local maven to see if things are working as expected, branch https://github.com/sonus21/rqueue/tree/pause-unpause

And I'm removing Bug label from this as this does not seem to be a bug. I did some experiments to see how many jobs can be processed by Rqueue in a second, it seems it can process 6-8K jobs using 20 threads and 2 milliseconds as execution time.

You can also run this test on your machine to see what happens

https://github.com/sonus21/rqueue/blob/aa4b20ef38cb77fffa3189a2b7f57bac24f89366/rqueue-core/src/test/java/com/github/sonus21/rqueue/listener/ConcurrentListenerTest.java

from rqueue.

Hi @sonus21 ,
I also did some more testing and it turned out as you mentioned
the result of my tests were as follow:

• with thread sleep of 20 ms i have 2 concurrent threads
• with thread sleep of 200 ms i have 5-6 concurrent threads
• with thread sleep of 2000 ms i have the full capacity of the pool (20 thread)

I will run the test that you send and give you the results . also i will test the new release ^-^ .

thank you for your great support.

from rqueue.

Hi Omar
There's a bug in the code, so you need to try the latest code from pause-unpause branch. BatchSize was ignore and it's using default value(10).

Let me know the performance metrics, one thing I've observed on my system is it runs out of connection with local redis, so I get many errors and many jobs goes for retry. Not sure about your setup so I can't comment, how it's able to cope with the demand when your concurrency is high and batchsize is high too.

from rqueue.

One improved version 2.8.0 has been released to fix this issue at some extent.

from rqueue.