Git Product home page Git Product logo

disruptor's Introduction

Conversant ConcurrentQueue, Disruptor BlockingQueue and ConcurrentStack

Disruptor is the highest performing intra-thread transfer mechanism available in Java. Conversant Disruptor is the highest performing implementation of this type of ring buffer because it has almost no overhead and it exploits a particularly simple design.

2017 Conversant Disruptor - Still the World's Fastest

Getting Started

Run the maven build to build and use the package.

$ mvn -U clean package

Conversant Disruptor is on Maven Central

For Java 9 and above:

<dependency>
  <groupId>com.conversantmedia</groupId>
  <artifactId>disruptor</artifactId>
  <version>1.2.16</version>
</dependency>

A classifier is not required in the latest release.

Java 8 is only supported in 1.2.15 and earlier.

Java 7 is only supported in 1.2.10 and earlier.

<dependency>
  <groupId>com.conversantmedia</groupId>
  <artifactId>disruptor</artifactId>
  <version>1.2.10</version>
  <classifier>jdk7</classifier>
</dependency>

Discussion Forum

Conversant Disruptor has a google group so you can follow releases and changes:
https://groups.google.com/forum/#!forum/conversant-disruptor

disruptor's People

Contributors

dependabot[bot] avatar fgdrf avatar jac18281828 avatar press0 avatar sullis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

disruptor's Issues

There are too few issues with this library

Hi, I'd like to complain about the lack of issues with this library. All open source libraries need to have issues, otherwise I can't be sure the adopters aren't robots trying to take over the galaxy. Can you please introduce a bug we can spot and fix?

Infinite loop in `MultithreadConcurrentQueue` constructor

When parameter capacity is Integer.MAX_VALUE in MultithreadConcurrentQueue's constructor,

    public MultithreadConcurrentQueue(final int capacity) {
        int c = 1;
        while(c < capacity) c <<=1;
        size = c;
        mask = size - 1L;
        buffer = (E[])new Object[size];
    }

it goes into infinite loop at line while(c < capacity) c <<=1;.

Because, c will left shift to 2^30, which is < Integer.MAX_VALUE; and then 2^31 - unfortunately, it's -2147483648 in signed integer in Java. And after that, c will be 0 forever, causing an infinite loop.

A simple fix would be: while (c < capacity && c << 1 > 0) c <<= 1;. However, it will break the javadoc in the constructor:

Note: actual capacity will be the next power of two larger than capacity.

make AbstractWaitingCondition public?

Hey, I'd like to leverage AbstractWaitingCondition directly in my codebase. I notice that AbstractSpinningCondition is public, but AbstractWaitingCondition is not.

Looking through the source for AbstractWaitingCondition I don't see any specific reason to not make it public. Can we make it publicly accessible? Happy to open a PR to do this if needed.

If I just want the wait/signal functionality, are there better alternatives? I searched a bit and couldn't find any literature online about other condition implementations that make use of progressive yield.

If it's not public by design, I understand and the issue may be closed.

Performance in virtual threads

Hi,

As JDK gradually shift from platform threads to virtual threads, some of the optimization probably are not needed in the future. What's the roadmap of this project and change for the virtual threads? Do we have a performance analysis or something for virtual threads?

Thanks,
Daniel

Support for timeouts

Would it be possible to add support for timeouts? For example:

final T entry = handoffQueue.poll(timeout, NANOSECONDS);

In our case, MultithreadConcurrentQueue is the queue we would desire to use.

why SpinPolicy.BLOCKING seems to be the fatest?

`
package com.example.demo;

import com.conversantmedia.util.concurrent.DisruptorBlockingQueue;
import com.conversantmedia.util.concurrent.SpinPolicy;
import lombok.extern.slf4j.Slf4j;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.LinkedBlockingQueue;

@slf4j
public class SimpleTest {
public static int TASK_SUM = 0;
public static int PRODUCER = 0;
public static int CONSUMER = 0;
public static int QUE_SIZE = 0;
public static CountDownLatch countDownLatch;

public void test(final BlockingQueue<String> q) throws InterruptedException {
    //生产者线程
    class Producer implements Runnable {
        @Override
        public void run() {
            for (int i = 0; i < TASK_SUM * CONSUMER; i++) {
                try {
                    q.put("");
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            countDownLatch.countDown();
        }

    }
    ;
    //消费者线程
    class Consumer implements Runnable {
        @Override
        public void run() {
            for (int i = 0; i < TASK_SUM * PRODUCER; i++) {
                try {
                    q.take();
                } catch (InterruptedException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
            countDownLatch.countDown();
        }
    }
    countDownLatch = new CountDownLatch(PRODUCER + CONSUMER);
    Thread[] arrProducerThread = new Thread[PRODUCER];
    for (int i = 0; i < PRODUCER; i++) {
        arrProducerThread[i] = new Thread(new Producer());
    }
    Thread[] arrConsumerThread = new Thread[CONSUMER];
    for (int i = 0; i < CONSUMER; i++) {
        arrConsumerThread[i] = new Thread(new Consumer());
    }
    //go!
    long t1 = System.currentTimeMillis();
    for (int i = 0; i < PRODUCER; i++) {
        arrProducerThread[i].start();
    }
    for (int i = 0; i < CONSUMER; i++) {
        arrConsumerThread[i].start();
    }
    countDownLatch.await();
    long t2 = System.currentTimeMillis();
    log.info("task: {}, producer: {}, consumer: {}, queue size: {}, time: {} ms", TASK_SUM, PRODUCER, CONSUMER, QUE_SIZE, t2-t1);
}

public static void main(String[] args) throws InterruptedException {

    SimpleTest.TASK_SUM = 100000;
    SimpleTest.PRODUCER = 5;
    SimpleTest.CONSUMER = 5;
    SimpleTest.QUE_SIZE = 512;
    final BlockingQueue<String> q1 = new LinkedBlockingQueue<>(SimpleTest.QUE_SIZE );
    new SimpleTest().test(q1);

    final BlockingQueue<String> q2 = new ArrayBlockingQueue<>(SimpleTest.QUE_SIZE );
    new SimpleTest().test(q2);

    final BlockingQueue<String> q3 = new DisruptorBlockingQueue<>(SimpleTest.QUE_SIZE, SpinPolicy.BLOCKING);
    new SimpleTest().test(q3);

}

}
`

I tested the disruptor blocking queue for its performance, and it seems only SpinPolicy.BLOCKING strategy is faster than ArrayBlockingQueue or LinkedBlockingQueue, SpinPolicy.WAITING is extremely slow! am I wrong or where is the problem?

Declare an Automatic Module Name

Log4j 2 uses this library as an optional dependency. One of the goals of Log4j 2 3.x is to support the Java Platform Module System. To do that every dependency it uses must declare its module name. The fully compliant way to do that would be to create a module-info.java file, but at the very least the Automatic-Module-Name header needs to be added to META-INF/MANIFEST.MF to declare the name of the module. Otherwise Java will use the jar name minus the version, which is almost always going to be a problem.

ConcurrentStack peek method exception when empty

Hi,

just started looking at your project especially interested in ConcurrentStack for an object pool usage. While reviewing the code I found the peek method implementation very strange.

Could you please comment on following findings?

    @Override
    public final N peek() {
        // read the current cursor
        int spin = 0;
        for(;;) {

            final long readLock = seqLock.readLock();
            final int stackTop = this.stackTop.get();
            final N  n = stack.get(stackTop-1); << empty stack has stackTop 0, you will get here IndexOutOfBoundsException
            if(seqLock.readLockHeld(readLock)) {  << this calls tells you that there was no write lock between the readLock and readLockHeld method call, right?
                if(stackTop>0) {
                    return stack.get(stackTop-1); << you cannot read here as the stack could be already modified by another writer thread
                } else {
                    return null;
                }
            }
            spin = Condition.progressiveYield(spin);
        }
    }

I think this way it would read better

  @Override
  public final N peek() {
      // read the current cursor
      int spin = 0;
      for(;;) {

          final long readLock = this.seqLock.readLock();
          final int stackTop = this.stackTop.get();
          final N n;
          if(stackTop > 0) {
            n = this.stack.get(stackTop-1);
          } else {
            n = null;
          }
          if(this.seqLock.readLockHeld(readLock)) {
            return n;
          }

          spin = Condition.progressiveYield(spin);
      }
  }

Additionally I think your SequenceLock is just very simplified version of the JDK StampedLock (you always spin/yield where the JDK creates wait nodes under high contention). I am somehow missing there the loadFence call in the readLockHeld method which is nothing else than the validate method on the StampedLock. Currently it does not create any problem as the only usage is the peek method in the ConcurrentStack where both reads are volatile, so there is no risk of reordering. I think the loadFence is mandatory there to get the optimistic read sequence validation working.

 * As noted in Boehm's paper (above), sequence validation (mainly
 * method validate()) requires stricter ordering rules than apply
 * to normal volatile reads (of "state").  To force orderings of
 * reads before a validation and the validation itself in those
 * cases where this is not already forced, we use
 * Unsafe.loadFence.

Regards,
Michal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.