Git Product home page Git Product logo

Comments (5)

beyondfengyu avatar beyondfengyu commented on July 26, 2024 2

上面说得很明白,一个节点就一个单例的SnokeFlake

from snowflake.

hartmut-co-uk avatar hartmut-co-uk commented on July 26, 2024

Hi @huxiaofei, you need to initialise each SnowFlake instance with a different tuple of <datacenterId, machineId>.

You can generate up to
12bit = 4096 unique IDs
per datacenterId
per machineId
per millisecond

Valid values for datacenterId & machineId are 5bit each -> 0...31

Try following instead:

package com.example.demo.utils;

import java.util.HashSet;
import java.util.Set;

class SnowFlakeKeyTest {
    private static Set keys = new HashSet();

    public static void main(String[] args) {
        long start = System.currentTimeMillis();

        for (int i = 0; i < 31; i++) {
            for (int j = 0; j < 31; j++) {
                SnowFlakeKeyTest.GetKey runner = new SnowFlakeKeyTest().new GetKey(i, j);
                new Thread(runner).start();
            }
        }

        long end = System.currentTimeMillis();
        System.out.println(keys.size() + " unique ids have been generated in " + (end - start) + "ms!\n" + keys);

    }

    class GetKey implements Runnable {

        private final int datacenterId;
        private final int machineId;

        public GetKey(int datacenterId, int machineId) {
            this.datacenterId = datacenterId;
            this.machineId = machineId;
        }

        @Override
        public void run() {
            SnowFlake key = new SnowFlake(datacenterId, machineId);
            long k = key.nextId();
            if (!keys.contains(k)) {
                keys.add(k);
            } else {
                System.out.println("重复:" + k);
            }
        }
    }
}

from snowflake.

huxiaofei avatar huxiaofei commented on July 26, 2024

thank s for @hartmut-co-uk reply.

在实际部署时,纵然有很多的数据中心,上百台服务器,具体到某一个服务器上,
datacenterID & machineId 就是一个常量。
此时在多线程并发请求产生ID,就会出现重复。

from snowflake.

hartmut-co-uk avatar hartmut-co-uk commented on July 26, 2024

Hi @huxiaofei
datacenterId & machineId shouldn't be constant but different for each instance of SnowFlake. On the individual instance multi-threading is a common scenario. You need to instantiate SnowFlake as a singleton / bean. The implementation itself is thread safe.

If you can't guarantee singletons by tuple <datacenterId, machineId> you logically will have duplicates.

If you actually have big data / high volumes of IDs to generate (hundreds of servers?) - you'd need to provide IDs as-a-service and request IDs via API. So you can run a cluster of your 'ID Service' (with different configuration datacenterId & machineId) and request new IDs from your servers/applications via API.

from snowflake.

hartmut-co-uk avatar hartmut-co-uk commented on July 26, 2024

Hi, still not a real Unit Test - but I've put together a more proper example of above's example (concurrency/synchronization) which I think might be worth sharing...

import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

class SnowFlakeKeyTest {
    private static Set keys = Collections.synchronizedSet(new HashSet(9610000));
    private static AtomicInteger x = new AtomicInteger(0);

    public static void main(String[] args) {
        long start = System.currentTimeMillis();

        List<Thread> threads = new ArrayList<>(961);
        for (int i = 0; i < 31; i++) {
            for (int j = 0; j < 31; j++) {
                SnowFlakeKeyTest.GetKey runner = new SnowFlakeKeyTest().new GetKey(i, j);
                Thread thread = new Thread(runner);
                thread.start();
                threads.add(thread);
            }
        }

        System.out.println(threads.size() + " threads have been started...");

        try {
            System.out.println("Waiting for threads to finish...");
            for (Thread thread: threads) {
                thread.join();
            }
            long end = System.currentTimeMillis();
            System.out.println(keys.size() + " unique ids (should: " + x + ") have been generated in " + (end - start) + "ms!");
        } catch (InterruptedException e) {
            System.out.println("Main thread Interrupted");
        }
    }

    class GetKey implements Runnable {

        private final int datacenterId;
        private final int machineId;

        public GetKey(int datacenterId, int machineId) {
            this.datacenterId = datacenterId;
            this.machineId = machineId;
        }

        @Override
        public void run() {
            SnowFlake key = new SnowFlake(datacenterId, machineId);

            for (int i = 0; i < 10000; i++) {
                long k = key.nextId();
                if (!keys.add(k)) {
                    System.out.println("duplicate:" + k);
                }
                x.incrementAndGet();
            }
        }
    }
}

Output:

961 threads have been started...
Waiting for threads to finish...
9610000 unique ids (should: 9610000) have been generated in 3757ms!

from snowflake.

Related Issues (12)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.