edwingeng / wuid Goto Github PK

View Code? Open in Web Editor NEW

529.0 17.0 48.0 215 KB

An extremely fast globally unique number generator.

License: BSD 3-Clause "New" or "Revised" License

Go 78.01% Shell 21.99%

uuid guid snowflake

wuid's Introduction

Overview

WUID is a universal unique identifier generator.
WUID is much faster than traditional UUID. Each WUID instance can even generate 100M unique identifiers in a single second.
In the nutshell, WUID generates 64-bit integers in sequence. The high 28 bits are loaded from a data source. By now, Redis, MySQL, MongoDB and Callback are supported.
The uniqueness is guaranteed as long as all WUID instances share a same data source or each group of them has a different section ID.
WUID automatically renews the high 28 bits when the low 36 bits are about to run out.
WUID is thread-safe, and lock free.
Obfuscation is supported.

Benchmarks

BenchmarkWUID           159393580          7.661 ns/op        0 B/op       0 allocs/op
BenchmarkRand           100000000         14.95 ns/op         0 B/op       0 allocs/op
BenchmarkTimestamp      164224915          7.359 ns/op        0 B/op       0 allocs/op
BenchmarkUUID_V1        23629536          43.42 ns/op         0 B/op       0 allocs/op
BenchmarkUUID_V2        29351550          43.96 ns/op         0 B/op       0 allocs/op
BenchmarkUUID_V3         4703044         254.2 ns/op        144 B/op       4 allocs/op
BenchmarkUUID_V4         5796310         210.0 ns/op         16 B/op       1 allocs/op
BenchmarkUUID_V5         4051291         310.7 ns/op        168 B/op       4 allocs/op
BenchmarkRedis              2996       38725 ns/op          160 B/op       5 allocs/op
BenchmarkSnowflake       1000000        2092 ns/op            0 B/op       0 allocs/op
BenchmarkULID            5660170         207.7 ns/op         16 B/op       1 allocs/op
BenchmarkXID            49639082          26.21 ns/op         0 B/op       0 allocs/op
BenchmarkShortID         1312386         922.2 ns/op        320 B/op      11 allocs/op
BenchmarkKsuid          19717675          59.79 ns/op         0 B/op       0 allocs/op

Getting Started

go get -u github.com/edwingeng/wuid

Usages

Redis

import "github.com/edwingeng/wuid/redis/v8/wuid"

newClient := func() (redis.UniversalClient, bool, error) {
    var client redis.UniversalClient
    // ...
    return client, true, nil
}

// Setup
w := NewWUID("alpha", nil)
err := w.LoadH28FromRedis(newClient, "wuid")
if err != nil {
    panic(err)
}

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", w.Next())
}

MySQL

import "github.com/edwingeng/wuid/mysql/wuid"

openDB := func() (*sql.DB, bool, error) {
    var db *sql.DB
    // ...
    return db, true, nil
}

// Setup
w := NewWUID("alpha", nil)
err := w.LoadH28FromMysql(openDB, "wuid")
if err != nil {
    panic(err)
}

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", w.Next())
}

MongoDB

import "github.com/edwingeng/wuid/mongo/wuid"

newClient := func() (*mongo.Client, bool, error) {
    var client *mongo.Client
    // ...
    return client, true, nil
}

// Setup
w := NewWUID("alpha", nil)
err := w.LoadH28FromMongo(newClient, "test", "wuid", "default")
if err != nil {
    panic(err)
}

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", w.Next())
}

Callback

import "github.com/edwingeng/wuid/callback/wuid"

callback := func() (int64, func(), error) {
    var h28 int64
    // ...
    return h28, nil, nil
}

// Setup
w := NewWUID("alpha", nil)
err := w.LoadH28WithCallback(callback)
if err != nil {
    panic(err)
}

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", w.Next())
}

Mysql Table Creation

CREATE TABLE IF NOT EXISTS `wuid` (
    `h` int(10) NOT NULL AUTO_INCREMENT,
    `x` tinyint(4) NOT NULL DEFAULT '0',
    PRIMARY KEY (`x`),
    UNIQUE KEY `h` (`h`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Options

WithSection brands a section ID on each generated number. A section ID must be in between [0, 7].
WithStep sets the step and the floor for each generated number.
WithObfuscation enables number obfuscation.

Attentions

It is highly recommended to pass a logger to wuid.NewWUID and keep an eye on the warnings that include "renew failed". It indicates that the low 36 bits are about to run out in hours to hundreds of hours, and the renewal program failed for some reason. WUID will make many renewal attempts until succeeded.

Special thanks

dustinfog

Ports

swift - https://github.com/ekscrypto/SwiftWUID

wuid's People

Contributors

Stargazers

Watchers

wuid's Issues

兵哥，tag参数是做什么用的啊

optimize too many tables in mysql question

this wuid design make too many tables in mysql .

What about segmentio/ksuid?

What about https://github.com/segmentio/ksuid?

调用wuid生成id后，会自动关闭数据库连接，导致后续数据库操作无法继续执行

调用wuid生成id后，会自动关闭数据库连接，导致后续数据库操作无法继续执行
package tools
import (

"database/sql"
"github.com/astaxie/beego/orm"
"github.com/edwingeng/wuid/mysql"
_"github.com/go-sql-driver/mysql"

)

var newDB = func() (*sql.DB,bool,error){
db,_ := orm.GetDB()
return db,true,nil
}

var g = wuid.NewWUID("default", nil)

func NewId() (uint64){
g.LoadH28FromMysql(newDB, "wuid")
return g.Next()
}

调用代码：

func AddUser(u *User) (int64, error) {
// db,err := orm.GetDB()
//fmt.Println(err)
o := orm.NewOrm()
u.Id = 74837477878
for i:=0;i<100;i++ {
fmt.Println(tools.NewId())
}

fmt.Println(u)
return o.Insert(u)

}

插入操作无法继续执行，
[ORM]2019/05/12 22:09:56 -[Queries/default] - [FAIL / db.Exec / 0.0ms] - [INSERT INTO sys_user (id, username, password, phone, email, name, avatar, salt, status, is_staff, last_login_time, login_error_times, remark, delete_flag, create_by, create_time, update_by, update_time) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)] - 74837477878, qingmumao,
xxxxxxxx, 13332978703, [email protected], , , , `0`, `0`, `0`, `0`, , 0, 0, 0, 0, 0 - sql: database is closed

能否把go-redis依赖调整到最新v8版本

现在项目本身在使用最新v8版本，然后又需要单独实例化一个v6版本

问下啊，ID是趋势递增的么

问下啊，ID是趋势递增的么？

请问高28位超过后，有什么解决方案？

2^64总量始终有限，低36位用完可以更新高28位+1，当高位超过后，有什么解决方案？

为什么将返回结果从uint64改成了int64呢？

Are the IDs generated time sortable?

请问一下,如果不同机房,通过设置Section ID可以实现跨机房全局唯一吗

// A机房
once.Do(func() {
	g=wuid.NewWUID("default",nil,wuid.WithSection(1))
        g.LoadH24FromMysql(....
})
// B机房
once.Do(func() {
	g=wuid.NewWUID("default",nil,wuid.WithSection(2))
        g.LoadH24FromMysql(....
})
// 然后g在程序中作为全局变量,这样,两个g都不共享mysql,可以达到全局唯一吗

callback method after restart

If the process is restarted, the callback method is high 28, is it possible to repeat the id

请问这个id 生成器是有序的吗？

这个id生成器是有序的吗？我需要做排序

在32位系统上编译报错

我使用 GO111MODULE=on GOARCH=386 来编译，报空指针：

Document expected behaviour of floor

The behaviour of how the floor impacts the output isn't really documented and the term itself brings confusion.

From https://en.wikipedia.org/wiki/Floor_and_ceiling_functions :

In mathematics and computer science, the floor function is the function that takes as input a real number x, and gives as output the greatest integer less than or equal to x.

Based on this definition, when I specified a step of 1024 with a floor of 897 and H28 of 1 I didn't expect to receive a value of 0x00000010000003a7 followed by 0x0000001000000728 and 0x0000001000000aa9; given that 0xAA9 is much higher than the "floor" value. It's obvious why this is happening when looking at the code where next() returns n / floor * floor.

The best that I can see is this is somehow useful in order to make it less obvious what the "step" actually is, using a Real number as step and benefiting from rounding to nearest Integer, causing the steps to be +/- 1. A floor > 1 is also a requirement when used with obfuscation to hide the obfuscation values used.

However the main issue is that I couldn't find any documentation about this expected behaviour, so as a user of this library I ended up having to dig through the code to figure it out.

请问wuid生成的id拿来做主键合适吗

假如用wuid来表示一个订单的ID, 存储到数据里里面推荐用它的字符串表示还是直接用数字呢?
如果存储的是数字, wuid返回uint64类型, 拿来做主键合适吗?