Git Product home page Git Product logo

sentinel-golang's Introduction

Sentinel Logo

Sentinel: The Sentinel of Your Microservices

CI codecov GoDoc Go Report Card License Gitter GitHub last commit GitHub repo size GitHub closed issues

Introduction

As distributed systems become increasingly popular, the reliability between services is becoming more important than ever before. Sentinel takes "flow" as breakthrough point, and works on multiple fields including flow control, traffic shaping, concurrency limiting, circuit breaking and system adaptive overload protection, to guarantee reliability and resiliency of microservices.

flow-overview

Sentinel provides the following features:

  • Rich applicable scenarios: Sentinel has been wildly used in Alibaba, and has covered almost all the core-scenarios in Double-11 (11.11) Shopping Festivals in the past 10 years, such as “Second Kill” which needs to limit burst flow traffic to meet the system capacity, throttling, circuit breaking for unreliable downstream services, distributed rate limiting, etc.
  • Real-time monitoring: Sentinel also provides real-time monitoring ability. You can see the runtime information of a single machine in real-time, and pump the metrics to outside metric components like Prometheus.
  • Cloud-native ecosystem: Sentinel Go provides out-of-box integrations with cloud-native components.

Documentation

GoDoc

See the 中文文档 for the document in Chinese.

See the Wiki for full documentation, examples, blog posts, and other information.

If you are using Sentinel, please leave a comment here to tell us your scenario to make Sentinel better. It's also encouraged to add the link of your blog post, tutorial, demo or customized components to Awesome Sentinel.

Sub-projects

Bugs and Feedback

For bug report, questions and discussions please submit GitHub Issues.

Contributing

Contributions are always welcomed! Please see CONTRIBUTING for detailed guidelines.

You can start with the issues labeled with good first issue.

Communication

  • DingTalk Group (钉钉群): 23339422
  • Gitter

sentinel-golang's People

Contributors

alexstocks avatar alipebt avatar binbin0325 avatar blanet avatar cafra avatar chenjiandongx avatar cncal avatar dependabot[bot] avatar gorexlv avatar horoc avatar jnan806 avatar li-jin-gou avatar liqiangz avatar ljun20160606 avatar louyuting avatar luckyxiaoqiang avatar mstch avatar novoland avatar pantianying avatar sczyh30 avatar sdttttt avatar skyenought avatar testwill avatar tylitianrui avatar weiyuanke avatar wenxuwan avatar wuxuer avatar yedamao avatar zhou-haowei avatar zsy619 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sentinel-golang's Issues

FileRefreshable DataSource implementation

Issue Description

Type: feature request

Describe what feature you want

FileRefreshable DataSource implementation

Additional context

Add any other context or screenshots about the feature request here.

util/time.go 方法参数重命名并添加参数注释

Issue Description

Type: feature request

Describe what feature you want

给FormatTimeMillis参数重命名并添加注释
给FormatDate添加注释

Additional context

Add any other context or screenshots about the feature request here.
1582601608577

[Feature] Statistic data structure, sliding window

statistic data structure based on sliding window

Type: feature request

Describe what feature you want

basic data structure implemented by sliding window is to record basic metrics, including PASS, BLOCKED, SUCCESS, ERROR, RT.

Add unit test for common util

Issue Description

Type: feature request
Add unit test for common util

Describe what feature you want

Code path: util/*
Target coverage: 50+% statements.

Additional context

Add any other context or screenshots about the feature request here.

Complete Sentinel Golang Code Specification

Issue Description

Type: feature request

Describe what feature you want

  1. Complete code specifications wiki;
  2. Based on code specifications to refine all code.

Including:

  1. Log standardization
    2.Comments standardization
    3.Code Related.
  2. Format(using go fmt)
  3. Pr standardization.

Additional context

Add any other context or screenshots about the feature request here.

Add unit test for statistic module

Issue Description

Type: feature request
Add unit test for statistic module

Describe what feature you want

Code path: core/stat/*
Target coverage: 80+% statements.

Additional context

Add any other context or screenshots about the feature request here.

Basic abstraction for data-source extension

Issue Description

Type: feature request

Describe what feature you want

Define datasource extension framework
implement FileRefreshableDataSource

Additional context

Add any other context or screenshots about the feature request here.

Provides logging component extension capabilities and Standardized log format

#Issue Description

Type: feature request

Describe what feature you want

Currently, logging module define the standard log interface (Logger and provide the capacity to extend this own logging component.

User could use other logging component, like Logrus and zap, to replace the default logging.

Besides, Sentinel should standardized log format. using log message + KV parameters to replace current implementation,

Add integration module for dubbo-go

Issue Description

Type: feature request

Describe what feature you want

Add integration module for dubbo-go client and server.

Additional context

Discussions are welcomed!

Add any other context or screenshots about the feature request here.

[BUG] Data-Race for metricBucket.addRt

Issue Description

func (mb *metricBucket) addRt(rt int64) {
	mb.add(base.MetricEventRt, rt)
	if rt < atomic.LoadInt64(&mb.minRt) {
		atomic.StoreInt64(&mb.minRt, rt)
	}
}

here, not concurrent safe for mb.minRt. It's better to use Mutex

another missing:

func (n *BaseStatNode) AddRtAndCompleteRequest(rt, count uint64) {
	n.rollingCounter.AddCount(base.MetricEventComplete, int64(count))
	n.rollingCounter.AddCount(base.MetricEventRt, int64(count)) // should be `rt`
}

Type: bug report

Describe what happened

Describe what you expected to happen

How to reproduce it (as minimally and precisely as possible)

Tell us your environment

Anything else we need to know?

Thanks for your work.

Add unit test for flow control

Issue Description

Type: feature request
Add unit test for flow control

Describe what feature you want

Code path: core/flow/*
Target coverage: 50+% statements.

Additional context

Add any other context or screenshots about the feature request here.

Adapt to the dashboard of Java version

Issue Description

sentinel will reuse the dashboard of java version.

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

unify configuration

Issue Description

Type: feature request

Currently, logging config and general config is separated.
Unifying all configurations makes sense

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

Kubernetes CRD data source

Issue Description

Type: feature request
Investigate the feasibility of adapting Sentinel to the cloud native

Describe what feature you want

Need detail investigation document.

Additional context

Add any other context or screenshots about the feature request here.

Add etcd data-source extenstion

Issue Description

Type: feature request
Adapt etcd as dynamic datasource for Sentinel

Describe what feature you want

Need detail design.
PR

Additional context

Add any other context or screenshots about the feature request here.

Add integration module for RocketMQ

Issue Description

Type: feature request

Describe what feature you want

Add integration module for RocketMQ.

Consumer/Provider filter.

Additional context

Add any other context or screenshots about the feature request here.

Support go 1.14 in CI

Issue Description

Type: feature request

Describe what feature you want

Support go 1.14 in CI

Additional context

Add any other context or screenshots about the feature request here.

Circuit breaking implementation

Issue Description

Circuit breaker

Describe what feature you want

                                          |                                                                                                                    
                                          |                                                                                                                    
                                          |                                                                                                                    
                                          |                                                                                                                    
                                          |  update rules dynamically                                                                                          
                                          |                                                                                                                    
                                          |                                                                                                                    
                                          |                                                                                                                    
                +-------------------------+-------------------------------------------------------------------------------------------------------------------+
     entry      |                         |                                  Circuit Breaker Slot                                                             |
                |                         v                                                                                                                   |
  --------------+->                                                                                                                                           |
                |                                                                                                                                             |
                |  +-----------------------------+    +-----------------------------+    +-----------------------------+   +-----------------------------+    |
                |  | +-------------------------+ |    | +-------------------------+ |    | +-------------------------+ |   | +-------------------------+ |    |
                |  | |                         | |    | |                         | |    | |                         | |   | |                         | |    |
                |  | |  AverageRtBreakerRule   | |    | |  ErrorRatioBreakerRule  | |    | |  ErrorCountBreakerRule  | |   | |     XXXBreakerRule      | |    |
                |  | |                         | |    | |                         | |    | |                         | |   | |                         | |    |
                |  | +-----------^-------------+ |    | +------------^------------+ |    | +-----------^-------------+ |   | +------------^------------+ |    |
                |  |             |               +--->|              |              +--->|             |               +-->|              |              |    |
                |  | +-----------+-------------+ |    | +------------+------------+ |    | +-----------+-------------+ |   | +------------+------------+ |    |
                |  | | AverageRtCircuitBreaker | |    | |ErrorRatioCircuitBreaker | |    | |ErrorCountCircuitBreaker | |   | | XXXCountCircuitBreaker  | |    |
                |  | |       .CanPass()?       | |    | |       .CanPass()?       | |    | |       .CanPass()?       | |   | |       .CanPass()?       | |    |
                |  | |                         | |    | |                         | |    | |                         | |   | |                         | |    |
                |  | +-------------------------+ |    | +-------------------------+ |    | +-------------------------+ |   | +-------------------------+ |    |
                |  +-----------------------------+    +-----------------------------+    +-----------------------------+   +-----------------------------+    |
                |                                                                                                                                             |
                +---------------------------------------------------------------------------------------------------------------------------------------------+

There are Four main entity:
XXXBreakerRule: rule status machine
XXXCircuitBreaker: the check logic whether
RuleManager: manage all XXXCircuitBreakers and update rules dynamically.
CircuitBreakerSlot: As a CheckSlot to encapsulate all logic about circuit breaker.

Sentinel Golang Version V1 Discussions

Describe what feature you want

Implement Golang version of Sentinel, including basic statistic and flow functions. Some expected features:

  • Basic statistic structure (sliding window that can achieve high concurrency and accuracy)
  • Rate limiting (based on token bucket algorithm, maybe we can leverage golang.org/x/time/rate.Limiter)
  • Logging
  • Metrics interface that can pump real-time metrics to outside consumer
  • Embedded API server
  • Integrate with Sentinel dashboard (machine discovery)

Discussions are welcomed!

polish the method LoadRules() in rule_manager.go

Issue Description

Type: feature request

The implementation of LoadRules() in rule_manager.go might be not elegant, and the channel ruleChan seems to have no obvious meaning.

Describe what feature you want

Call onRuleUpdate() to load rules in LoadRules().

Additional context

Add any other context or screenshots about the feature request here.

Add unit test for system slot

Issue Description

Type: feature request
Add unit test for system slot

Describe what feature you want

Code path: core/system/*
Target coverage: 50+% statements.

Additional context

Add any other context or screenshots about the feature request here.

Improve example code

Issue Description

  • flow
  • circuit breaker
  • hotspot
  • isolation
  • system-adaptive
  • data-sources

[Feature] Add leaky bucket in traffic shaping

Issue Description

Type: feature request

Describe what feature you want

Add local leaky bucket in traffic shaping

Additional context

Add any other context or screenshots about the feature request here.

need a chat room

Issue Description

沟通开发事项

Describe what feature you want

dingtalk or Gitter 都行

Additional context

None.

[Feature] System adaptive rule

Issue Description

Type: feature request

Describe what feature you want

Add support for system adaptive rule (adaptive traffic shaping based on metrics like CPU usage, system load and overall QPS/RT.

Improve the logic of updating flow rules

Issue Description

Type: enhancement

Describe what feature you want

In previous implementations, when users update the flow rules with flow.LoadRules(xxx) function, the previous traffic shaping controllers will be cleared and replaced by the new ones. Sometimes it's more sensible to keep the original traffic shaping controller if the fundamental property items of the flow rule remain unchanged, especially for stateful traffic shaping controllers. This could be improved.

generate mocks for interface automation

Issue Description

While doing unit tests, we usually need to use mock functions. In the current project, mock codes seem to be implemented manually, such as StatNodeMock in core/base/stat_test.go, MetricItemRetrieverMock in core/log/metric/aggregator_test.go, and so on.
As the project progresses, more and more mock functions will be available. To improve efficiency, we can consider automatically generating mock functions, by using mockery(https://github.com/vektra/mockery).

Type: feature request

Describe what feature you want

By running "go generate", we can get all mocks for all interfaces to be tested.

Additional context

I found this while solving issue 46, and prepare to implement the test function based on the new mock function.

Add any other context or screenshots about the feature request here.

Add unit test for metric log

Issue Description

Type: feature request

Add metric log unit test

Describe what feature you want

Code path: core/log/metric/*
Target coverage: 80+% statements.

Additional context

Add any other context or screenshots about the feature request here.

Rule based flow control

Issue Description

Rule based flow control

Describe what feature you want

Implement flow slot based on core pipeline. flow slot is stateless and implements flow control based on rules.

Each rule can have its own window length, different window length is very important to degrade service.

Additional context

  1. timestamp align for all sliding windows.
  2. rule can be injected or deleted dynamically.
  3. multi traffic shaping controllers

[Feature] Data-source and data property listener

Issue Description

Type: feature request

Describe what feature you want

Data-source and data property listener (SentinelProperty and PropertyListener in Java). Some considerations:

  • We might need to support multiple SentinelProperty (with different parsers) shared by the single data-source.

Add consul as dynamic datasource

Issue Description

Type: feature request

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

Implement Datasource And PropertyListener

Issue Description

  1. Implement DatasourceExtension and PropertyListener
  2. Multiple configuration formats are supported

Type: feature request

Describe what feature you want

if possible, avoid passing interface{} in Rule converter function, same or like this:

// *_rule_manager.go
func Convert(data interface{}) error {
        config := data.([]byte)
        // convert config to FlowRule/SystemRule
}

which is not elegant and maybe not friendly for multiple configuration format.

Additional context

Add any other context or screenshots about the feature request here.

[Feature] Add Option for api.Entry

Issue Description

Type: feature request

Describe what feature you want

use Functional Options Pattern to create Entry.

Easy to expand, and more readable.

Additional context

Add any other context or screenshots about the feature request here.

Investigate golang type assertion performance

Issue Description

Here is a benchmark:

type Student struct {
	Name  string
	Age   int
	Class string
	Score int
}

func DirectInvoke(s *Student) {
	s.Name = "Jerry"
	s.Age = 18
	s.Class = "20005"
	s.Score = 100
}

func PointerInvoke(p unsafe.Pointer)  {
	s := (*Student)(p)
	s.Name = "Jerry"
	s.Age = 18
	s.Class = "20005"
	s.Score = 100
}
func InterfaceInvoke(i interface{}) {
	s := i.(*Student)
	s.Name = "Jerry"
	s.Age = 18
	s.Class = "20005"
	s.Score = 100
}

func BenchmarkAssertDirectInvoke(b *testing.B) {
	s := new(Student)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		DirectInvoke(s)
	}
	_ = s
}

func BenchmarkAssertPointerInvoke(b *testing.B) {
	s := new(Student)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		PointerInvoke(unsafe.Pointer(s))
	}
	_ = s
}

func BenchmarkAssertInterfaceInvoke(b *testing.B) {
	s := new(Student)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		InterfaceInvoke(s)
	}
	_ = s
}

The result is:(Macbook pro, 2015year, 13in, 8GB memory)
go test -bench='BenchmarkAssert*' -benchmem

goos: darwin
goarch: amd64
pkg: study_golang/study/basic/reflect
BenchmarkAssertDirectInvoke-4           1000000000               0.346 ns/op           0 B/op          0 allocs/op
BenchmarkAssertPointerInvoke-4          1000000000               0.352 ns/op           0 B/op          0 allocs/op
BenchmarkAssertInterfaceInvoke-4        544477173                2.09 ns/op            0 B/op          0 allocs/op
PASS
ok      study_golang/study/basic/reflect        2.147s

It seems the performance using unsafe.Pointer is better than interface{} in some scenario.
Such as update MetricBucket.

Describe what feature you want

Additional context

Need to discuss.

Custom Metrics Export Standardization

Issue Description

Type: feature request

Describe what feature you want

Currently, the metrics is record in monitor log file。
We might need to standardize the metric export. So we could export the metrics information to downstream monitor system. Such as prometheus。

We might need standardize the interface. The way to downstream could be log file、kafka and so on,

The use case:
K8s HPA

Additional context

Add any other context or screenshots about the feature request here.

[Enhancement] refactor base statNode

Issue Description

refactor statNode

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.