Comments (6)
i think that this related to latest changes for MarkCommitRecords
from franz-go.
i'm check code and dont see any errors getUncommittedLocked always protected via g.mu.Lock/g.mu.Unlock
@twmb can you check?
from franz-go.
another trace
fatal error: concurrent map writes
fatal error: concurrent map writes
goroutine 439 [running]:
runtime.throw(0x136e55b, 0x15)
/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc000727db0 sp=0xc000727d80 pc=0x445332
runtime.mapassign_fast32(0x116e0e0, 0xc0001c4e10, 0xc00000000f, 0xc0001c05c8)
/usr/local/go/src/runtime/map_fast32.go:176 +0x325 fp=0xc000727df0 sp=0xc000727db0 pc=0x41f725
github.com/twmb/franz-go/pkg/kgo.(*Client).MarkCommitRecords(0xc00016ea00, 0xc0063db020, 0x1, 0x1)
github.com/twmb/franz-go/pkg/kgo/consumer_group.go:1850 +0x151 fp=0xc000727ed0 sp=0xc000727df0 pc=0xa8a791
some.ru/service-platform/framework/framework-modules/broker-kgo.(*subscriber).handleFetches.func1(0x1, 0x0)
some.ru/service-platform/framework/framework-modules/broker-kgo/kgo.go:448 +0x2b3 fp=0xc000727f78 sp=0xc000727ed0 pc=0xadf493
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0001c6de0, 0xc00491d580)
golang.org/x/sync/errgroup/errgroup.go:57 +0x59 fp=0xc000727fd0 sp=0xc000727f78 pc=0xada039
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc000727fd8 sp=0xc000727fd0 pc=0x47d9c1
created by golang.org/x/sync/errgroup.(*Group).Go
golang.org/x/sync/errgroup/errgroup.go:54 +0x66
from franz-go.
Good find! I've merged your PR, that lock is critical, I'm surprised I missed this. I'll tag this directly to 1.0 quite soon.
from franz-go.
thanks, can you wait 1-2 days? i'm profile my app to check differences with github.com/segmention/kafka-go performance
may be i can find some interesting stuff or have some questions, ok ?
from franz-go.
Hopefully nothing surprising at this point, I've benchmarked differences myself with the bench example. I was planning to tag in an hour, I can wait until tomorrow or at the latest (hopefully), Monday.
from franz-go.
Related Issues (20)
- Align connects metrics to prevent no data scenarios HOT 4
- Guidance on Manual Commit error with group consumer HOT 1
- record failed after being retried too many times HOT 2
- Metadata Refresh Does't Update Topic IDs After Topic Recreation HOT 3
- Library doesn't fail if it cannot update metadata HOT 1
- [CustomPartitioner] Help required HOT 2
- Inconsistent partitioning strategy HOT 6
- Batch and retry uncommitted records with group consumer and kotel HOT 6
- Kip-848: Next Generation Consumer Group HOT 4
- Consumer group rebalancing bug when switching from eager to cooperative consumers HOT 8
- topic creation configs HOT 3
- reverted offset commit in kafka HOT 2
- Clarification NoOffsetReset HOT 2
- v1.17 release status
- Support force using ListOffsetsRequest to fetch offset HOT 8
- kgo.FetchMaxPartitionBytes: increasing this option causes high memory usage HOT 3
- Any way to assign only commited offsets on rebalance? HOT 2
- Produce batch time metrics HOT 3
- OnPartitionsAssigned called twice for the same partitions HOT 6
- ListOffsetsAfterMilli returns unexpected results when called against kfake cluster
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from franz-go.