Git Product home page Git Product logo

Comments (2)

twmb avatar twmb commented on August 16, 2024 1

There are two potential culprits,

  1. On any new group, Kafka internally forces a 3s wait, check the config value group.initial.rebalance.delay.ms for more information
  2. If you are restarting your consumer using the same group, then your new consumer gets a new member ID in the existing group. Kafka triggers a join group, and then only after the previous consumer "dies" (misses the session timeout) does Kafka allow the rebalance to continue. In your above code, I don't see any room for a ctrl+c to be handled / it doesn't look like there's a proper LeaveGroup. The defer cl.Close internally does a LeaveGroup, but your for loop is infinite, so the close is never properly leaving the group.

Given the timeout in the logs is 38s, I suspect it's (2).

Let me know if you think it's something else.

from franz-go.

secmask avatar secmask commented on August 16, 2024

I add a logger, it spent a lot of time on read JoinGroup

time=2024-05-23T10:45:17.230+07:00 level=DEBUG msg="read FindCoordinator v4" broker=1 bytes_read=57 read_wait=38.879µs time_to_read=4.379832ms err=<nil>
time=2024-05-23T10:45:17.231+07:00 level=DEBUG msg="opening connection to broker" addr=localhost:9093 broker=1
time=2024-05-23T10:45:17.231+07:00 level=DEBUG msg="connection opened to broker" addr=localhost:9093 broker=1
time=2024-05-23T10:45:17.231+07:00 level=DEBUG msg="connection initialized successfully" addr=localhost:9093 broker=1
time=2024-05-23T10:45:17.231+07:00 level=DEBUG msg="wrote JoinGroup v9" broker=1 bytes_written=137 write_wait=587.209µs time_to_write=13.092µs err=<nil>
time=2024-05-23T10:45:17.240+07:00 level=DEBUG msg="read JoinGroup v9" broker=1 bytes_read=66 read_wait=38.105µs time_to_read=8.385049ms err=<nil>
time=2024-05-23T10:45:17.240+07:00 level=INFO msg="join returned MemberIDRequired, rejoining with response's MemberID" group=my-group-identifier member_id=kgo-2d3e0e05-dd52-4cb6-bb09-fcc2969a1d7a
time=2024-05-23T10:45:17.240+07:00 level=DEBUG msg="wrote JoinGroup v9" broker=1 bytes_written=177 write_wait=26.809µs time_to_write=10.857µs err=<nil>
time=2024-05-23T10:45:52.167+07:00 level=DEBUG msg="read JoinGroup v9" broker=1 bytes_read=193 read_wait=30.731µs time_to_read=34.926236626s err=<nil>
time=2024-05-23T10:45:52.167+07:00 level=INFO msg="joined, balancing group" group=my-group-identifier member_id=kgo-2d3e0e05-dd52-4cb6-bb09-fcc2969a1d7a instance_id=<nil> generation=2 balance_protocol=sticky leader=true
time=2024-05-23T10:45:52.167+07:00 level=INFO msg="balancing group as leader"
time=2024-05-23T10:45:52.167+07:00 level=INFO msg="balance group member" id=kgo-2d3e0e05-dd52-4cb6-bb09-fcc2969a1d7a interests="interested topics: [foo], previously owned: "
time=2024-05-23T10:45:52.167+07:00 level=INFO msg=balanced plan=kgo-2d3e0e05-dd52-4cb6-bb09-fcc2969a1d7a{foo[0]}
time=2024-05-23T10:45:52.167+07:00 level=INFO msg=syncing group=my-group-identifier protocol_type=consumer protocol=sticky

from franz-go.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.