Spring Boot App to recreate "binding corruption" where after a TAS recreate of a 3 node rabbitmq cluster, we see issues where a queue and binding exists but messages published to the routing key do not route to the queue.

This app is creating transient, auto-delete queues that are bound to a durable topic exchange.

We also have the following server-side policy applied to these queues:

Finds all the routing keys belonging to the durable topic exchange and publishes a mandatory message on each one. The expectation is that no mandatory messages are returned. If they are, then this is what I am referring to as a "corrupt binding".

Re-declaring the binding does not fix it. You have to first unbind and then re-bind and then messages will start flowing to the queue again.

To Run BindingCorruptionTest, activate the test spring profile or configure --binding-corruption-test.enabled=true

To Run BindingCorruptionDetector, activate the detector spring profile or configure --binding-corruption-detector.enabled=true

See the following classes for additional configuration:

1. RabbitProperties
2. BindingCorruptionTestProperties
3. BindingCorruptionDetectorProperties

Recreating seems to be based on the load on the cluster and how fast the connections reconnect after connection loss.

On a cluster with 5k queues and no message traffic:

• I can only recreate with a network recovery delay < 1 second

On a cluster with 5k queues and roughly 50-100 messages/sec of traffic:

• network recovery delay settings less than 1 second generally create 2k+ corrupt bindings
• I've tested all the way up to a 10 second recovery delay and still seeing hundreds of binding corruptions