event.payload["last_modified"] is not the timestamp to be notified.

Recently we had a situation where a lot of records were published. The timestamp that was sent was the timestamp of the first record of the list, and not the collection timestamp.

See Kinto/kinto#945

Some of the tests in test_listeners aren't really about the listener. I just dumped them in there because I wanted to re-use the fixture. Split everything up a little bit more.

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Right now we push a notification on e.g. every change to the monitor-changes bucket. That means we trigger when records are first copied into the main-preview bucket as well as after they are approved and copied into the main bucket. However, the version ID is the same in both cases, so megaphone doesn't treat the second push as an actual push. Instead, the client gets notified when the preview collection is updated. This isn't ideal.

Discussing this with Mat on IRC, here are some proposed approaches:

• Maybe the monitor/changes collection should get a new ETag when the records are copied to main. But this seems hard because the records should be the same records from the main-preview collection, and the ETag comes from the latest last_modified in the collection.
• Maybe we could remove the preview buckets from the monitor/changes resource. But this is inconvenient because the client relies on finding things in monitor/changes to load them, even for preview collections.
• Finally, maybe we can exclude the preview buckets from kinto-megaphone events. This means setting up a new type of listener that is somehow smart enough to decompose events on the kinto-changes collection, filter out records that are relevant to it, and perform kinto-megaphone updates only sometimes. This seems like the most plausible solution, although it requires a certain amount of work.

The filtering proposed here goes a bit beyond the notification filtering proposed in https://kinto.readthedocs.io/en/stable/tutorials/notifications-custom.html and Kinto/kinto#1499, so maybe we should just build something into this package.

Currently the listener runs at startup and makes some database queries to try to figure out what the "current" version of the megaphone version for monitor/changes should be. Unfortunately, during a migration, the application comes up before the migration runs, and the listener may not be able to execute the queries it wants to. Instead, is it maybe possible to do queries in a "just-in-time" fashion on a per-request basis, perhaps in a ResourceRead event handler?

/cc @leplatrem @autrilla