trotzig / burrito Goto Github PK

When the number of subscribers for a feed grows beyond approximately 2000, we started seeing DataStoreTimeoutExceptions from app engine. Since broadcasts are made from a task queue, the broadcast is retrtied until successful. Most of the times, the second or third attempt succeeded.

There is a side effect to retrying a broadcast: Messages can be sent more than once - since the task knows nothing about which subscribers it has already sent to. This is not a major issue for live (real-time) sitelets since most likely, the user will never notice. There might however be problems with other feeds (for example a chat feed).

Datastore queries in App Engine seem to timeout after 10 seconds. If the full query has not finished within that time, a timeout exception will occur.