shoryuken-later
A scheduling plugin for Shoryuken that uses Dynamo DB to delay messages arbitrarily far into the future.
Features
Supports distributed architectures
An SQS message is only queued if a conditional delete of the DDB item is successful. This eliminates any potential race condition, so if more than one shoryuken-later process is polling the same schedule table then no redundant SQS messages will be queued.
NOTE: You shouldn't really need to run more than one process, but if you do it will be safe.
One or more schedule tables
Supports polling one or more DynamoDB tables for messages.
later:
tables:
- default_schedule
- other_scheduleNamespaced configuration
You can use the same configuration file for both Shoryuken and Shoryuken::Later, because the new configuration options are namespaced.
# These keys are used by both Shoryuken and Shoryuken::Later
aws:
access_key_id: ... # or <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: ... # or <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
region: us-east-1 # or <%= ENV['AWS_REGION'] %>
logfile: some/path/to/file.log
# This key is only used by Shoryuken::Later
later:
delay: 5 * 60 # How frequently to poll the schedule table, in seconds.
pidfile: some/path/to/file.pid
tables:
- table1
# These keys are only used by Shoryuken
concurrency: 3
delay: 0
queues:
- [queue1, 1]
- [queue2, 2]Usage
Starting the schedule poller
Start the shoryuken-later schedule poller with a command like:
bundle exec shoryuken-later --config shoryuken.ymlRun it as a daemon inside of your Rails app with a command like:
bundle exec shoryuken-later --config shoryuken.yml --rails --daemonIntegration with ActiveJob
A custom ActiveJob adapter can used to support delaying messages arbitrarily far into the future.
# config/application.rb
config.active_job.queue_adapter = :shoryuken_laterWhen you use the :shoryuken_later queue adapter, jobs to be performed farther than 15 minutes into the future (by setting the wait or wait_until ActiveJob options), will be inserted into the default schedule table. You can set the default schedule table in an initializer.
# config/initializers/shoryuken_later.rb
Shoryuken::Later.default_table = "#{Rails.env}_myapp_later"Integration with Shoryuken::Worker
A new method named perform_later is added to Shoryuken::Worker allowing messages to be delayed arbitrarily far into the future. If the delay is 15 minutes or less, then the message is enqueued into the specified SQS :queue as usual. Otherwise, the message is inserted into the specified DynamoDB :schedule_table.
require 'shoryuken-later'
class MyWorker
include Shoryuken::Worker
shoryuken_options queue: 'default', schedule_table: 'default_schedule'
end
# Schedules a message to be processed 30 minutes from now.
MyWorker.perform_later(Time.now + 30 * 60, 'Foobar')Requirements
Ruby 2.0 or greater. Ruby 1.9 is no longer supported.
Installation
Add this line to your application's Gemfile:
gem 'shoryuken-later'Or to get the latest updates:
gem 'shoryuken-later', github: 'joekhoobyar/shoryuken-later', branch: 'master'And then execute:
$ bundle
Or install it yourself as:
$ gem install shoryuken-later
Documentation
Learn about using Shoryuken::Later at the Shoryuken::Later Wiki.
Learn about using Shoryuken at the Shoryuken Wiki.
Credits
Pablo Cantero, creator of Shoryuken, and everybody who contributed to it. I borrowed a lot of code from Shoryuken itself as a shortcut to making this gem.
Contributing
- Fork it ( https://github.com/joekhoobyar/shoryuken-later/fork )
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent