nickelser / activejob-traffic_control Goto Github PK
View Code? Open in Web Editor NEWRate limiting/job enabling for ActiveJob using distributed locks in Redis or Memcached.
Rate limiting/job enabling for ActiveJob using distributed locks in Redis or Memcached.
I am using this gem with good_job as a backend
I figured it out, I will gladly do a PR with example for docs, after I get back from vacation. Leaving today, so maybe in 10 days from now :)
Hi @nickelser . I've looked through the codebase but have not been able to find a license file or similar information regarding the license for activejob-traffic_control
. Could you please advise what license governs this package? Many thanks.
Just found this gem and it looks really neat. I'm looking for a possible gem to provide a solution to an issue we recently encountered. Our issue is quite similar to the one mentioned in #4 but slightly different. Let me explain:
One of our systems(our Android/iOS app) reports both some user data and client data(like its current version) with each API call to our main system. We do not have any request limitation on this API so we can receive lots of calls per second. This information is stored in the system on the user model but also reported to another external system. The external system has rate limitation though.
The throttling functionality provided by this gem is perfect to not exceed the rate limit of the external system. Though part of the user data we submit to the external system is the timestamp of the last API request for each user. Therefore we would like to replace any previous queued jobs rather then drop new jobs that are exceeding the rate limit.
I looked through the code briefly to see if something like this could easily be achieved but it does not seem like that. Or do you have any suggestion how this could be achieved with this gem?
In general I think this could be very useful for any kind of REST based API with rate limitation when posting data to that API since one wants the last state transfered and none of the previous ones.
How is enable! and disable! supposed to work?
I'm using sidekiq and check a certain condition throughout the execution of the job and disable or enable it based ons said condition. However the job seems to get perpetually stuck in a planned state.
I'm using memory_store as cache client.
ActiveJob::TrafficControl.cache_client = ActiveSupport::Cache.lookup_store(:memory_store)
code managing the state:
Job.disable! unless @asset.ready?
Job.enable! if @asset.ready?
Hello.
Should we do unlock
of token?
see:
# manually locking/unlocking
# the return value from lock without a block is a unique token valid only for the current lock
# which must be unlocked manually
token = suo.lock
foo.baz!
suo.unlock(token)
From @psteininger (#3):
I did experience a problem with a Sidekiq backed job, which had named (not positional) parameters. The job would be called seemingly w/o parameters. I did change the signature of the perform method and got around it. Perhaps I should open another issue?
Can this be used along with Sidekiq?
I haven't tried it myself, but I am wondering if there is a way to stack throttle
definitions. Here is a use case:
We have a portion of a system that receives callbacks about incoming calls from provider A, and hangs up the call, and then schedules a job to start a call from provider B. Provider B has a rate limit of 2 requests per second. However, quite often provider A hits us twice with a callback for the same call, 3-15 seconds apart. So I would like to have 2 throttle guards:
How could I get this accomplished?
Hi @nickelser
This error is happening on production only. Could this be a cause of missing this code below in the configs?
ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Redis.new }
In the docs it mentions we need to run:
ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Redis.new } # set poolthresholds as needed
I assume it can just go in any initializer. I have put that in my config/initializers/active_job.rb
.
Is this correct? I'm happy to submit a PR to update the docs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.