nickelser / activejob-traffic_control Goto Github PK

I am using this gem with good_job as a backend

I figured it out, I will gladly do a PR with example for docs, after I get back from vacation. Leaving today, so maybe in 10 days from now :)

Hi @nickelser . I've looked through the codebase but have not been able to find a license file or similar information regarding the license for activejob-traffic_control. Could you please advise what license governs this package? Many thanks.

Just found this gem and it looks really neat. I'm looking for a possible gem to provide a solution to an issue we recently encountered. Our issue is quite similar to the one mentioned in #4 but slightly different. Let me explain:

One of our systems(our Android/iOS app) reports both some user data and client data(like its current version) with each API call to our main system. We do not have any request limitation on this API so we can receive lots of calls per second. This information is stored in the system on the user model but also reported to another external system. The external system has rate limitation though.

The throttling functionality provided by this gem is perfect to not exceed the rate limit of the external system. Though part of the user data we submit to the external system is the timestamp of the last API request for each user. Therefore we would like to replace any previous queued jobs rather then drop new jobs that are exceeding the rate limit.

I looked through the code briefly to see if something like this could easily be achieved but it does not seem like that. Or do you have any suggestion how this could be achieved with this gem?

In general I think this could be very useful for any kind of REST based API with rate limitation when posting data to that API since one wants the last state transfered and none of the previous ones.

How is enable! and disable! supposed to work?

I'm using sidekiq and check a certain condition throughout the execution of the job and disable or enable it based ons said condition. However the job seems to get perpetually stuck in a planned state.

I'm using memory_store as cache client.

ActiveJob::TrafficControl.cache_client = ActiveSupport::Cache.lookup_store(:memory_store)

code managing the state:

Job.disable! unless @asset.ready?
Job.enable!  if     @asset.ready?

Hello.

Should we do unlock of token?

see:

• https://github.com/nickelser/activejob-traffic_control/blob/v0.1.3/lib/active_job/traffic_control/throttle.rb#L38
• https://github.com/nickelser/suo/tree/v0.3.2#basic

# manually locking/unlocking
# the return value from lock without a block is a unique token valid only for the current lock
# which must be unlocked manually
token = suo.lock
foo.baz!
suo.unlock(token)

• https://github.com/nickelser/suo/blob/v0.3.2/lib/suo/client/base.rb#L29-L41

From @psteininger (#3):

I did experience a problem with a Sidekiq backed job, which had named (not positional) parameters. The job would be called seemingly w/o parameters. I did change the signature of the perform method and got around it. Perhaps I should open another issue?

Can this be used along with Sidekiq?

I haven't tried it myself, but I am wondering if there is a way to stack throttle definitions. Here is a use case:
We have a portion of a system that receives callbacks about incoming calls from provider A, and hangs up the call, and then schedules a job to start a call from provider B. Provider B has a rate limit of 2 requests per second. However, quite often provider A hits us twice with a callback for the same call, 3-15 seconds apart. So I would like to have 2 throttle guards:

1. Standard 2 per second limit
2. Parameter based throttle (i.e don't call the same number twice in a 20 second window)

How could I get this accomplished?

Hi @nickelser

This error is happening on production only. Could this be a cause of missing this code below in the configs?

ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Redis.new }

In the docs it mentions we need to run:

ActiveJob::TrafficControl.client = ConnectionPool.new(size: 5, timeout: 5) { Redis.new } # set poolthresholds as needed

I assume it can just go in any initializer. I have put that in my config/initializers/active_job.rb.

Is this correct? I'm happy to submit a PR to update the docs.