kamui / retriable Goto Github PK
View Code? Open in Web Editor NEWRetriable is an simple DSL to retry failed code blocks with randomized exponential backoff.
Home Page: kamui.github.io/retriable
License: MIT License
Retriable is an simple DSL to retry failed code blocks with randomized exponential backoff.
Home Page: kamui.github.io/retriable
License: MIT License
Dependabot can't resolve your Ruby dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
Bundler::VersionConflict with message: Bundler found conflicting requirements for the Ruby version:
In Gemfile:
Ruby (~> 2.0.0.0)
listen (~> 3.1) was resolved to 3.1.1, which depends on
Ruby (~> 2.2)
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Currently there's the following code in the gemspec:
Lines 26 to 31 in e3f9bf4
This doesn't work because gemspecs evaluate tot static content. This means it's evaluated only when gem build is called. That Ruby version used there determines the actual resulting gem dependencies.
I'm not aware of any method to do this more dynamically for users.
Hi,
First of all, thanks for this library, I like it very much.
What do you think of possibility to declare methods as retriable at class level. Like this:
class A
include Retriable
def my_method(a, b)
# do something
end
retriable :my_method, on: Timeout::Error, tries: 3, base_interval: 1
# equal to
def noisy_method(a, b)
retriable on: Timeout::Error, tries: 3, base_interval: 1 do
# do something
end
end
end
I believe it's pretty easy to implement and it'll allow to remove some noisiness from "retriable" methods. I can implement it myself but first I want to be sure that you'll accept this feature into upstream :) What do you think about it?
The sleep_disabled
configuration option is not documented in the README. Is this intentional? I'm happy to add it in a PR if not.
In the docs it says:
"In your ruby script:
require 'retriable', '~> 2.0'"
This throws and error:
`require': wrong number of arguments (2 for 1) (ArgumentError)
Did you guys mean to put this in bundler? If so there's no 2.0 repo on rubygems.org so this breaks as well.
Just a heads up, Timeout::timeout is inherently unsafe (see https://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/ and http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html).
Thankfully using it isn't on by default so that's good for those of us that don't wish to use it, but perhaps a note to users that they are better off implementing their own timeout methods (e.g. with socket timeout options) might save some people some headache down the line.
Anyway just throwing it out there, thanks for the awesome library!
Currently if retriable anticipates that the max_elapsed_time
will be reached, it just raises the error right then.
This is fine however as a user, I'd like to know if this has occurred and if this is the reason the code is no longer being retried (as opposed to if it was just because we reached the number of tries)
So this issue proposes the following interface:
do_this_when_max_elapsed_time_reached = Proc.new do |exception, try, elapsed_time|
# ex: log here that the max elapsed time has been reached
end
Retriable.retriable(on_max_elapsed_time_reached: do_this_when_max_elapsed_time_reached) do
# code here...
end
This mainly comes up b/c I have an idea for how long the code will run, but it can sometimes run for longer and I'd like to log these instances
This is probably more of a question than an issue but what's the recommended way to set retriable up in the test environment. Without any specific setup I run into
Current I made a config/initializers/retriable.rb
with
if Rails.env.test?
Retriable.configure do |c|
c.tries = 1
end
end
but I was wondering if there is a better way? Or if there is some documentation on how to do testing that I just didn't find...
Thanks for a great gem!
not sure why it's not updating to reflect the gemspec
https://rubygems.org/gems/retriable
obviously not a real bug but a bit of an issue; i would look ever so slightly askance at a gem with bad grammar in the description if trying to choose between retryable
and retriable
Sometimes APIs wrap exceptions so that the top-level exception by itself isn't enough to determine whether or not to retry the code. However, the cause
exception could be used to determine that the code should be retried.
Currently retriable only allows retrying based on the raised exception, and doesn't provide a way to examine the caused exception.
Here is an example. In this case, we want to retry based on the RootCauseError
, but since the API we're using actually raises DemoError
, the code block doesn't get retried.
class RootCauseError < StandardError; end
class DemoError < StandardError; end
begin
i = 0
Retriable.retriable on: [RootCauseError] do
begin
puts "Try #{i += 1}..."
raise RootCauseError.new("BUT THIS IS THE CAUSE")
rescue => e
raise DemoError.new("RETRIABLE ONLY LOOKS AT THIS")
end
end
rescue => e
puts "Raised error: #{e.inspect}"
puts "Cause error: #{e.cause.inspect}"
end
Two ideas for solving this:
on
option that would then be invoked with the error and would return true or false to determine if the error should be retried. This proc could then examine the error class and its cause, and would have the benefit of also supporting lots of advanced cases that are not possible right now.Retriable.retriable(tries: 3, on_retry: ->(*args) { puts 'on_retry' }) do
puts 'execute'
raise 'err'
end
version 1.4.1
execute
on_retry
execute
on_retry
execute
RuntimeError: err
version 2.1.0
execute
on_retry
execute
on_retry
execute
on_retry
RuntimeError: err
code to recreate:
class SomeOtherError < StandardError; end
RETRY_PARAMS = {
on: { StandardError => /pattern/ },
on_retry: Proc.new do |exception, try, elapsed_time, next_interval|
puts "#{exception.class}: '#{exception.message}' - #{try} tries in #{elapsed_time} seconds and #{next_interval} seconds until the next try."
puts exception.backtrace.join("\n")
end
}
Retriable.retriable(RETRY_PARAMS) do
raise SomeOtherError.new('paxern')
end
# Block retries, even though the pattern is not matched
Retriable.retriable(RETRY_PARAMS) do
raise StandardError.new('paxern')
end
# Block does not retry
I would expect SomeOtherError
to have its message matched against /pattern/
, but this does not happen. The gem instead tries to find the pattern with on[StandardError]
, comes up with nothing, decides it is an empty list, and retries even when the pattern is not matched.
Hi, I'm trying to use your gem but when I test it I get this error"
cannot load such file -- retriable
This is my code:
module Integration
require 'retriable'
class Mailchimp
def self.method1
Retriable.retriable do
raise StandardError
end
end
end
end
If I comment out require 'retriable'
I get the error uninitialized constant Integration::Mailchimp::Retriable
Thanks
Currently the on_retry
parameter is called each time the passed in block of code raises a matching exception.
I am proposing that there be a on_final_failure
(or similarly named) parameter which is a proc that is called after all retries have been exhausted.
The use case is that:
I know there are current ways to do this outside of the gem, but I think the following interface could be nice:
do_this_on_each_retry = Proc.new { }
do_this_on_final_retry = Proc.new { }
Retriable.retriable(on_retry: do_this_on_each_retry, on_final_failure: do_this_on_final_retry) do
# code here..
end
This allows me to have custom logic for the final failure, while still allowing the error to be raised by Retriable
I know there is the CHANGELOG.md
file.
But it's unclear to me that what is breaking or not (since it got changes for each beta, not from 1.4.1
to 2.0.0
.
Travis build is currently failing with this:
gem update --system
Updating rubygems-update
Fetching: rubygems-update-3.0.3.gem (100%)
ERROR: Error installing rubygems-update:
rubygems-update requires Ruby version >= 2.3.0.
ERROR: While executing gem ... (NoMethodError)
undefined method `version' for nil:NilClass
The command "gem update --system" failed and exited with 1 during .
Relevant issue on rubygems
The readme mentions being able to turn of retries in tests by doing the following:
Retriable.configure do |c|
if Rails.env.test?
c.tries = 1
end
But from what I can tell is only works when Retriable.retriable
is itself called without the tries
parameter. The default global config is being set, while the individual invocation parameters still take precedence. The same is probably true for all other parameters being set in this way.
Looking at the code this seems to indeed what is happening:
Line 26 in 83aa4d4
Is there something I'm missing? I could create my own wrapper function and check for the test environment there, but that is not what the readme suggests.
Wdyt about replacing noisy houndci by plain running of rubocop alongside of testing on travis?
Currently running rubocop locally is not working, but on hound is everything is ok, so how it checks for errors?!
If you set a high retry limit (e.g. 100), Retriable will exceed the max elapsed time, but the error message says it reached 100 tries:
27 tries in 872.859824735 seconds and 75.33571386221345 seconds until the next try.
Failed after 100 retries with RestClient::ServiceUnavailable: 503 Service Unavailable!
It would be nice if it could say that it reached the max elapsed time instead.
I believe it is best practices to do so in ruby (as it helps ensure consistent testing/debugging) but it's especially relevant here, as nothing in the Gemfile
is version locked.
An example why this is needed:
I just forked this repository and ran bundle update
(I couldn't use bundle install
due to the lack of a lockfile), and I am immediately seeing rubocop
erroring out due to the fact two of the cops (Layout/IndentArray
and Layout/IndentHash
) listed in the config have since been renamed.
My choices are to either:
a) Guess at the intended version of rubocop
using the times gem log .rubocop.yml
, adding the version lock to the Gemfile
and re-running bundle update
, or;
b) Update the .rubocop.yml
.
Obviously, either solution requires changes to a file tracked by git, which I would then have to pick around.
Dependabot can't resolve your Ruby dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
Bundler::VersionConflict with message: Bundler found conflicting requirements for the Ruby version:
In Gemfile:
Ruby (~> 2.0.0.0)
listen (~> 3.1) was resolved to 3.1.1, which depends on
Ruby (~> 2.2)
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Hi,
I was wondering if you would accept a pull request which allowed the option to ignore specific types of errors. Right now it seems like the only option is to rescue from within the block.
Let me know if you have any questions or comments!
Just got my production crash after 3.0.2 just updated
Exception is "interval is not a valid option"
/app/vendor/bundle/ruby/2.2.0/gems/retriable-3.0.2/lib/retriable/config.rb:32
This gem was pulled by google-api-client (that I locked at 0.7.1)
(still looking into what happened... but release of the gem just happened between me looking on staging and deploying prod. Testing staging after the gem update experience same error)
i fixed one issue with the rbx build not even starting by upgrading it to 3.99 (see rubygems/bundler#6163 for details) but now it just looks like every spec that compares a many decimal-ed float is failing. not sure what the answer is or if it's even worth trying to deal with.
check out the build log:
https://travis-ci.org/kamui/retriable/jobs/347208017
I want to retry my logic for 1 hour if something is not true. So I added tries as 360
and base_interval
: as 10. But for some reason, it always exists out in 20 times. Following is my code and log:
Code
count = 0
# 1 hour : 10 second interval x 360 tries = 3600 seconds
Retriable.retriable(on: [MyModule::MyClass::CustomError, Net::OpenTimeout],
tries: 360, base_interval: 10) do
count += 1
print "count: #{count} \n"
raise MyModule::MyClass::CustomError if @food != "tacos"
end
Log:
[5] pry(main)> work.run
count: 1
count: 2
count: 3
count: 4
count: 5
count: 6
count: 7
count: 8
count: 9
count: 10
count: 11
count: 12
count: 13
count: 14
count: 15
count: 16
count: 17
count: 18
count: 19
count: 20
MyModule::MyClass::CustomError: MyModule::MyClass::CustomError
Am I missing anything? Any idea on how I can accomplish this?
Hey all,
I am new to ruby and trying to use this gem.
Running following code:
Retriable.retriable intervals: [1, 2, 3, 4] do
puts "time: #{Time.now}"
raise 'An error has occured.'
end
the result is:
time: 2015-09-30 04:04:11 +0000
time: 2015-09-30 04:04:12 +0000
time: 2015-09-30 04:04:14 +0000
time: 2015-09-30 04:04:17 +0000
but should it re-try one more time? and the last interval is 4 seconds? I thought the number of execution is original one time plus the size of the interval array?
Thanks.
i added this ``gem 'retriable', '> 2.0'` into my gemfile as per README, when i run my bundle install it says 'Could not find gem 'retriable (> 2.0) ruby' in the gems available on this machine'
i just realised there is no 2.0 stable version, it's only beta, wondering when the stable version will be available
Hi ๐
From:
Lines 18 to 20 in d66e4db
I am working on a SDK that has clients to several internal services and I would like to use with_context
before hitting each service so I can let users configure the retry ability of each service independently and I would like to fallback to a default configuration if such context is not configured yet.
For example, mixing what's available from the current documentation:
Retriable.configure do |c|
c.tries = 5
c.max_elapsed_time = 3600 # 1 hour
c.contexts[:aws] = {
tries: 3,
base_interval: 5,
on_retry: Proc.new { puts 'Curse you, AWS!' }
}
c.contexts[:mysql] = {
tries: 10,
multiplier: 2.5,
on: Mysql::DeadlockException
}
end
If later I use postgres
context, I would like to fallback to the global config:
Retriable.with_context(:postgres) do
# use global config, tries: 5, max_elapsed_time: 3600
end
One idea on how to implement this feature would be as a new option to with_context
, for example fallback_global_config: true
, would you accept a PR with such a thing?
Retriable.with_context(:postgres, fallback_global_config: true) do
# use global config, tries: 5, max_elapsed_time: 3600
end
Thanks for sharing this awesome project and considering my feature request ๐
An issue in the CI:
$ gem install bundler -v 1.17.3
Fetching: bundler-1.17.3.gem (100%)
ERROR: Error installing bundler:
invalid gem: package is corrupt, exception while verifying: wrong number of arguments (given 2, expected 1) (ArgumentError) in /home/travis/.rvm/gems/ruby-head/cache/bundler-1.17.3.gem
The command "gem install bundler -v 1.17.3" failed and exited with 1 during .
Assume the following context definition:
c.contexts[:mysql] = {
tries: 10,
multiplier: 2.5,
on: Mysql::DeadlockException
}
When using the context and providing an on
parameter, I first assumed it would be added to the list of exceptions handled by the context. Instead, it overwrites:
Retriable.with_context(:mysql, on: SomeOtherException) do
# This will only retry on `SomeOtherException`!
end
At the very least, it might warrant a callout in the README.
This is the only parameter I'd expect to behave in an additive way. All of the others make sense as overrides to me.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.