inkstak / activejob-status Goto Github PK

Please, cut a release with the recent added information to the job status.

Commit 82c062b sets the progress and total on the job itself as well as setting it in the storage object.

The Status object can either be initialized with the job argument as an ActiveJob instance or with the id (string type) of an instance. This method fails if the Status is initialized with the id since strings do not have a progress attribute/setter.

For example:

job = MyJob.perform_later

status1 = ActiveJob::Status.get(job)
status1[:progress] = 1  # this succeeds because the @job instance variable refers to the job itself

status2 = ActiveJob::Status.get(job.job_id)
status2[:progress] = 1 # this fails because the @job instance variable refers to the id of the job (a string):
                       # NoMethodError Exception: undefined method `progress' for "XXXXXXXXXXXXXXXXXXXX":String

The expected outcome is that setting the progress or total values of a Status instance should work the same no matter how the status instance is initialized (ie. the old behaviour from before this commit).

Currently, it seems that we have expires_in which apply to all jobs no matter that are complete or not. If a job is complete, in most cases they maybe needed for shorted time than jobs that are in progress, scheduled, failed etc.

What do you think of an option like: completed_job_expires_in in config so that we can set it separately for completed jobs and remove them in a diff timeframe from the queue if completed.?

We could probably default it and make it optional.

The Status.progress method performs two calls to Storage.read and so two calls to the cache store.

class Status

  def read
    @storage.read(@job)
  end

  def progress
    read[:progress].to_f / read[:total].to_f
  end 

it could be returned with one single call.

The status of tasks with a long run time of more than 30 minutes is lost.

Using this code did not help:

def expiration
    @expiration ||= 60 * 60 * 24 * 30 # 30 days
end

How can I fix this?

I have a use case for a job to be searchable by the name of a job. Say MyAwesomeJob.

By that, I want to remove a job from the queue or act on it somehow.

One thought came to my mind is to restructure how we store that data in redis, maybe add it to the key or even make a nested structure to include job name too.

Other one is maybe search by the value and somehow scan for job name, not sure if it's possible with redis.

@inkstak any suggestion on this, how I can accomplish this?

Thanks

hey, I want to report that the new version breaks Ruby 3.0 support

      ArgumentError: wrong number of arguments (given 0, expected 1)
           /usr/local/rvm/gems/ruby-3.0.0-railsexpress/gems/activejob-status-0.2.0/lib/activejob-status/status.rb:22:in `update'
           /usr/local/rvm/gems/ruby-3.0.0-railsexpress/gems/activejob-status-0.2.0/lib/activejob-status.rb:17:in `block (2 levels) in <module:Status>'

ActiveJob::Status::Status#update method definition has been changed

see https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/

@inkstak ? could you take a look?

The method retry_on from ActiveJob is using rescue_from.
If a job was enqueued from a retry and failed for the last time, It will be kept as working instead of failure.

I am using the ActiveJob adapter for Sneakers, but this looks that will happen with any adapter.

I will try to work on a solution but for now I am doing this inside my Job

  def perform(params)
# DOING 
    rescue Exception => e
      status.update(status: 'failed')
      raise e
    end

I was trying to do this in a controller:

def status
  status = ActiveJob::Status.get(params[:job])
  render json: status.to_json
end

But it was not working, I've discovered I had to do status.read.to_json. I could also fix it by adding to_json to line 3 of status.rb. I think that should be added, or maybe add some documentation for ActiveStatus::Status::Status.read.

Hi there! After upgrading to version 0.4.1 I'm faced with the following error:

NoMethodError: undefined method `without' for {:expires_in=>1800, :throttle_interval=>0, :includes=>[:status]}:Hash
Did you mean?  with_options

Shouldn't the correct method be except? I'm not found references to this method without for the Hash class.

Thanks.

My code within my controller looks like this:

   include ActiveJob::Status

    job = MyJob.perform_later(response[:access_token])

    status = ActiveJob::Status.get(job.job_id)

    # Need to work out if the job has finished
    while !status.completed? do
      sleep(10)
      status = ActiveJob::Status.get(job.job_id)
      debug('status', status)
      debug('completed', status.completed?)
      debug('progress', status.progress)
    end

which gives:

** status = --- !ruby/object:ActiveJob::Status::Status
job: b6422d6f-7c98-400e-a709-9718443ce1da

** completed = false
** progress = NaN

The status never moves to completed even when the Job has run.

Within MyJob I am doing:

      progress.total = source_urls.count

and then within my loop

        progress.increment

        debug('progress', progress)

which gives:

progress = --- !ruby/object:ActiveJob::Status::Progress
job: &1 !ruby/object:GeniJob
  arguments:
  - 8VkXh0ifrm4U79Se5oUTIjPWw3tVdlbieIaRcHwr
  job_id: b6422d6f-7c98-400e-a709-9718443ce1da
  queue_name: default
  priority: 
  executions: 1
  exception_executions: {}
  timezone: Brisbane
  provider_job_id: 911bd9f17d730ae61213e8b6
  serialized_arguments: 
  locale: en
  enqueued_at: '2021-03-31T01:57:34Z'
  status: !ruby/object:ActiveJob::Status::Status
    job: *1
  progress: !ruby/object:ActiveJob::Status::Progress
    job: *1
    total: 32
    progress: 31
total: 32
progress: 31

So you can see that the progress is incrementing within the Job but not visible to the Controller.

What am I doing wrong?