tdr-autosync / mi-lib-django_cronman Goto Github PK

• do not require raven to be installed
• use SENTRY_CONFIG or RAVEN_CONFIG, with fallback to {}
• allow to enable sentry via CRONMAN_SENTRY_ENABLED (False by default)

This may be or seem obvious, but nowhere in the documentation is it mentioned that in order to have the manager commands available you need to add 'cronman' to INSTALLED_APPS in settings.py. The only search hit for INSTALLED_APPS is in settings.py under tests, and sure enough. For the sake of completeness the README should include this probably right before "Define a new cron job". I can tell you that a developer I work with was stymied by this. Thanks.

There is a setting for CRONMAN_JOBS_MODULE and I'm not sure why. The reason is that it is only respected by one part of the system, cron_scheduler. If you create a module called, let's say, cron_job and put that in settings, the cron_scheduler command will find it but the cron_worker command won't. On the other hand, cron_worker will find it regardless of this setting if it is in a package or module under the app which is exactly named cron_jobs. However, the scheduler won't find it if it does not agree with the setting. It seems that the only way a job is found by the cron_worker command is through the autodiscover method on the package cronman, which runs autodiscover_modules("cron_jobs", register_to=cron_job_registry). It took me a long time to work this out because I'd named my module cron_job instead of cron_jobs. I should also point out that after installing another couple dependencies I got all of the bundled tests working, so I think there's a gap in coverage there.

Edit: I should make an effort to be more helpful by saying that I'm using the lastest release version 2.1.0 running on Ubuntu 20.04 LTS with python 3.8.2 and django 3.0.8.

Suggestions how to achieve this are welcome.

Idea from this discussion:

Maybe we could isolate core functionality from extra (Cronitor, Slack, Sentry) providing optionally plugged add-ons (plug-ins). Applying Django signal system would be an option https://docs.djangoproject.com/en/2.1/topics/signals/#defining-and-sending-signals

• do not require redis to be installed
• allow to override get_strict_redis function via settings
• use CRONMAN_ prefixed settings for default redis connection parameters

(title says it all)

• allow switching CronTask model via CRONMAN_TASK_MODEL ("<app>.<model>" format, defaults to cronman.CronTask)
• no models/migrations magic required for now, cronman.CronTask model may be created when different model is set in the settings.

i have a cron_job.py one task
too try to create a new dir
with name cron_job y create this .py but doesn't work when i type python manage.py cron_scheduler run

• use CRONMAN settings prefix for all app-specific settings

NOTE: leave cron_ command prefix - it will be fixed later in 1.0

• app-specific settings should not be required, default values should be used at code level when settings are missing

Currently, the order of execution is LIFO, which means that if a user creates a task and within the 5 minutes frame another user creates another task the newest one is executed first. That is problematic as multiple tasks can be created by multiple users making older ones wait for hours.

In the image below the cron marked in blue was created first (11:58AM) that the one marked with red (12:56 PM). But the red one was started first at 2:48 PM and then the blue at 2:56 PM. Eight minutes later.

In Django 3.0 the django.utils.encoding.force_text was deprecated. Based on this StackOverflow answer it looks like force_str should be used in it's place.

Example of the error given below.

Traceback (most recent call last):
  File "/Users/ryan/PycharmProjects/core/manage.py", line 22, in <module>
    main()
  File "/Users/ryan/PycharmProjects/core/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 425, in execute_from_command_line
    utility.execute()
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute
    django.setup()
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/apps/registry.py", line 122, in populate
    app_config.ready()
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/contrib/admin/apps.py", line 27, in ready
    self.module.autodiscover()
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/contrib/admin/__init__.py", line 24, in autodiscover
    autodiscover_modules('admin', register_to=site)
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/utils/module_loading.py", line 57, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/cronman/admin.py", line 10, in <module>
    from cronman.forms import CronTaskAdminForm
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/cronman/forms.py", line 13, in <module>
    from cronman.utils import cron_jobs_module_config, parse_params
  File "/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/cronman/utils.py", line 16, in <module>
    from django.utils.encoding import force_text
ImportError: cannot import name 'force_text' from 'django.utils.encoding' (/Users/ryan/PycharmProjects/core/venv/lib/python3.9/site-packages/django/utils/encoding.py)

Steps to replicate:

1. pip install django-cronman
2. add cronman to INSTALLED_APPS list in settings.py
3. run python manage.py migrate
4. Get the error as described above

• do not register CronTask in the Admin when CRONMAN_ADMIN_SITE is set to None explicitly

fix issue:

cronman.CronTask.user: (fields.E301) Field defines a relation with the model 'auth.User', which has been swapped out.