Git Product home page Git Product logo

django-db-mutex's People

Contributors

akolpakov avatar benthuffine avatar geophphrie avatar jaredlewis avatar joshmarlow avatar justinsg avatar somewes avatar swans-one avatar wesleykendall avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

wesleykendall swans-one sodacard robdmc ttokic genehallman minervaproject pyman jaredlewis joshmarlow somewes pombredanne nephila jherencia akolpakov maxnaude optimal-outsource thinkt4nk digital-assets-data aimagelab-zip bellyfat warpengineer kkiatz hamzabenkebil

django-db-mutex's Issues

`psycopg2.errors.DeadlockDetected` error on PostgreSQL

On a production environment where Django was deployed via Apache and mod_wsgi in 8 processes, we encountered a "deadlock detected" error during this statement:

Mar 14 12:43:53 ERROR   django.request - 500 Internal Server Error: /region/pages/de/14907/edit/
Traceback (most recent call last):
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DeadlockDetected: deadlock detected
DETAIL:  Process 1054249 waits for ShareLock on transaction 18359673; blocked by process 1054250.
Process 1054250 waits for ShareLock on transaction 18359675; blocked by process 1054249.
HINT:  See server log for query details.
CONTEXT:  while inserting index tuple (0,11) in relation "db_mutex_dbmutex_lock_id_key"


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/views/pages/page_form_view.py", line 324, in post
    page_translation_form.instance.page = page_form.save()
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/forms/custom_model_form.py", line 133, in save
    return super().save(commit=commit)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/treebeard/forms.py", line 153, in save
    self.instance.move(reference_node, pos=position_type)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/models/pages/page.py", line 327, in move
    super().move(target, pos)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/models/abstract_tree_node.py", line 270, in move
    with db_mutex(self.__class__.__name__):
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/db_mutex/db_mutex.py", line 93, in __enter__
    self.start()
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/db_mutex/db_mutex.py", line 107, in start
    self.lock = DBMutex.objects.create(lock_id=self.lock_id)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/cacheops/transaction.py", line 98, in execute
    result = self._no_monkey.execute(self, sql, params)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: deadlock detected

Implement waiting lock

In some cases, I'd want to have a lock that does not immediately throw an exception, but that waits until the resource is released.
Probably this would need another timeout setting to configure how long the thread waits for the lock (and only raises an exception when the lock is not released within that time frame).
In its most basic form, I guess this would just be a busy wait.
I can also think of scenarios where this might be interesting in an asynchronous context, but this would probably require a rewrite of substantial parts of the library, due to the normal database access in Django being synchronized?

Do you agree with this idea, or do you think it's up to the application to wait and retry the corresponding task in case an exception occurs?

DateTimeField received a naive datetime

Is there a more universal solution that uses timezone.now instead of datetime.utcnow? Right now I'm getting a bunch of runtimewarnings because the datetime isn't timezone aware, but I realize that timezone.now isn't necessarily UTC. If someone has USE_TZ = False, I'm sure there would be problems during savings time changes.

RuntimeWarning: DateTimeField DBMutex.creation_time received a naive datetime (2015-03-24 15:50:28.141220) while time zone support is active.```

Update changelog

Could you please provide updated changelog? There is a release notes section in the docs but it ends with v1.1.0 while the latest released version is 3.0.0. Thank you.

Unique CharField with max_length > 255 not allowed by MySQL 

$ ./manage.py migrate
SystemCheckError: System check identified some issues:

ERRORS:
db_mutex.DBMutex.lock_id: (mysql.E001) MySQL does not allow unique CharFields to have a max_length > 255.

This error is fixed by changing max_length to 255 (instead of 256) on the lock_id field in models.py

Configuration steps

Would be useful to have a few words about configuration somewhere in the docs.
Namely, that one should add 'db_mutex' to INSTALLED_APPS and then run migrations with
python manage.py migrate.

Lock timeout exceptions may not always be raised correctly

https://github.com/ambitioninc/django-db-mutex/blob/develop/db_mutex/db_mutex.py#L112

If another process grabbed a timed-out lock and hasn't released it yet, the simple check for "if not DBMutex.objects.filter(id=self.lock.id).exists()" doesn't verify current ownership of the lock, and so it may not raise the timeout exception when it should.

It would be best if the local object and SQL object contained an owner identifier that was based on a unique machine+process+thread ID.

Then this code could double-check that the lock exists and is still owned by itself as expected.

Alternatively, it might be good-enough to just generate a random 64-bit integer to use for this purpose (would work in 99.9999% of cases).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.