django / django-docker-box Goto Github PK

This project forked from orf/django-docker-box

Run the Django test suite across all supported databases and python versions

Dockerfile 10.03% Shell 3.28% Python 86.69%

django-docker-box's Introduction

Django

Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. Thanks for checking it out.

All documentation is in the "docs" directory and online at https://docs.djangoproject.com/en/stable/. If you're just getting started, here's how we recommend you read the docs:

First, read docs/intro/install.txt for instructions on installing Django.
Next, work through the tutorials in order (docs/intro/tutorial01.txt, docs/intro/tutorial02.txt, etc.).
If you want to set up an actual deployment server, read docs/howto/deployment/index.txt for instructions.
You'll probably want to read through the topical guides (in docs/topics) next; from there you can jump to the HOWTOs (in docs/howto) for specific problems, and check out the reference (docs/ref) for gory details.
See docs/README for instructions on building an HTML version of the docs.

Docs are updated rigorously. If you find any problems in the docs, or think they should be clarified in any way, please take 30 seconds to fill out a ticket here: https://code.djangoproject.com/newticket

To get more help:

Join the #django channel on irc.libera.chat. Lots of helpful people hang out there. Webchat is available.
Join the django-users mailing list, or read the archives, at https://groups.google.com/group/django-users.
Join the Django Discord community.
Join the community on the Django Forum.

To contribute to Django:

Check out https://docs.djangoproject.com/en/dev/internals/contributing/ for information about getting involved.

To run Django's test suite:

Follow the instructions in the "Unit tests" section of docs/internals/contributing/writing-code/unit-tests.txt, published online at https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/unit-tests/#running-the-unit-tests

Supporting the Development of Django

Django's development depends on your contributions.

If you depend on Django, remember to support the Django Software Foundation: https://www.djangoproject.com/fundraising/

django-docker-box's People

Contributors

Stargazers

Watchers

django-docker-box's Issues

Image not buildable due to default Python image now referring to `bullseye`

The Dockerfile in django-docker-box is not buildable in the current state, because:

It uses python:${PYTHON_VERSION}-slim which implies latest
latest has been updated from buster to bullseye as of August this year (discussion)
some of the dependencies that are apt-get installed no longer exist with the same name, causing the build to fail (libgdal, libenchant)

So we should either pin the python image to the now older buster, or pin it to bullseye and update the packages.
I will submit & propose a PR to do the latter.

Fix Maxmind GeoIP downloads

As per https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/, the GeoIP database for Maxmind won’t be available without a usage key. When this change is made the docker image will no longer build.

Missing these tests is a shame, but I cannot see how we can include the database in the image at all from now on.

Use `POSTGRES_VERSION` > 10 in .env file

Inside the README, it is mentioned that the default version for postgres is 10.0

https://github.com/django/django-docker-box/blob/66e23309e61f499ffa151ebff2e63583f8f38514/README.md#configuration

Although the environment variable passed to Dockerfile sets the value for POSTGRES_VERSION to 9.6

django-docker-box/.env

Line 2 in 66e2330

POSTGRES_VERSION=9.6

The support for postgres 9.6 was dropped in django/django@5371342 and hence the tests crash for main branch on django.

Add MariaDB GIS builds.

multiple check_framework.test_caches tests fail due to wrong working directory

runtests.py is intended to be run from the tests/ directory. The current docker-compose.yml file however runs it from the parent directory, causing a number of tests to fail due to a lack of permissions (as they are pointing at a non-existant directory).

For example:

ERROR [0.004s]: test_absolute_path (check_framework.test_caches.CheckCacheAbsolutePath)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/unittest/case.py", line 60, in testPartExecutor
    yield
  File "/usr/local/lib/python3.8/unittest/case.py", line 676, in run
    self._callTestMethod(testMethod)
  File "/usr/local/lib/python3.8/unittest/case.py", line 633, in _callTestMethod
    method()
  File "/tests/django/tests/check_framework/test_caches.py", line 132, in test_absolute_path
    self.assertEqual(check_file_based_cache_is_absolute(None), [])
  File "/tests/django/django/core/checks/caches.py", line 63, in check_file_based_cache_is_absolute
    cache = caches[alias]
  File "/tests/django/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
  File "/tests/django/django/core/cache/__init__.py", line 44, in create_connection
    return backend_cls(location, params)
  File "/tests/django/django/core/cache/backends/filebased.py", line 23, in __init__
    self._createdir()
  File "/tests/django/django/core/cache/backends/filebased.py", line 121, in _createdir
    os.makedirs(self._dir, 0o700, exist_ok=True)
  File "/usr/local/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/tests/django/cache'

For future compatibility as well, it would be better to change the way runtests.py is run.

Support different versions of SQLite (libsqlite)

Originally orf#25.

Add support for MariaDB 10.4.

We should add support for MariaDB 10.4.

Crash when running docs

$ docker-compose run --rm docs
Creating django-docker-box_docs_run ... done
sphinx-build -b spelling -n -d _build/doctrees -D language=   . _build/spelling
Running Sphinx v4.0.2
loading translations []... not available for built-in messages
Initializing Spelling Checker 7.2.1
loading pickled environment... failed
failed: __new__() missing 1 required positional argument: 'aliased'
Ignoring wiki words
Ignoring acronyms
Ignoring Python builtins
Ignoring importable module names
Ignoring contributor names
Looking for custom word list in /tests/django/docs/spelling_wordlist
Scanning contributors

Exception occurred:
  File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'git'
The full traceback has been saved in /tmp/sphinx-err-i4m5dh4k.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:159: spelling] Error 2
ERROR: 2

Test via Github actions?

I just saw that Github Actions nowadays allows docker-compose tests, which would allow us to fully run (aside from oracle and the likes) the Django testsuite in github actions. I think if we were able to get that running we'd have a really nice testsuite, what do you think?

https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/#fast-ci-cd-for-any-os-any-language-and-any-cloud and an example in https://github.com/peter-evans/docker-compose-actions-workflow

If we get this running I think we would be able to fully enable this for django itself.

Selenium tests are broken

For both Chorme and Firefox it seems.

For Firefox, I get, e.g.:

======================================================================
ERROR: test_select_all_across_pages (admin_changelist.tests.SeleniumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/unittest/suite.py", line 163, in _handleClassSetUp
    setUpClass()
  File "/tests/django/django/test/selenium.py", line 114, in setUpClass
    cls.selenium = cls.create_webdriver()
  File "/tests/django/django/test/selenium.py", line 92, in create_webdriver
    return webdriver.Remote(
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 276, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 369, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 434, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Could not start a new session. New session request timed out
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: '809a51a82548', ip: '172.18.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
Stacktrace:
    at org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue.addToQueue (LocalNewSessionQueue.java:179)
    at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.lambda$new$0 (NewSessionQueue.java:68)
    at org.openqa.selenium.remote.http.Route$TemplatizedRoute.handle (Route.java:192)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.remote.http.Route$CombinedRoute.handle (Route.java:336)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.execute (NewSessionQueue.java:120)
    at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute (SpanWrappedHttpHandler.java:86)
    at org.openqa.selenium.remote.http.Filter$1.execute (Filter.java:64)
    at org.openqa.selenium.remote.http.Route$CombinedRoute.handle (Route.java:336)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.grid.router.Router.execute (Router.java:91)
    at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0 (EnsureSpecCompliantResponseHeaders.java:34)
    at org.openqa.selenium.remote.http.Filter$1.execute (Filter.java:64)
    at org.openqa.selenium.remote.http.Route$CombinedRoute.handle (Route.java:336)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.remote.http.Route$NestedRoute.handle (Route.java:270)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.remote.http.Route$CombinedRoute.handle (Route.java:336)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.remote.http.Route$CombinedRoute.handle (Route.java:336)
    at org.openqa.selenium.remote.http.Route.execute (Route.java:68)
    at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0 (AddWebDriverSpecHeaders.java:35)
    at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0 (ErrorFilter.java:44)
    at org.openqa.selenium.remote.http.Filter$1.execute (Filter.java:64)
    at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0 (ErrorFilter.java:44)
    at org.openqa.selenium.remote.http.Filter$1.execute (Filter.java:64)
    at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0 (SeleniumHandler.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)

And Chrome:

======================================================================
ERROR: test_select_multiple (admin_views.test_autocomplete_view.SeleniumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/unittest/suite.py", line 163, in _handleClassSetUp
    setUpClass()
  File "/tests/django/django/test/selenium.py", line 114, in setUpClass
    cls.selenium = cls.create_webdriver()
  File "/tests/django/django/test/selenium.py", line 92, in create_webdriver
    return webdriver.Remote(
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 276, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 369, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 434, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: crashed.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: 'd33366264b2d', ip: '172.18.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: 'd33366264b2d', ip: '172.18.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
Stacktrace:
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:200)
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:68)
    at org.openqa.selenium.grid.node.local.SessionSlot.apply (SessionSlot.java:145)
    at org.openqa.selenium.grid.node.local.LocalNode.newSession (LocalNode.java:362)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession (LocalDistributor.java:639)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession (LocalDistributor.java:558)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest (LocalDistributor.java:812)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1 (LocalDistributor.java:773)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)

PermissionError: [Errno 13] Permission denied: '/tests/django/cache'

Some of the tests (not all of them) are failed with this error


ERROR [0.005s]: test_absolute_path (check_framework.test_caches.CheckCacheAbsolutePath)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/local/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/tests/django/tests/check_framework/test_caches.py", line 103, in test_absolute_path
    self.assertEqual(check_file_based_cache_is_absolute(None), [])
  File "/tests/django/django/core/checks/caches.py", line 62, in check_file_based_cache_is_absolute
    cache = caches[alias]
  File "/tests/django/django/core/cache/__init__.py", line 79, in __getitem__
    cache = _create_cache(alias)
  File "/tests/django/django/core/cache/__init__.py", line 54, in _create_cache
    return backend_cls(location, params)
  File "/tests/django/django/core/cache/backends/filebased.py", line 23, in __init__
    self._createdir()
  File "/tests/django/django/core/cache/backends/filebased.py", line 121, in _createdir
    os.makedirs(self._dir, 0o700, exist_ok=True)
  File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/tests/django/cache'

Oracle setup seems to be broken.

It looks like referenced image is not publicly available anymore

docker-compose run oracle-db
Pulling oracle-db (sath89/oracle-12c:)...
ERROR: pull access denied for sath89/oracle-12c, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

https://hub.docker.com/r/sath89/oracle-12c/ is a 404

Test suite failing under PostgresSQL due to missing collation

I'm not able to run the test suite under PostgreSQL with POSTGRES_VERSION=9.6, ~~but they seem to fail with newer versions as well.~~

I might be missing something here but it could also be something introduced by django/django#13207.

https://github.com/django/django/pull/13207/files#diff-9385ef62a9e10556c6768c66e8f6e2ebR62

$ docker-compose run --rm postgres
Creating network "django-docker-box_default" with the default driver
Creating volume "django-docker-box_oracle" with default driver
Creating django-docker-box_memcached2_1  ... done
Creating django-docker-box_postgres-db_1 ... done
Creating django-docker-box_memcached_1   ... done
Creating django-docker-box_postgres_run  ... done
wait-for-it.sh: waiting 20 seconds for postgres-db:5432
wait-for-it.sh: postgres-db:5432 is available after 3 seconds
Testing against Django installed in '/tests/django/django' with up to 6 processes
Creating test database for alias 'default'...
Traceback (most recent call last):
  File "/tests/django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.UndefinedObject: collation "sv-x-icu" for encoding "UTF8" does not exist
LINE 1: ...al NOT NULL PRIMARY KEY, "char_field" varchar(10) COLLATE "s...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "tests/runtests.py", line 589, in <module>
    options.timing,
  File "tests/runtests.py", line 320, in django_tests
    extra_tests=extra_tests,
  File "/tests/django/django/test/runner.py", line 717, in run_tests
    old_config = self.setup_databases(aliases=databases)
  File "/tests/django/django/test/runner.py", line 637, in setup_databases
    debug_sql=self.debug_sql, parallel=self.parallel, **kwargs
  File "/tests/django/django/test/utils.py", line 183, in setup_databases
    serialize=connection.settings_dict['TEST'].get('SERIALIZE', True),
  File "/tests/django/django/db/backends/base/creation.py", line 77, in create_test_db
    run_syncdb=True,
  File "/tests/django/django/core/management/__init__.py", line 172, in call_command
    return command.execute(*args, **defaults)
  File "/tests/django/django/core/management/base.py", line 394, in execute
    output = self.handle(*args, **options)
  File "/tests/django/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/tests/django/django/core/management/commands/migrate.py", line 214, in handle
    self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/tests/django/django/core/management/commands/migrate.py", line 349, in sync_apps
    editor.create_model(model)
  File "/tests/django/django/db/backends/base/schema.py", line 329, in create_model
    self.execute(sql, params or None)
  File "/tests/django/django/db/backends/base/schema.py", line 143, in execute
    cursor.execute(sql, params)
  File "/tests/django/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/tests/django/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/tests/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/tests/django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
django.db.utils.ProgrammingError: collation "sv-x-icu" for encoding "UTF8" does not exist
LINE 1: ...al NOT NULL PRIMARY KEY, "char_field" varchar(10) COLLATE "s...

django / django-docker-box Goto Github PK

django-docker-box's Introduction

Django

Supporting the Development of Django

django-docker-box's People

Contributors

Stargazers

Watchers

Forkers

django-docker-box's Issues

Recommend Projects

Recommend Topics

Recommend Org