imagemarkup / stade Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 1.0 7.05 MB

ISIC Challenge submission platform.

Home Page: https://challenge.isic-archive.com/

License: Apache License 2.0

Dockerfile 0.18% Python 28.22% CSS 7.41% HTML 63.83% Shell 0.32% Procfile 0.04%

isic isic-challenge isic-archive skin-cancer

stade's Introduction

Stade

ISIC Challenge submission platform.

https://challenge.isic-archive.com

Develop with Docker (recommended quickstart)

This is the simplest configuration for developers to start with.

Initial Setup

Run docker-compose run --rm django ./manage.py migrate
Run docker-compose run --rm django ./manage.py createsuperuser and follow the prompts to create your own user

Run Application

Run docker-compose up
Access the site, starting at http://localhost:8000/admin/
When finished, use Ctrl+C

Application Maintenance

Occasionally, new package dependencies or schema changes will necessitate maintenance. To non-destructively update your development stack at any time:

Run docker-compose pull
Run docker-compose build --pull --no-cache
Run docker-compose run --rm django ./manage.py migrate

Develop Natively (advanced)

This configuration still uses Docker to run attached services in the background, but allows developers to run Python code on their native system.

Initial Setup

Run docker-compose -f ./docker-compose.yml up -d
Install Python 3.12
Install psycopg2 build prerequisites
Create and activate a new Python virtualenv
Run pip install -e .[dev]
Run source ./dev/export-env.sh
Run ./manage.py migrate
Run ./manage.py createsuperuser and follow the prompts to create your own user

Run Application

Ensure docker-compose -f ./docker-compose.yml up -d is still active
Run:
1. source ./dev/export-env.sh
2. ./manage.py runserver
Run in a separate terminal:
1. source ./dev/export-env.sh
2. celery --app stade.celery worker --loglevel INFO --without-heartbeat
When finished, run docker-compose stop

Remap Service Ports (optional)

Attached services may be exposed to the host system via alternative ports. Developers who work on multiple software projects concurrently may find this helpful to avoid port conflicts.

To do so, before running any docker-compose commands, set any of the environment variables:

DOCKER_POSTGRES_PORT
DOCKER_RABBITMQ_PORT
DOCKER_MINIO_PORT

The Django server must be informed about the changes:

When running the "Develop with Docker" configuration, override the environment variables:
- DJANGO_MINIO_STORAGE_MEDIA_URL, using the port from DOCKER_MINIO_PORT.
When running the "Develop Natively" configuration, override the environment variables:
- DJANGO_DATABASE_URL, using the port from DOCKER_POSTGRES_PORT
- DJANGO_CELERY_BROKER_URL, using the port from DOCKER_RABBITMQ_PORT
- DJANGO_MINIO_STORAGE_ENDPOINT, using the port from DOCKER_MINIO_PORT

Since most of Django's environment variables contain additional content, use the values from the appropriate dev/.env.docker-compose* file as a baseline for overrides.

Testing

Initial Setup

tox is used to execute all tests. tox is installed automatically with the dev package extra.

When running the "Develop with Docker" configuration, all tox commands must be run as docker-compose run --rm django tox; extra arguments may also be appended to this form.

Running Tests

Run tox to launch the full test suite.

Individual test environments may be selectively run. This also allows additional options to be be added. Useful sub-commands include:

tox -e lint: Run only the style checks
tox -e type: Run only the type checks
tox -e test: Run only the pytest-driven tests

To automatically reformat all code to comply with some (but not all) of the style checks, run tox -e format.

stade's People

Contributors

Stargazers

Watchers

Forkers

aashish24

stade's Issues

Use Joist in place of CollisionSafeFileField

https://github.com/danlamanna/joist

SystemExit: 1

Sentry Issue: STADE-1N

SystemExit: 1
(34 additional frame(s) were not displayed)
...
  File "gunicorn/http/body.py", line 212, in read
    data = self.reader.read(1024)
  File "gunicorn/http/body.py", line 123, in read
    data = self.unreader.read()
  File "gunicorn/http/unreader.py", line 38, in read
    d = self.chunk()
  File "gunicorn/http/unreader.py", line 65, in chunk
    return self.sock.recv(self.mxchunk)
  File "gunicorn/workers/base.py", line 196, in handle_abort
    sys.exit(1)

Investigate django-migration-linter

https://github.com/3YOURMIND/django-migration-linter

Allow non-users to subscribe to updates

We should have a form to allow non-logged-in users to submit their email address.

Put successful_approaches into model as manager + queryset.

Put into model as manager + queryset.

Originally posted by @brianhelba in #79

HTTPClientError: An HTTP Client raised and unhandled exception: SoftTimeLimitExceeded()

Sentry Issue: STADE-23

SoftTimeLimitExceeded: SoftTimeLimitExceeded()
(5 additional frame(s) were not displayed)
...
  File "urllib3/util/ssl_.py", line 355, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "ssl.py", line 412, in wrap_socket
    session=session
  File "ssl.py", line 853, in _create
    self.do_handshake()
  File "ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
  File "billiard/pool.py", line 227, in soft_timeout_sighandler
    raise SoftTimeLimitExceeded()

HTTPClientError: An HTTP Client raised and unhandled exception: SoftTimeLimitExceeded()
(20 additional frame(s) were not displayed)
...
  File "botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "botocore/endpoint.py", line 200, in _do_get_response
    http_response = self._send(request)
  File "botocore/endpoint.py", line 244, in _send
    return self.http_session.send(request)
  File "botocore/httpsession.py", line 298, in send
    raise HTTPClientError(error=e)

internal error scoring submission XXXX

See this discussion on configuring boto timeouts/retries: jschneier/django-storages#279 (comment).

Clean up task styles on front page

KeyError: 'name' when submitting create-team form without a name

Sentry Issue: STADE-3D

KeyError: 'name'
(5 additional frame(s) were not displayed)
...
  File "django/forms/forms.py", line 185, in is_valid
    return self.is_bound and not self.errors
  File "django/forms/forms.py", line 180, in errors
    self.full_clean()
  File "django/forms/forms.py", line 382, in full_clean
    self._clean_form()
  File "django/forms/forms.py", line 409, in _clean_form
    cleaned_data = self.clean()
  File "core/forms.py", line 147, in clean
    if Team.objects.filter(name=self.cleaned_data['name'], challenge=challenge).exists():

Add landing page

Menubar:

Home (What is the ISIC Challenge)
Challenges (Index to various /challenge/landing pages)
Data
ISIC Live
Submit
Leaderboards

Add URL slugs to challenge and task fields

These will be used to resolve landing page URLs.

Run system checks on deploy

Add submenu for challenge landing pages

Add a more sophisticated ordering to Tasks

The ordering by id, added in #135, may not always be correct.

order_with_respect_to may be appropriate for this.

Add support for Google OAuth

On the task_detail page, the "New Submission" button shouldn't always be enabled

If the Challenge for that Task is locked, then the "New Submission" button links to create_team, which will 404.

ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Sentry Issue: STADE-4C

ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
(6 additional frame(s) were not displayed)
...
  File "s3transfer/futures.py", line 265, in result
    raise self._exception
  File "s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "s3transfer/download.py", line 345, in _submit
    **transfer_future.meta.call_args.extra_args
  File "botocore/client.py", line 272, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "botocore/client.py", line 576, in _make_api_call
    raise error_class(parsed_response, operation_name)

internal error scoring submission XXXX: An error occurred (403) when calling the HeadObject operation: Forbidden

This happens very rarely when scoring submissions despite the IAM policy not changing.

Add manuscript statistics through AWS Textract

Add a section with the Challenge 2020 training data to the data page

Also, consider issuing / mentioning a DOI for the data.

Race condition when creating approaches

There could potentially be more approaches than the MAX_APPROACHES setting if submitting forms at the same time.

Use unique constraints in favor of custom form validation

This is fine for right now, but we should be adding unique constraints to the model layer. See https://docs.djangoproject.com/en/2.2/ref/models/constraints/#django.db.models.UniqueConstraint.

Originally posted by @danlamanna in #25

Add trailing slashes to all URLs

https://docs.djangoproject.com/en/3.0/ref/urls/#path

https://docs.djangoproject.com/en/3.0/ref/settings/#append-slash

IntegrityError: duplicate key value violates unique constraint "core_team_challenge_id_name_98c91bb9_uniq"

Sentry Issue: STADE-1K

UniqueViolation: duplicate key value violates unique constraint "core_team_challenge_id_name_98c91bb9_uniq"
DETAIL:  Key (challenge_id, name)=(38, DAISYLabs) already exists.

  File "django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)

IntegrityError: duplicate key value violates unique constraint "core_team_challenge_id_name_98c91bb9_uniq"
DETAIL:  Key (challenge_id, name)=(38, DAISYLabs) already exists.

(14 additional frame(s) were not displayed)
...
  File "django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)

ERROR: build-system section missing inside /home/dan/p/stade/pyproject.toml

The error disappears when I remove the isolated_build line from tox.

imagemarkup / stade Goto Github PK

stade's Introduction

Stade

Develop with Docker (recommended quickstart)

Initial Setup

Run Application

Application Maintenance

Develop Natively (advanced)

Initial Setup

Run Application

Remap Service Ports (optional)

Testing

Initial Setup

Running Tests

stade's People

Contributors

Stargazers

Watchers

Forkers

stade's Issues

Recommend Projects

Recommend Topics

Recommend Org