Django application for the DDP platform's management backend. Exposes API endpoints for the management frontend to communicate with, for the purposes of
- Onboarding an NGO client
- Adding users from the client-organization
- Creating a client's workspace in our Airbyte installation
- Configuring that workspace i.e. setting up sources, destinations and connections
- Configuring data ingest jobs in our Prefect setup
- Connecting to the client's dbt GitHub repository
- Configuring dbt run jobs in our Prefect setup
- REST conventions are being followed.
- CRUD end points for a User resource would look like:
- GET /api/users/
- GET /api/users/user_id
- POST /api/users/
- PUT /api/users/:user_id
- DELETE /api/users/:user_id
- Route parameteres should be named in snake_case as shown above.
Pep8has been used to standardized variable names, classes, module names etc.Pylintis the linting tool used to analyze the code as per Pep8 style.Blackis used as the code formatter.
- Recommended IDE is VsCode.
- Install the pylint extension in vscode and enable it.
- Set the default format provider in vscode as
black - Update the vscode settings.json as follows
{ "editor.defaultFormatter": null, "python.linting.enabled": true, "python.formatting.provider": "black", "editor.formatOnSave": true }
- In your virtual environment run
pylint ddpui/
- In your virtual environment run:
celery -A ddpui worker -n ddpui - For windows run:
celery -A ddpui worker -n ddpui -P solo - To start celery beat run:
celery -A ddpui beat
-
pyenv local 3.10 -
pyenv exec python -m venv venv -
source venv/bin/activate -
pip install --upgrade pip -
pip install -r requirements.txt
- create
.envfrom.env.template
-
create a SQL database and populate its credentials into
.env -
You can use a postgresql docker image for local development
docker run --name postgres-db -e POSTGRES_PASSWORD=<password> -p 5432:5432 -d <db name>
- Add the environment variable to .env
DBNAME=<db name>
DBHOST=localhost
DBPORT=5432
DBUSER=postgres
DBPASSWORD=<password>
DBADMINUSER=postgres
DBADMINPASSWORD=<password>
- Open a new terminal
- Start Airbyte and populate connection info in
.env
AIRBYTE_SERVER_HOST=
AIRBYTE_SERVER_PORT=
AIRBYTE_SERVER_APIVER=
AIRBYTE_API_TOKEN= <token> # base64 encryption of username:password. Default username and password is airbyte:password and token will be YWlyYnl0ZTpwYXNzd29yZA==
AIRBYTE_DESTINATION_TYPES=
- Start Prefect Proxy and populate connection info in
.env
PREFECT_PROXY_API_URL=
- Set
DEV_SECRETS_DIRin.envunless you want to use Amazon's Secrets Manager
-
Open a new terminal
-
Create a local
venv, installdbtand put its location intoDBT_VENVin.env
pyenv local 3.10
pyenv exec python -m venv <env-name>
source <env-name>/bin/activate
python -m pip install \
dbt-core \
dbt-postgres \
dbt-bigquery
- Create empty directories for
CLIENTDBT_ROOT
CLIENTDBT_ROOT=
DBT_VENV=<env-name>/bin/activate
- The
SIGNUPCODEin.envis for signing up using the frontend. If you are running the frontend, set its URL inFRONTEND_URL
DJANGOSECRET=
-
Create logs folder in
ddpui -
create
whitelist.pyfrom.whitelist.template.pyin ddpui > assets folder -
Run DB migrations
python manage.py migrate -
Seed the DB
python manage.py loaddata seed/*.json -
Create the system user
python manage.py create-system-orguser -
Start the server
python manage.py runserver
- Run
python manage.py createorganduser <Org Name> <Email address>
Follow the steps below:
- Install docker
- Install docker-compose
- create
.envfrom.env.templateinside the Docker folder
- Copy the file in ddpui/assets/ to Docker/mount
If using M1-based MacBook run this before building image
export DOCKER_DEFAULT_PLATFORM=linux/amd64
docker build -f Docker/Dockerfile.main --build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') -t dalgo_backend_main_image:0.1 .This will create the main imagedocker build -f Docker/Dockerfile.dev.deploy --build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') -t dalgo_backend:0.1 .
docker-compose -f Docker/docker-compose.dev.yml up
![deepsource-autofix[bot] avatar](https://avatars.githubusercontent.com/in/57168?v=4 "deepsource-autofix[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent