Git Product home page Git Product logo

bensi94 / django-requests-tracker Goto Github PK

View Code? Open in Web Editor NEW
34.0 2.0 0.0 241 KB

The Django Requests Tracker is designed for local development, particularly for Rest API development. It provides various debugging information, such as SQL queries, headers, and status codes.

License: MIT License

Python 78.90% Sass 2.78% HTML 18.16% Dockerfile 0.16%
api bulma debug django django-ninja django-rest-framework htmx middleware python rest

django-requests-tracker's Introduction

Django Requests Tracker

Test & Quality Coverage Package version Supported Python versions

A convenient Django development tool based on the great Django Debug Toolbar but aimed towards rest API development. It collects and displays information on requests, responses, SQL queries, headers, Django settings and more.

Table of contents

  1. Features
    1. Requests list
    2. Request details
  2. The example Project
  3. Installation
    1. Install the package
    2. Configure project settings
    3. Configure URLs
    4. Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django
  4. Configuration
    1. IGNORE_SQL_PATTERNS
    2. IGNORE_PATHS_PATTERNS
    3. ENABLE_STACKTRACES
    4. HIDE_IN_STACKTRACES
    5. SQL_WARNING_THRESHOLD
    6. TRACK_SQL

Features

Requests list

Django Requests Tracker registers every request sent to your Django application and displays them in a tidy list. Each element in the list contains information about the request's HTTP method, path, Django view, status code, database information and query count and execution time and duration.

The requests list can be:

  • Searched by path, Django view, sql and headers. The search is quite simple and a request is only filtered from the list if the search term does not exist in any of theses elements.
  • Ordered in ascending and descending order by time, duration, Django view, query count, similar query count and duplicate query count.
  • Auto-refreshed so that new requests will automatically show up in the list.
  • Manually refreshed.
  • Cleared.

The requests list in action ๐ŸŽฅ

requests-list

Request details

Each list element can be clicked where further information collected about the request such as SQL queries and headers can be found.

SQL queries

In request details, every SQL query executed in the context of the Django request should be shown, along with the execution time and a timeline bar that shows how big a chunk of the total time belongs to the given query. A stacktrace is shown for each query that helps with finding the origin of it.

Some queries are labelled with a tag X similar queries or X duplicate queries this can often indicate a problem and can be very handy when debugging or in development.

  • Similar Queries means that the same query is executed more than once but with different parameters. This can for example happen when iterating over a list of IDs and fetching one item by ID at a time.
  • Duplicate Queries means that the exact same query with the same parameters is executed more than once. This can for example happen when iterating over a list child items and fetching same parent multiple times. Also known as an N-plus-1 query which is quite common problem with ORMs.

The request details view in action ๐ŸŽฅ

request-details

Django Settings

Django settings very often contain some logic, and usage of environment variables and can even be spread out over multiple files. So it can be very beneficial to be able to see the current computed settings being used in the running process. Django Requests Tracker offers a simple way to view this. The view can be accessed by clicking on Django settings in the right corner of the requests tracker view.

All information determined to be sensitive, such as keys and passwords, are masked in the displayed settings.

Screenshot 2023-01-31 at 00 24 32

The Example Project

This repository includes an example project to try out the package and see how it works. It can also be a great reference when adding the package to your project. To try it out, clone this project and follow the instructions on the example project README

Installation

If any of the following steps are unclear, check out the Example Project for reference.

Install the package

pip install requests-tracker

or install with you're chosen package tool, e.g. poetry, pipenv, etc.

Configure project settings

Settings prerequisites

First, ensure that django.contrib.staticfiles is in your INSTALLED_APPS setting and configured properly:

INSTALLED_APPS = [
    # ...
    "django.contrib.staticfiles",
    # ...
]

STATIC_URL = "static/"

Second, ensure that your TEMPLATES setting contains a DjangoTemplates backend whose APP_DIRS options is set to True:

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "APP_DIRS": True,
        # ...
    }
]

Install the app, add middleware and configure internal ips

  • Add requests_tracker to your INSTALLED_APPS setting.
  • Add requests_tracker.middleware.requests_tracker_middleware to your MIDDLEWARE setting.
  • Add your internal IP addresses to INTERNAL_IPS setting.
if DEBUG:
    INSTALLED_APPS += ["requests_tracker"]
    MIDDLEWARE += ["requests_tracker.middleware.requests_tracker_middleware"]
    INTERNAL_IPS = ["127.0.0.1"]

โš ๏ธ If using Docker the following will set your INTERNAL_IPS correctly in Debug mode:

if DEBUG:
    import socket  # only if you haven't already imported this
    hostname, _, ips = socket.gethostbyname_ex(socket.gethostname())
    INTERNAL_IPS = [ip[: ip.rfind(".")] + ".1" for ip in ips] + ["127.0.0.1", "10.0.2.2"]

๐Ÿšจ ๏ธย  It's recommended to only configure these settings in DEBUG mode. Even though Django Requests Tracker will only track requests in DEBUG mode it's still a good practice to only have it installed in DEBUG mode.

Configure URLs

Add Django Requests Tracker URLs to your project's URLconf:

if settings.DEBUG:
    urlpatterns += [path("__requests_tracker__/", include("requests_tracker.urls"))]

๐Ÿšจ๏ธย  Again it's recommended to only add the URLs in DEBUG mode.

Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django

Add static root to settings

# ๐Ÿšจ Your project might not include BASE_DIR setting but likely some variation of it ๐Ÿšจ
BASE_DIR = Path(__file__).resolve().parent.parent

STATIC_ROOT = os.path.join(BASE_DIR, "static")

Add static root URLs to your project's URLconf:

urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

Collect static files

python manage.py collectstatic

Configuration

Django Requests Tracker provides a few very simple settings. The settings are applied by setting REQUESTS_TRACKER_CONFIG setting in your settings.py file. REQUESTS_TRACKER_CONFIG takes a dictonary. Example:

# settings.py

REQUESTS_TRACKER_CONFIG = {
    "IGNORE_PATHS_PATTERNS": (".*/api/keep-alive.*",),
    "ENABLE_STACKTRACES": False",
}

IGNORE_SQL_PATTERNS

Takes a tuple of strings. Each string is a regular expression pattern. If a SQL query matches any of the patterns it will be ignored and not shown in the requests list or request details.

Default: ()

Example:

REQUESTS_TRACKER_CONFIG = {
    "IGNORE_SQL_PATTERNS": (
        r"^SELECT .* FROM django_migrations WHERE app = 'requests_tracker'",
        r"^SELECT .* FROM django_migrations WHERE app = 'auth'",
    ),
}

IGNORE_PATHS_PATTERNS

Takes a tuple of strings. Each string is a regular expression pattern. If a request path matches any of the patterns it will be ignored and not tracked.

Default: ()

Example:

REQUESTS_TRACKER_CONFIG = {
    "IGNORE_PATHS_PATTERNS": (
        r".*/api/keep-alive.*",
    ),
}

SQL_WARNING_THRESHOLD

Represents the threshold in milliseconds after which a SQL query is considered slow and will be marked with a warning label in the SQL list.

Default: 500 (500 milliseconds)

Example:

REQUESTS_TRACKER_CONFIG = {
    "SQL_WARNING_THRESHOLD": 50,
}

ENABLE_STACKTRACES

If set to False stacktraces will not be shown in the request details view.

Default: True

HIDE_IN_STACKTRACES

Takes a tuple of strings. Each string represents a module name. If a module name is found in a stacktrace that part of the stacktrace will be hidden.

Default:

(
     "socketserver",
     "threading",
     "wsgiref",
     "requests_tracker",
     "django.db",
     "django.core.handlers",
     "django.core.servers",
     "django.utils.decorators",
     "django.utils.deprecation",
     "django.utils.functional",
)

TRACK_SQL

If set to False SQL queries will not be tracked.

Default: True

django-requests-tracker's People

Contributors

bensi94 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

django-requests-tracker's Issues

`Django-Request-Tracker` breaks on `psycopg3`

Hey!

We are trying to switch to psycopg 3.1.10 using django-ninja 0.22.2, Django 4.2.4, requests-tracker 0.3.2. Unfortunately, requests-tracker breaks with psycopg > 2

Stack trace:

  File "/Users/Tosuto/PycharmProjects/project/.venv/lib/python3.11/site-packages/debug_toolbar/panels/sql/tracking.py", line 177, in _record
    return method(sql, params)
           โ”‚      โ”‚    โ”” ('param',)
           โ”‚      โ”” 'SELECT "authentication_usermodel"."password", "authentication_usermodel"."last_login", "authentication_usermodel"."...
           โ”” <bound method CursorDebugWrapper.execute of <debug_toolbar.panels.sql.tracking.patch_cursor_wrapper_with_mixin.<locals>.DjDTC...
  File "/Users/Tosuto/PycharmProjects/project/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
                           โ”‚    โ”” ('param',)
                           โ”” 'SELECT "authentication_usermodel"."password", "authentication_usermodel"."last_login", "authentication_usermodel"."...
  File "/Users/Tosuto/PycharmProjects/ts-backend/.venv/lib/python3.11/site-packages/requests_tracker/sql/sql_hook.py", line 34, in execute
    return sql_tracker.record(
           โ”‚           โ”” <function SQLTracker.record at 0x104fdc040>
           โ”” <requests_tracker.sql.sql_tracker.SQLTracker object at 0x106744d90>
  File "/Users/Tosuto/PycharmProjects/project/.venv/lib/python3.11/site-packages/requests_tracker/sql/sql_tracker.py", line 239, in record
    initial_conn_status = conn.status
                          โ”” <psycopg.Connection [IDLE] (host=localhost user=user database=database) at 0x106729c90>

Any plan to support psycopg3, or should this project be pinned to support only โ‰ค 3?

Thanks in advance.

I am working on a awesome-python-htmx, seeking your feedback

A few of us at PyCon this year got together and brainstormed a new Web Stack that we are calling PyHAT (Python, htmx, ASGI, TailwindCSS). The first thing we set out to do is create awesome-python-htmx; a collection of active tools/libraries/projects in that space.

Your project seems like an obvious thing to include, so I did. I'd appreciate your feedback if you have any on it's inclusion.

In addition to that, if you could also participate in PyHAT-stack/awesome-python-htmx#1 that would be greatly appreciated! I'd be particularly interested to hear how your experience shipping something using Django and HTMX was.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.