jazzband / django-silk Goto Github PK

View Code? Open in Web Editor NEW

4.3K 49.0 330.0 7.16 MB

Silky smooth profiling for Django

License: MIT License

JavaScript 21.99% Python 54.08% CSS 7.37% HTML 13.11% SCSS 3.46%

django-silk's People

Stargazers

Watchers

Forkers

wavelets rosscdh rthille svisser rmoorman idonethis vandersonmota mackeian lmohring vichali joshdata joaofrancese resmio ilvar macroeyes edelvalle utkuyaman brmc gitter-badger tomako elmit forcityplatform cloudncodeinc pablasso karabijavad kkaehler bartoszhernas stripathi669 viciu nikolas rodcloutier rentapplication lmortimer msaelices florisdenhengst jvlomax davinirjr trik blag citos88 wrhector digitaldavenyc jing30 wm3ndez anthonydeniau vuchau macndesign szysteve hanleyhansen chid javaguirre 9thbit shanx leifdenby aljp mattjegan hryngo fallenflint lockie r3m0t drache91 sportstech phuong gsshakil chocobn69 viteran perdy dreadatour spoonmeiser agm jeffreyckchau lammertw harolcalzada realsby gdelnegro crunchr rjwebb smcoll hehaha erwinhaasnoot davidwallacejackson mohamed-taha showheroes imaia boilerroomtv basifat manlan2 rvignesh89 zupr-io albertyw ckshekhar mengyou658 xrmx dennybiasiolli aadu lokeshatbigbasket deankim07 yolley dzankovicirfan pikhovkin

django-silk's Issues

null value in column "view_name" violates not-null constraint

I installed silk as instructions say, but then I got exception

null value in column "view_name" violates not-null constraint

Any ideas? By adding to models "null=True" it works fine

Silky doesn't work when django.middleware.gzip.GZipMiddleware is enabled

When the 'django.middleware.gzip.GZipMiddleware' is enabled the following error happens:
File "...r/lib/python2.7/site-packages/silk/sql.py", line 39, in execute_sql
sql_query = q % params
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)

Deploying silk site-wide

I have two questions:

Is it possible to use silk site-wide, ie without having to add a decorator before each view? If yes, how do we do it?
Is it recommended to use silk in production? If any if you is already using it in production, would you care to tell me what's the performance overhead?

Thanks.

Allowing profiling outside of the request/response cycle

e.g. a profile should not have to be associated with a request, perhaps by detecting that no request is in process and then committing profile data on the fly if that's the case.

Don't pin versions in setup.py

Especially, don't pin versions like pytz==2014.2

Silk fails on binary staticfiles content

When I'm in production mode I'll let Nginx go straight to the filesystem on these things but when in debug mode I add this to my root urls.py:

if settings.DEBUG:
    urlpatterns += patterns('', url(r'^silk', include('silk.urls', namespace='silk')))
    urlpatterns += static(
        settings.MEDIA_URL,
        document_root=settings.MEDIA_ROOT
    )

Any request on something going through /cache/foo/bar.jpg will cause this traceback.

Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
    return self.application(environ, start_response)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/core/handlers/base.py", line 205, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/core/handlers/base.py", line 201, in get_response
    response = middleware_method(request, response)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/silk/middleware.py", line 183, in process_response
    silky_response.save()
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/base.py", line 545, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/base.py", line 635, in _save_table
    forced_update)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/base.py", line 679, in _do_update
    return filtered._update(values) > 0
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/query.py", line 507, in _update
    return query.get_compiler(self.db).execute_sql(None)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 975, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/silk/sql.py", line 49, in execute_sql
    return self._execute_sql(*args, **kwargs)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 781, in execute_sql
    cursor.execute(sql, params)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/backends/util.py", line 73, in execute
    sql = self.db.ops.last_executed_query(self.cursor, sql, params)
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/operations.py", line 214, in last_executed_query
    return cursor.query.decode('utf-8')
  File "/Users/peterbe/virtualenvs/dailycookie/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 82: invalid start byte

Tests broken

Since the way in which request ids are constructed changed, tests are now broken

Collapse profiles with the same name, in the same request

e.g. if the same function/block is getting profiled again and again it's not very useful just to get a huge list of each individual hit. it would be nice to have a collapse feature that allows grouping of identical profiles. This is similar to the output of cProfile's percall column. The number of individual calls to a function are profiled and percall gives us the average per call.

static files not found problem

I use uwsgi to run my django server.
In order to open silk admin panel properly the static files of silk must be copied to static folder, or i fixed this issue like that.
Is there simething I miss? or do we always need to copy the files

Switch to PyPI for managing releases

As your dependencies are being pulled from the Python Package Index, is there any reason why silk relies on Github tags for managing releases rather than PyPI?

Conform to charset flag in Content-Type header of request/response

At the moment we attempt UTF-8 decoding on responses and no decoding at all on request bodies. We should be checking for the charset flag within the Content-Type header e.g. Content-Type: text/html; charset=utf-8 and then apply that decoding. Not sure what to default to if that isn't present though, probably just attempt both ascii and UTF-8?

This is what is causing issue #19

Support binary response bodies

Currently binary response bodies such as images are ignored and cannot be displayed in the silk UI. Can probably take advantage of BinaryField to support this

No data profiled

I am not sure but for me the URL of 'summary' is /silk, so I had to change this code in middleware.py:

def _should_intercept(request):
    """we want to avoid recording any requests/sql queries etc that belong to Silky"""
    fpath = silky_reverse('summary')
    path = '/'.join(fpath.split('/')[0:-1])
    silky = request.path.startswith(path)
    ignored = request.path in SilkyConfig().SILKY_IGNORE_PATHS
    return not (silky or ignored)

To this code:

def _should_intercept(request):
    """we want to avoid recording any requests/sql queries etc that belong to Silky"""
    fpath = silky_reverse('summary')
    silky = request.path.startswith(fpath)
    ignored = request.path in SilkyConfig().SILKY_IGNORE_PATHS
    return not (silky or ignored)

Problems with `six.moves.urllib`

On a fresh new python 2.7 virtualenv I installed silk and I got this:

Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/contrib/staticfiles/handlers.py", line 72, in __call__
    return self.application(environ, start_response)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/handlers/wsgi.py", line 255, in __call__
    response = self.get_response(request)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/handlers/base.py", line 176, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/handlers/base.py", line 90, in get_response
    response = middleware_method(request)
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/middleware.py", line 70, in process_request
    if _should_intercept(request):
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/middleware.py", line 34, in _should_intercept
    fpath = silky_reverse('summary')
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/middleware.py", line 23, in silky_reverse
    r = reverse('silk:%s' % name, *args, **kwargs)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/urlresolvers.py", line 491, in reverse
    app_list = resolver.app_dict[ns]
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/urlresolvers.py", line 330, in app_dict
    self._populate()
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/urlresolvers.py", line 268, in _populate
    for pattern in reversed(self.url_patterns):
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/urlresolvers.py", line 366, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/core/urlresolvers.py", line 361, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/airmozilla/urls.py", line 38, in <module>
    urlpatterns += patterns('', url(r'^silk', include('silk.urls', namespace='silk')))
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/conf/urls/__init__.py", line 25, in include
    urlconf_module = import_module(urlconf_module)
  File "/Users/peterbe/dev/MOZILLA/AIRMOZILLA/airmozilla/vendor/lib/python/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/urls.py", line 3, in <module>
    from silk.views.profile_detail import ProfilingDetailView
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/views/profile_detail.py", line 4, in <module>
    from silk.auth import login_possibly_required, permissions_possibly_required
  File "/Users/peterbe/virtualenvs/airmozilla2.7/lib/python2.7/site-packages/silk/auth.py", line 12, in <module>
    from six.moves.urllib.parse import urlparse
ImportError: No module named urllib.parse

I tried pip install -U six which then installed six==1.7.2 but that didn't help :(

Check to see if process_request of SilkyMiddleware has been called, and issue warnings on middleware placement if not

When I use django.middleware.locale.LocaleMiddleware together with silk, I'm able to trigger the following error by going to /admin/ and /example_app/ (you need to go to both of them to trigger the error):

AttributeError: 'thread._local' object has no attribute 'temp_identifier'

Trace and settings:
https://gist.github.com/grumpi/f0867ac818b6c193fa62

Modified example app to trigger this behavior:
https://github.com/grumpi/silk/tree/localemiddleware-bug

I tested this on Django 1.7.1 and Django 1.6.5. It looks like, somehow self.local.temp_identifier isn't set up properly. Might or might not be related to the duplicate key issue.

Incorrect string value: '\xCE\xBB, \xCF\x86...' for column 'raw_body' at row 1

Happens when I visit any page after installing silk.

Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.6.5
Python Version: 2.7.6
Installed Applications:
('django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django.contrib.sitemaps',
 'observatory',
 'blog',
 'silk')
Installed Middleware:
('silk.middleware.SilkyMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.locale.LocaleMiddleware')


Traceback:
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  199.                 response = middleware_method(request, response)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/middleware.py" in process_response
  96.             self._process_response(response)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/middleware.py" in _process_response
  86.                 silk_response.save()
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in save
  545.                        force_update=force_update, update_fields=update_fields)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in save_base
  573.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in _save_table
  635.                                       forced_update)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in _do_update
  679.         return filtered._update(values) > 0
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/query.py" in _update
  510.         return query.get_compiler(self.db).execute_sql(None)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  980.         cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/sql.py" in execute_sql
  42.             return self._execute_sql(*args, **kwargs)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  786.         cursor.execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
  69.             return super(CursorDebugWrapper, self).execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
  53.                 return self.cursor.execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py" in execute
  124.             return self.cursor.execute(query, args)
File "/srv/atlas/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py" in execute
  207.         if not self._defer_warnings: self._warning_check()
File "/srv/atlas/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py" in _warning_check
  117.                     warn(w[-1], self.Warning, 3)

Exception Type: Warning at /
Exception Value: Incorrect string value: '\xCE\xBB, \xCF\x86...' for column 'raw_body' at row 1

raw_body seems to contain the page content, the generated sql looks like this:

u'UPDATE `silk_response` SET `request_id` = 4, `status_code` = 200, `raw_body` = <!doctype html>\n<html lang="en">\n<head>\n  <meta charset="utf-8">\n  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n\n  <base href="/">\n\n  <title>The Atlas of Economic Complexity | Home</title>\n  <meta name="description" content="Networks in Macro Economics with World Trade Data">\n  <link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="Atlas of Economic Complexity" />\n\n  <link rel="shortcut icon" type="image/x-icon" href="media/img/favicon.ico">\n\n  <!-- Load non-standard fonts -->\n  <link href=\'http://fonts.googleapis.com/css?family=Open+Sans+Condensed:300\' rel=\'stylesheet\' type=\'text/css\'>\n  <link href=\'http://fonts.googleapis.com/css?family=PT+Sans+Narrow\' rel=\'stylesheet\' type=\'text/css\' />\n  <link href=\'http://fonts.googleapis.com/css?family=Cardo\' rel=\'stylesheet\' type=\'text/css\' />\n  <link rel="stylesheet" href="media/css/normalize.css">\n  <link rel="stylesheet" href="media/css/style.css">\t\n  <link rel="stylesheet" href="media/js/libs/chosen/chosen.css" />\n\n  <!--[if lt IE 9]>\n    <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>\n  <![endif]-->\n\n   \n\n<style>\n\n.carousel {\n  position: relative;\n}\n\n.carousel-inner {\n  position: relative;\n  width: 100%;\n  overflow: hidden;\n}\n\n.carousel-inner > .item {\n  position: relative;\n  display: none;\n  -webkit-transition: 0.6s ease-in-out left;\n          transition: 0.6s ease-in-out left;\n}\n\n.carousel-inner > .item > img,\n.carousel-inner > .item > a > img {\n  display: block;\n  height: auto;\n  max-width: 100%;\n  line-height: 1;\n}\n\n.carousel-inner > .active,\n.carousel-inner > .next,\n.carousel-inner > .prev {\n  display: block;\n}\n\n.carousel-inner > .active {\n  left: 0;\n}\n\n.carousel-inner > .next,\n.carousel-inner > .prev {\n  position: absolute;\n  top: 0;\n  width: 100%;\n}\n\n.carousel-inner > .next {\n  left: 100%;\n}\n\n.carousel-inner > .prev {\n  left: -100%;\n}\n\n.carousel-inner > .next.left,\n.carousel-inner > .prev.right {\n  left: 0;\n}\n\n.carousel-inner > .active.left {\n  left: -100%;\n}\n\n.carousel-inner > .active.right {\n  left: 100%;\n}\n\n.carousel-control {\n  position: absolute;\n  top: 0;\n  bottom: 0;\n  left: 0;\n  width: 15%;\n  font-size: 50px;\n  color: #ffffff;\n  text-align: center;\n/*  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);\n  opacity: 0.5;\n  filter: alpha(opacity=50);\n  */\n}\n\n.carousel-control.left {\n  /*background-color:black;*/\n  opacity: 0;\n  background-repeat: repeat-x;\n  /*\n  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr=\'#80000000\', endColorstr=\'#00000000\', GradientType=1);\n  */\n}\n\n.carousel-control.right {\n  right: 0;\n  left: auto;\n  /*background-color:black;*/\n  opacity: 0;\n  background-repeat: repeat-x;\n  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr=\'#00000000\', endColorstr=\'#80000000\', GradientType=1);\n}\n\n\n.carousel-control:hover,\n.carousel-control:focus {\n  color: black;\n  text-decoration: none;\n  opacity: 0.9;\n\n  margin-top:-100px;\n  /*\n  filter: alpha(opacity=90);\n   */\n}\n\n.carousel-control .icon-prev,\n.carousel-control .icon-next,\n.carousel-control .glyphicon-chevron-left,\n.carousel-control .glyphicon-chevron-right {\n  position: absolute;\n  top: 50%;\n  left: 50%;\n  z-index: 5;\n  display: inline-block;\n}\n\n.carousel-control .icon-prev,\n.carousel-control .icon-next {\n  width: 20px;\n  height: 20px;\n  margin-top: -10px;\n  margin-left: -10px;\n  font-family: serif;\n}\n\n.carousel-control .icon-prev:before {\n  content: \'\\2039\';\n  font-size: 100px;\n  color:gray;\n}\n\n.carousel-control .icon-next:before {\n  content: \'\\203a\';\n  font-size: 100px;\n  color:gray;\n}\n\n.carousel-indicators {\n  position: absolute;\n  bottom: 10px;\n  left: 50%;\n  z-index: 15;\n  width: 60%;\n  padding-left: 0;\n  margin-left: -30%;\n  text-align: center;\n  list-style: none;\n}\n\n.carousel-indicators li {\n  display: inline-block;\n  width: 10px;\n  height: 10px;\n  margin: 1px;\n  text-indent: -999px;\n  cursor: pointer;\n  border: 1px solid #ffffff;\n  border-radius: 10px;\n}\n\n.carousel-indicators .active {\n  width: 12px;\n  height: 12px;\n  margin: 0;\n  background-color: lightgray;\n}\n\n.carousel-caption {\n  position: absolute;\n  bottom: 0px;\n  z-index: 10;\n  padding-top: 0px;\n  padding-bottom: 0px;\n  color: #ffffff;\n  text-align: center;\n /* text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);*/\n  padding-left:0px;\n  margin:0px; background-color: white; opacity:.8; width:300px; left:0px; \n}\n\n.carousel-caption .btn {\n  text-shadow: none;\n}\n\n@media screen and (min-width: 768px) {\n  .carousel-control .icon-prev,\n  .carousel-control .icon-next {\n    width: 30px;\n    height: 30px;\n    margin-top: -15px;\n    margin-left: -15px;\n    font-size: 30px;\n  }\n  .carousel-caption {\n    right: 0%;\n    left: 0%;\n    width: 100%;\n   \n  }\n  .carousel-indicators {\n    bottom: 20px;\n  }\n}\n\n.products-tmb, a.products-tmb {\n  float:left;\n  width: 50px;\n  height: 50px;\n  margin: 5px;\n  width: 30px; \n  height:30px; \n}\n\n.products-tmb:hover {\n  border: solid 2px #ccc;\n\n}\n\n#main #content .didyouknow_container{\n\theight: 415px;\n}\n.app_title {\n  display: none;\n}\n#main {\n  border: none;\n}\n\n#main #content {\n  border-radius: 10px;\n  margin: 10px 0 0 0;\n  width: 998px;\n}\n\n#main #content .didyouknow_container{\n  height: 450px;\n}\n\n#main #content .didyouknow{\n  display: none;\n}\n\n#main #content .headline {\n  background: rgba(255, 255, 255, 0.7);\n  margin: 0 0 10px 0;\n  overflow: hidden;\n  width: 100%;\n}\n#main #content .headline img.pin{\n  float: left;\n  margin: 0 25px 0 0;\n  width: 80px;\n}\n#main #content .headline h2{\n  font-size: 45px;\n  font-weight: 100;\n  float: left;\n  margin: 0;\n  width: 400px;\n  text-transform: uppercase;\n}\n\n#main #content img.pin{\n  float: left;\n  margin: 0 10px 0 0;\n  width: 80px;\n}\n#main #content h2{\n  font-size: 50px;\n  font-weight: 100;\n  float: left;\n  margin: 0;\n  width: 260px;\n  text-transform: uppercase;\n}\n\n#main #content .text{\n  float: left;\n  margin: 0 30px 0 0;\n  width: 365px;\n}\n#main #content .text h3{\n  line-height: 32px;\n  margin: 20px 0 10px 0;\n  font-size: 32px;\n  font-weight: 100;\n}\n#main #content .text p{\n  font-size: 22px;\n}\n#main #content .text ul{\n  font-size: 18px;\n  list-style: circle;\n  line-height: 26px;\n}\n#main #content .app {\n  margin: 10px 0 0 0;\n}\n#main #content .app h3{\n  font-size: 28px;\n  font-weight: normal;\n  margin: 0 0 0 0;\n  text-align: center;\n}\n#main #content .other_apps h3{\n  font-size: 28px;\n  font-weight: normal;\n  margin: 0px 0 10px 0;\n}\n#main #content .other_apps div{\n  float: left;\n  margin: 0 27px 15px 0;\n}\n#main #content .other_apps div a {\n  display: inline-block;\n  border: solid 2px white;\n  padding: 10px;\n}\n#main #content .other_apps div a:hover{\n  border: solid 2px #000;\n}\n#main #content .other_apps div a.credits:hover{\n  border: solid 2px white;\n}\n#main #content .other_apps div img{\n  width: 150px;\n}\n#main #content .other_apps div:last-child{\n  margin: 0;\n}\n\n/* New stuff */\n.other_apps:hover {\n   border: solid 2px gray;\n}\n\n.other_apps_first:hover {\n  background-color:white ;\n   border: solid 2px black;\n\n}\n\n\n.other_apps {\n\n  border: 1px solid white;\n}\n\n.earth_teaser {\n  background:url(\'media/img/home/teaser_earth_fade.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\n\n.earth_teaser:hover {\n  background:url(\'media/img/home/teaser_earth.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\n\n.products_teaser:hover {\n  background:url(\'media/img/home/teaser_products.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}     \n\n.products_teaser {\n  background:url(\'media/img/home/teaser_products_fade.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\nfooter.credits, footer.logos {\n  margin-left:0px;\n}\n\n</style>\n\n<!-- Hidden Modal Window for Video -->\n\n  <div id="videoModal">\n    <a href="#" id="exit">Close</a>\n    <div id="videoModalInner">\n        <iframe width="853" height="480" src="//www.youtube.com/embed/0JC24CBVsdo" frameborder="0" allowfullscreen></iframe>\n    </div>\n  </div>\n  \n</head>\n<body>\n  <div id="container">\n    <header>\n      <div class="logo" style="background-color: #fafafa; width:205px; height:66px;">\n        <a href="" title="Home" style="text-decoration: none;">\n        <h1 style="background: url(media/img/all/logo_rainbow.png) repeat-x 100% 100%; padding-bottom:22px; color: black; font-size:53px;">The Atlas</h1>\n        <h2 style="font-size:18px; margin-top:-33px; text-align:center; margin-left:0px; letter-spacing:1px; color:white; text-transform:uppercase; text-decoration:none; background-color: #a0a6a9;">of Economic Complexity</h2></a> \n      </div>  \n      <nav>\n        <ul>\n          <li><a href="explore/">Explore</a></li>\n          <li><a href="stories">Stories</a></li>\n          <li><a href="rankings">Rankings</a></li>\n          <li><a id="newBook" href="book">New Book</a></li>\n          <li><a href="about">About</a></li>\n        </ul>\n      </nav> \n      <br class="clear">\n    </header>\n\n    <div id="main" role="main" style="margin-top:70px">\n      <div class="app_title" id="title" ><h2 style="margin-left:300px; width:800px" id="text_title"></h2></div>\n\n    <style>\n      #content  h3, #content h2, h4 {\n        text-transform: uppercase;\n      }\n\n      #content h3:not(:first-child), #content h4:not(:first-child) {\n        margin: 50px 0px 0px 0px;\n      }\n    </style>\n      <!-- main content -->\n      \n  <div id="content">\n\n  <div id="carousel-example-generic" class="carousel slide" style="width:535px; height:300px;float:left; margin-left:20px; margin-top:20px; margin-right:15px; border:solid 1px #dddddd;">\n  <!-- Indicators -->\n  <!--\n  <ol class="carousel-indicators">\n    <li data-target="#carousel-example-generic" data-slide-to="0" class="active"></li>\n    <li data-target="#carousel-example-generic" data-slide-to="1"></li>\n    <li data-target="#carousel-example-generic" data-slide-to="2"></li>\n  </ol>\n  ->>\n  <!-- Wrapper for slides -->\n  <div class="carousel-inner">\n    <div class="item active">\n      <a href="explore/product_space/export/usa/all/show/2012/">\n        <img src="media/img/home/us_imports_ps.png" alt="..." style="width:550px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">What did the United States import in 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    <div class="item">\n      <a href="explore/stacked/export/usa/all/show/1996.2011.1/">\n        <img src="media/img/home/us_exports.png" alt="..." style="width:540px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">What did the United States export between 1996 and 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    <!--\n    <div class="item">\n      <a href="explore/map/export/show/all/0101/2011/">\n        <img src="media/img/home/horses_export.png" alt="..." style="width:550px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">Who exported Horses in 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    -->\n  </div>\n\n  <!-- Controls -->\n  <a class="left carousel-control" href="#carousel-example-generic" data-slide="prev">\n    <span class="icon-prev"></span>\n  </a>\n  <a class="right carousel-control" href="#carousel-example-generic" data-slide="next">\n    <span class="icon-next"></span>\n  </a>\n</div>\n\n    <div style="float:left; width:350px; margin-bottom:30px; font-size: 30px; font-style:italic; background: url(media/img/home/quote.jpg) no-repeat; background-size:100px; padding-left:40px; padding-top:40px; color:gray">\n      Explore Growth Opportunities\n    </div>\n\n  <a id="videoLaunch" href="#">\n\n    <div class="other_apps_first other_apps" style="float:left;width:370px; height:200px; padding:5px; margin-left:20px;border:1px solid lightgray; background-color:#fafafa;">\n     \n      <div style="width:210px; height:160px; text-align:justify;">\n        <h3 style="text-transform: none; font-size:30px">What is  <span style="text-transform: uppercase;">THE ATLAS?</span></h3> \n        <p style="text-transform: none; font-size:16px">This interactive tool can generate more than 20 million visualizations and map the path of diversification and prosperity for 128 countries. Learn more about the theory of Economic Complexity.</p>\n      </div>\n      <div style="background:url(\'media/img/home/teaser_video_obs.png\') no-repeat; background-size:130px 130px; width:130px; height:160px; margin-top:10px"></div>   \n      \n    </div>\n  \n  </a>\n\n  <p style="clear:both"></p>\n\n    <div class="other_apps" style="float:left; width:455px; margin:20px; border: solid 1px lightgray;">\n  \n  <a href="explore/tree_map/export/usa/all/show/2010/">    <h3 style="float:left; margin-left:50px;font-size:30px; text-transform: none;">Browse Countries \xbb</h3>\n\n      <!--<div class="earth_teaser" style="margin-top:0px"></div>-->\n      <div id="map" style="width:450px; height:250px"></div>\n      \n   \n  </a>\n   <div style="text-align:right; float:right; color:lightgray;" >Map transition by <a href="http://www.jasondavies.com/maps/transition/" style="text-decoration:none;color:lightgray; padding:0px; padding-top:10px" class="credits">Jason Davies</a></div>\n   </div> \n \n<style>\n\n\n\n\n</style>\n\n    <div class="other_apps" style="float:left; width:455px; margin:20px; border: solid 1px lightgray;">\n      <a href="explore/tree_map/export/show/all/8703/2010/"><h3 style="float:left; margin-left:50px;font-size:30px; text-transform: none;">Browse Products \xbb</h3></a><p style="clear:both">\n      <div style="padding-left:20px; padding-bottom:30px">\n        <a href="explore/tree_map/export/show/all/0106/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_0.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_10.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8509/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_11.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8802/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_12.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_13.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_14.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_15.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_20.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_21.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_22.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_30.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_31.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_40.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_41.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/4704/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_42.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/1121/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_43.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/5417/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_44.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_50.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_51.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/2709/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_52.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_54.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_60.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_61.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/1001/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_70.png\') no-repeat; background-size:50px;"></a>\n        \n      </div>\n    </div> \n\n\n\n\n    </div> <!-- END #main -->     \n\n  </div> <!-- END #container -->\n\n  <script src="media/js/libs/jquery/jquery-1.10.2.min.js"></script>\t\n\t<script src="media/js/libs/chosen/chosen.jquery.min.js" ></script>\n  <script src="media/js/libs/bootstrap/js/bootstrap.js"></script>\n\n  <script>\n\t\n  \t// language select dropdown\n  \t$(".language_select select").chosen();\n  \t$(".language_select .chzn-search").hide();\n  \t$(".language_select select").chosen().change(function(){\n  \t\t// set session data to new language\n  \t\tvar language = $(this).val();\n  \t\twindow.location = "/set_language/"+language+"/";\n  \t});\n  \t\n    // google analytics\n  \n    (function(i,s,o,g,r,a,m){i[\'GoogleAnalyticsObject\']=r;i[r]=i[r]||function(){\n    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),\n    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)\n    })(window,document,\'script\',\'//www.google-analytics.com/analytics.js\',\'ga\');\n\n    ga(\'create\', \'UA-41291966-1\', \'harvard.edu\');\n    ga(\'send\', \'pageview\');\n\n    // current nav styling\n  \t$(function(){\n  \t\tvar current_page = document.location.pathname.split("/")[1];\n  \t\t$("nav ul a").each(function(i, el){\n  \t\t\tif($(el).text().toLowerCase() == current_page){\n  \t\t\t\t$(el).addClass("current");\n  \t\t\t}\n  \t\t})\n  \t})\n\t</script>\n\n<style>\n#sticky {\n  padding: 0.5ex;\n  width: 600px;\n  border-radius: 0.5ex;\n}\n#sticky.stick {\n  position: fixed;\n  top: 0;\n  z-index: 10000;\n  border-radius: 0 0 0.5em 0.5em;\n}\n</style>\n\n<script>\nfunction sticky_relocate() {\n  var window_top = $(window).scrollTop();\n  var div_top = $(\'#sticky-anchor\').offset().top;\n  if (window_top > div_top) {\n    $(\'#sticky\').addClass(\'stick\');\n  } else {\n    $(\'#sticky\').removeClass(\'stick\');\n  }\n}\n\n$(function() {\n  $(window).scroll(sticky_relocate);\n  sticky_relocate();\n});\n</script>\n\n\t\n<script src="media/js/libs/jquery/jquery-1.10.2.js"></script>\n<script type="text/javascript">\n    $(document).ready(function(){\n      $("#videoLaunch").click(function(){\n        $("#videoModal").css("display","block");\n        $("#videoModal").animate({opacity:1.0},400);        \n      });\n      $("#exit, #videoModal").click(function(){\n        $("#videoModal").animate({opacity:0.0},200,function(){\n          $("#videoModal").css("display", "none");\n        });\n      });\n    });\n</script>\n<script>\n  // Use to be: Rotate featured video randomly\n  // -> Will be re-used for showing stories randomly\n  // var vid = Math.round(Math.random());\n // var vid = 0;\n //  $(".didyouknow").eq(vid).show();\n  // Set explore URL to be the default country aka where the \n  // user\'s IP address says they are\n // $("nav a").eq(0).attr("href", "explore/tree_map/export/usa/all/show/2012/")\n</script>\n\n\n<script src="media/js/libs/d3/d3.v3.min.js"></script>\n<script src="media/js/libs/d3/d3.geo.projection.min.js"></script>\n<script src="media/js/libs/d3/topojson.min.js"></script>\n<script src="media/js/libs/bootstrap/js/bootstrap.js"></script>\n\n<script>\n$(function() {\n\n  $(\'#myModal\').modal({\n    show: true,\n    keyboard: false\n    });\n\n  $("#carousel-example-generic").carousel({ interval: 2000 });\n\n});\n</script>\n\n<script>\n\nvar ratio = window.devicePixelRatio || 1,\n    width = 450 * ratio,\n    height = 250 * ratio;\n\nvar options = [\n  {name: "Van der Grinten IV", projection: d3.geo.vanDerGrinten4().scale(120)}\n];\n\nvar initial = [-71, 42, 0],\n    rotate = initial.slice(),\n    pause = 0,\n    velocity = [.010, -.002],\n    t0 = Date.now();\n\nvar \n    i = 0,\n    n = options.length - 1;\n\nvar projection = d3.geo.vanDerGrinten4().scale(127); //options[i].projection;\n\nvar graticule = d3.geo.graticule()(),\n    land,\n    boundaries;\n\nvar canvas = d3.select("#map").append("canvas")\n    .attr("width", width)\n    .attr("height", height)\n    .style("width", width / ratio + "px")\n    .style("height", height / ratio + "px")\n    .call(d3.behavior.drag()\n      .origin(function() { return {x: rotate[0], y: -rotate[1]}; })\n      .on("dragstart", function() { pause |= 1; })\n      .on("drag", function(d) {\n        rotate[0] = d3.event.x;\n        rotate[1] = -d3.event.y;\n        projection.rotate(rotate);\n        redraw(path);\n      })\n      .on("dragend", function() {\n        pause &= ~1;\n        t0 = Date.now();\n        initial = rotate.slice();\n      }));\n\nvar context = canvas.node().getContext("2d");\ncontext.fillStyle = "#f9f9f9";\ncontext.strokeStyle = "#000";\n\nvar path = d3.geo.path()\n    .projection(projection)\n    .context(context);\n\nvar menu = d3.select("#projection-menu")\n    .on("change", change);\n\nmenu.selectAll("option")\n    .data(options)\n  .enter().append("option")\n    .text(function(d) { return d.name; });\n\n\nfunction change() {\n  clearInterval(interval);\n  update(options[this.selectedIndex]);\n}\n\nfunction pathTween(projection0, projection1, rotate) {\n  projection0.rotate([0, 0, 0]);\n  projection1.rotate([0, 0, 0]);\n  var t = 0,\n      projection = d3.geo.projection(function(\u03bb, \u03c6) {\n          \u03bb *= 180 / Math.PI, \u03c6 *= 180 / Math.PI;\n          var p0 = projection0([\u03bb, \u03c6]), p1 = projection1([\u03bb, \u03c6]);\n          return [(1 - t) * p0[0] + t * p1[0], (1 - t) * -p0[1] + t * -p1[1]];\n        })\n        .rotate(rotate)\n        .scale(1)\n        .translate([width / 2, height / 2])\n        .clipExtent(projection0.clipExtent()),\n      path = d3.geo.path().projection(projection).context(context);\n  return function() {\n    return function(u) {\n      t = u;\n      redraw(path);\n    };\n  };\n}\n\nfunction update(option) {\n  pause |= 2;\n  canvas.transition()\n      .duration(750)\n      .tween("path", pathTween(projection, projection = option.projection, initial = rotate.slice()))\n      .each("end", function() {\n        pause &= ~2;\n        t0 = Date.now();\n      });\n  path.projection(projection);\n}\n\nd3.timer(function() {\n  if (pause) return;\n  var t = Date.now() - t0;\n  rotate[0] = initial[0] + velocity[0] * t;\n  rotate[1] = initial[1] + velocity[1] * t;\n  projection.rotate(rotate);\n  redraw(path);\n});\n\nfunction redraw(path) {\n  context.clearRect(0, 0, width, height);\n  context.lineWidth = .5 * ratio;\n  if (land) {\n    context.strokeStyle = "#000";\n    context.beginPath(), path(land), context.fill(), context.stroke();\n    context.beginPath(), path(boundaries), context.stroke();\n  }\n  context.strokeStyle = "#999";\n  context.beginPath(), path(graticule), context.stroke();\n  context.lineWidth = 2.5 * ratio, context.strokeStyle = "#000";\n  context.beginPath(), path({type: "Sphere"}), context.stroke();\n}\n\nd3.json("media/js/libs/d3/world-110m.json", function(error, world) {\n  land = topojson.feature(world, world.objects.land);\n  boundaries = topojson.mesh(world, world.objects.countries, function(a, b) { return a !== b; });\n});\n\n</script>\n\n\n</body>\n</html>\n, `body` = , `encoded_headers` = {"Content-Type": "text/html; charset=utf-8", "Vary": "Accept-Language, Cookie", "Content-Language": "en"} WHERE `silk_response`.`id` = 4 '

IntegrityError: duplicate key value violates unique constraint "silk_response_request_id_key"

After activating silk certain urls began erring with:

IntegrityError: duplicate key value violates unique constraint "silk_response_request_id_key"
DETAIL: Key (request_id)=(1166) already exists.

Sentry stack trace: http://toolbox1.tedc.de:9000/bidev/esldj/group/131/

requirements.txt must specify exact versions or version upper bounds

The current requirements.txt does not specify exact versions ( ==0.0.X and so on) but only requires a minimum version number. This means a new release of one of those dependencies could break an installation of this project (due to breaking changes).

Weird "Silk middleware has not been installed correctly" error

Hello, I'm encountering a weird problem with Silk.

First off: the silk middleware is at the top of my middleware stack (I've tried other positions as well) so nothing should be able to stop it from running. What happens though is that the first request issued when the server has started passes through fine. The second request gets stuck in the Silk middleware.

In line 94 of middleware.py:process_request, it throws a RuntimeError exception at

    request_model = RequestModelFactory(request).construct_request_model()

I dug a little deeper and it appears as though what fails is

    request_model = models.Request.objects.create(
            path=path,
            encoded_headers=self.encoded_headers(),
            method=self.request.method,
            query_params=query_params,
            view_name=view_name,
            body=body)

at lines 159–165 in model_factory.py:construct_request_model. I had this open in PDB, so I tried manually issuing the call to models.Request.objects.create, and got the same result. It appears as though it happens at some point when the Request model is save()d, but I haven't been able to figure out more accurately what happens there.

This is the error that is eventually thrown by save()

*** RuntimeError: Silk middleware has not been installed correctly. Ordering must ensure that Silk middleware can execute process_request and process_response. If an earlier middleware returns from either of these methods, Silk will not have the chance to inspect the request/response objects.

And it's worth noting that this does not happen the first request that is issued to the server, only all the ones after that!

I'm not sure how much of this is my fault, nor what I'm doing wrong if anything, but I figured it might be something you'd like to know about if it's an issue in Silk. This is with Django 1.5.9.

I'm don't know if this matters at all, but I noticed that configure() which creates the thread-local storage is called after the RequestModel is created. From what I can tell, the RuntimeError is issued when the thread-local storage is not yet created. On the other hand, configure must be called after the requestmodel is created because it takes it as a parameter.

Ability to clear old runs

It's now starting to get quite messy when I visit /silk/. I wish there was a button to clear it all out. Or a management command at least.

urls.py uses incorrect regex expressions

The regex expressions in urls.py use * which means zero or more.

However, what is likely meant is + as the id numbers must have one or more digits.

HttpRequest body has UTF-8 Character causes UnicodeDecodeError ?

I modify model_factory.py and it worked.
diff model_factory.py /tmp/old.py
76,80c76
< try:
< raw_body = self.request.body.decode("UTF-8")
< except Exception:
< pass

<

    raw_body = self.request.body

Remember selected filtering and ordering

e.g. in profiling, if you leave the page and come back, the filters + ordering + num. shows reset, which is really irritating!

No handlers could be found for logger "silk"

Every time I request , I get this error tip: No handlers could be found for logger "silk"
Except this error tip, everything is ok.
Django version 1.5.1
silk version 0.4

Middleware problem?? No request model was available when processing response. Did something go wrong in process_request/process_view?

I would like to get your suggestions on the Silk.

I'm deploying the django-silk 0.5 into existing django (version 1.6.9) framework environment for checking web application performance.
So I added a null logger for silk in the settings.py of django framework. Before it I used to get No handlers could be found for logger "silk" error. After adding it, no more such error. But still not sure it is correct solutions. Plase suggest on this

I have two ways to launch my web app: using run server manage.py and using nginx (Our production is using uwsgi and nginx).
When I use the first method, I can successfully launch my web application.

But I am facing error "The requested URL The requested URL /..../.../accounts/login/ was not found on this server" when I use nginx which is supported to be used.
I still cannot figure out the meaning of error in my uwsgi log "No request model was available when processing response. Did something go wrong in process_request/process_view?".

Also Suspect the middleware is not supporting correctly.

Still cannot find out any solutions.
Please suggest.

Should not have to manually add a logger for silk

As reported by @darxsys in #52

Silk's static assets are served from the wrong path

This could just be my setup, but Silk's template uses file paths like:

<link rel="stylesheet" href="/static/silk/css/fonts.css"/>

On my machine, this translates to a request for http://127.0.0.1:8000/static/silk/css/fonts.css which doesn't load. I would've expected Silk to use my existing STATIC_URL as a prefix for these files.

(I realise Silk is probably meant for production not local usage, but I'm curious about what it does before I deploy it anywhere)

Dynamic profiling using a decorator causes wrapped_target to be profiled?

See http://i.imgur.com/hdpPKiq.png

AttributeError: This StreamingHttpResponse instance has no `content` attribute

Middleware breaks loading some images with exception AttributeError.

getattr() helps:

# model_factory.py

def body(self):
        body = ''
        content_type, char_set = _parse_content_type(self.response.get('Content-Type', ''))
        # content = self.response.content
        content = getattr(self.response, 'content', '')

Migration error

I'm running Django 1.6.10 and silk 0.4. Here is the trace:

Running migrations for silk:
 - Migrating forwards to 0002_auto__add_field_request_pyprofile__add_index_request_start_time.
 > silk:0001_initial
 ! Error found during real run of migration! Aborting.

 ! Since you have a database that does not support running
 ! schema-altering statements in transactions, we have had 
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:   = DROP TABLE `silk_request` CASCADE; []
   = DROP TABLE `silk_response` CASCADE; []
   = DROP TABLE `silk_sqlquery` CASCADE; []
   = DROP TABLE `silk_profile` CASCADE; []
   = DROP TABLE `silk_profile_queries` CASCADE; []

 ! The South developers regret this has happened, and would
 ! like to gently persuade you to consider a slightly
 ! easier-to-deal-with DBMS (one that supports DDL transactions)
 ! NOTE: The error which caused the migration to fail is further up.
Error in migration: silk:0001_initial
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
    ignore_ghosts = ignore_ghosts,
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/migrators.py", line 254, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/migrators.py", line 329, in migrate_many
    result = self.migrate(migration, database)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration, database)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
    return self.run_migration(migration, database)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/migration/migrators.py", line 85, in run_migration
    south.db.db.execute_deferred_sql()
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/db/generic.py", line 318, in execute_deferred_sql
    self.execute(sql)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute
    cursor.execute(sql, params)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/MySQLdb/cursors.py", line 176, in execute
    if not self._defer_warnings: self._warning_check()
  File "/Users/ian/.virtualenv/flashnotes/lib/python2.7/site-packages/MySQLdb/cursors.py", line 92, in _warning_check
    warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Specified key was too long; max key length is 767 bytes

Enable/disable profiler on-the-fly (without stopping Django)

It would be useful to have a mechanism to start / stop profiler when it is at production (IMO).
I am not sure if it is possible or not, but a page for authenticated users that disables profiling or may be updating SILKY_INTERCEPT_PERCENT to 100 may be suitable for the task.

Post-processing static assets fails due to missing font files

The file silk/static/silk/lib/bootstrap.min.css references font files that have a relative path that exists outside of the repo. For example:

@font-face{font-family:'Glyphicons Halflings';src:url('../../../../../../../Downloads/bootstrap-datetimepicker-2.3.1/sample%20in%20bootstrap%20v3/bootstrap/fonts/glyphicons-halflings-regular.eot'); ...}

This causes post-processing to fail since django can't find the file for cache busting (if you're using CachedStaticFilesStorage):

Post-processing 'silk/lib/bootstrap.min.css' failed!

Traceback (most recent call last):
  File "./manage.py", line 20, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 533, in handle
    return self.handle_noargs(**options)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 168, in handle_noargs
    collected = self.collect()
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect
    raise processed
ValueError: The file 'Downloads/bootstrap-datetimepicker-2.3.1/sample in bootstrap v3/bootstrap/fonts/glyphicons-halflings-regular.eot' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x3191c10>.

Django 1.8 support

Enhance gathering of metadata on SQL queries

Atm the way in which metadata such as num. joins, tables involved for sql queries is gathered is rudimentary. Should be possible to use something like https://code.google.com/p/python-sqlparse/ to parse the SQL and pull out tables etc via the tokens generated

Enhance filtering

At the moment filtering is rather basic. If Silk is used for long enough it eventually becomes impossible to narrow down the requests that you want to look at, especially if is running in a busy QA/Production environment. I'm thinking of adding some way in which to "build up" filters as opposed to just having static filters like the current version. The most important will be a way in which to narrow down requests by path/view and time.

Django 1.7 support

Stacktrace inspector allows users to see any file on the filesystem

Including everything in settings.py ...

'thread._local' object has no attribute 'temp_identifier' (should log a warning stating that this is likely a middleware issue)

I've just installed silk and configured it as it says on your tutorial. However, as soon as I enable the middleware, my site breaks with this error inside the get_identifier method of DataCollector class. Why would this happen? As soon as I disable silk middleware, the site works again. Also, I had to add a logger handler for silk before that since it was throwing an error because of that too. This tool looks promising, I hope you can fix this asap.

UTC time in templates

Maybe should add "expects_localtime=True" to filter silk_date_time?

# templatetags/filters.py

@register.filter(expects_localtime=True)
def silk_date_time(dt):
    return _silk_date_time(dt)

Django 1.5 support broken

Use of atomic instead of commit_on_success appears to break django 1.5. Need to add some sort of shim

Live demo link is broken

The live demo link in the README (http://mtford.co.uk/silk/) gives a 404

Aggregation

It would be really useful to be able to aggregate requests/profiles based on the selected filtering. Perhaps displayed in a graph as a time series. Perhaps these aggregations could be built up and compared or something like that, as it's quite hard to get the overall picture atm.

Build fails occasionally due to "missing manage.py"

The test for permissions when viewing files in the SQL traceback occasionally fails.

Failed to install index for silk.Request model: (1071, 'Specified key was too long; max key length is 767 bytes')

I get this warning when run "python manage.py syncdb".

AttributeError: 'thread._local' object has no attribute 'temp_identifier'

Hi there

With both 0.1.1 and git HEAD, i'm getting the error pasted below.

Django 1.6.5 on OSX Python 2.7.5

Doesn't make a difference whether I'm using runserver or django-devserver for my local server.

Cheers
Steve

AttributeError: 'thread._local' object has no attribute 'temp_identifier'
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
    return self.application(environ, start_response)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/core/handlers/base.py", line 194, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/core/handlers/base.py", line 224, in handle_uncaught_exception
    'request': request
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1175, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1268, in _log
    self.handle(record)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1278, in handle
    self.callHandlers(record)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1318, in callHandlers
    hdlr.handle(record)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 749, in handle
    self.emit(record)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/errordite/handlers.py", line 90, in emit
    payload = self.enrich_errordite_payload(payload, record)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django_errordite/handlers.py", line 43, in enrich_errordite_payload
    if rq.user.is_anonymous():
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/utils/functional.py", line 213, in inner
    self._setup()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/utils/functional.py", line 298, in _setup
    self._wrapped = self._setupfunc()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/auth/middleware.py", line 18, in <lambda>
    request.user = SimpleLazyObject(lambda: get_user(request))
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/auth/middleware.py", line 10, in get_user
    request._cached_user = auth.get_user(request)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 140, in get_user
    user_id = request.session[SESSION_KEY]
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py", line 47, in __getitem__
    return self._session[key]
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py", line 173, in _get_session
    self._session_cache = self.load()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py", line 20, in load
    expire_date__gt=timezone.now()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
    num = len(clone)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
    self._fetch_all()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/src/django-silk/silk/sql.py", line 48, in execute_sql
    DataCollector().register_query(query_dict)
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/src/django-silk/silk/collector.py", line 72, in register_query
    ident = self.get_identifier()
  File "/Volumes/HDD/Users/steve/.virtualenvs/yj/src/django-silk/silk/collector.py", line 26, in get_identifier
    self.local.temp_identifier += 1
AttributeError: 'thread._local' object has no attribute 'temp_identifier'

Pip install direct from repo fails

> pip install -e git+https://github.com/mtford90/silk.git#egg=silk

Obtaining silk from git+https://github.com/mtford90/silk.git#egg=silk
  Cloning https://github.com/mtford90/silk.git to /Users/me/.virtualenvs/proj/src/silk
  Running setup.py (path:/Users/me/.virtualenvs/proj/src/silk/setup.py) egg_info for package silk
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
    IOError: [Errno 2] No such file or directory: '/Users/me/.virtualenvs/proj/src/silk/setup.py'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

IOError: [Errno 2] No such file or directory: '/Users/me/.virtualenvs/proj/src/silk/setup.py'

Organise cProfile output as a sortable, more organised table.

It's currently just text dumped into a code block. Would be nice if this could be organised as a HTML table and possibly change the sorting etc (i.e. only sorted by cumulative atm)

request: timestamp on list of requests

on the main /silk page that shows a block for each request, it would be helpful to have the request timestamp so you can more easily identify groups of requests (i.e. load/ajax) and ensure you are looking at the right profile

Incorrect interface for execute_sql

Hello,

I had some problem related to the function interface of execute_sql in silk/sql.py.

Your interface is:

def execute_sql(self, *args, **kwargs)

With some code to retrieve the result_type keyword:

kwargs.get('result_type', 'multi')

However the interface in Django is different (https://github.com/django/django/blob/master/django/db/models/sql/compiler.py):

def execute_sql(self, result_type=MULTI)

In the case where we call without specifying the keyword:
execute_sql('single')

Your code doesn't give the expected results.

Migrations do not work with Django 1.5.9

Hi, I'm using Django 1.5.9 and wanted to integrate silk since it should support Django 1.5 (see README).
Now, after I ran migrate I get this:

UnknownMigration: Migration 'silk:0001_initial' probably doesn't exist.
Traceback (most recent call last):
  File "/home/kai/IdeaProjects/core-tms/env/local/lib/python2.7/site-packages/south/migration/base.py", line 302, in migration
    migration = __import__(full_name, {}, {}, ['Migration'])
  File "/home/kai/IdeaProjects/core-tms/env/local/lib/python2.7/site-packages/silk/migrations/0001_initial.py", line 4, in <module>
    from django.db import models, migrations
ImportError: cannot import name migrations

The migrations module was integrated in Django 1.7 I think, so how do I use silk with Django 1.5?
Thanks!

jazzband / django-silk Goto Github PK

django-silk's People

Stargazers

Watchers

Forkers

django-silk's Issues

<

Recommend Projects

Recommend Topics

Recommend Org