Git Product home page Git Product logo

ocp8's People

Contributors

dependabot[bot] avatar freezed avatar

Stargazers

 avatar

Watchers

 avatar

ocp8's Issues

Add test for if non-fr tags & empty brands ie:

{'brands_tags': [],
 'categories_tags': ['en:plant-based-foods-and-beverages',
                     'en:plant-based-foods',
                     'en:fruits-and-vegetables-based-foods',
                     'en:fruits-based-foods',
                     'en:fruits',
                     'en:apples'],
 'code': '4020992004714',
 'nova_group': False,
 'nutrition_grades': False,
 'product_name': 'Granny Smith'}

Add `fake_get_json_count_zero()`

with open("ersatz/tests/samples/api-count-0.json", "r") as json_file:
    json_response_from_api = json.load(json_file)
return json_response_from_api

Wrong URL name

<p>Try to <a href="{% url 'search' %}?foo=bar&s=sel" >search something</a></p>

The URL name was changed in 3a1bc30

Got an error with an inconsistent line number when URL …/search/ was called :

Error during template rendering
In template ~/git/ocp8/omega/templates/base.html, error at line 13

Reverse for 'search' not found. 'search' is not a valid view function or pattern name.

3	<html lang="en">
4	  <head>
5	    <meta charset="utf-8">
6	    <title>omega{% block title %} - index{% endblock %}</title>
7	  </head>
8	  <body>
9	    <header><h1>omega</h1></header>
10	    <aside>
11	      <nav>
12	        <ul>
13	          <li style='display: inline;'>Menu : </li>
14	          <li style='display: inline;'><a href="{% url 'home' %}">home</a></li>
15	          <li style='display: inline;'><a href="{% url 'account-index' %}">account</a></li>
16	          <li style='display: inline;'><a href="{% url 'ersatz-index' %}">ersatz</a></li>
17	          <li style='display: inline;'><a href="{% url 'admin:index' %}">admin</a></li>
18	        </ul>
19	      </nav>
20	    </aside>
21	    <main>
22	    {% block content %}{% endblock %}
23	    </main>

Search result display : all products overriden by the last one

URL :
http://127.0.0.1:8000/ersatz/search/?s=poivre

Return :

ersatz -result
Status : True
Products
Code	Nom	Catégorie	Nutriscore
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b
3242272261650	XtremBox - Radiatori Bœuf Sauce au poivre	['en:meals', 'en:meat-based-products', 'en:meals-with-meat', 'en:beef-dishes', 'en:microwave-meals', 'en:pots', 'en:pasta-in-a-box']	b

Non regression test for #27

The test is written but commented, there is an issue with whitenoise that I cannot handle now.

Here is the pytest traceback

pytest -vv ersatz/tests/tests.py::test_search_return_status_is_false

==================================================== test session starts =====================================================
platform linux -- Python 3.6.6, pytest-3.8.1, py-1.6.0, pluggy-0.7.1 -- ~/git/ocp8/.venv/bin/python3
cachedir: .pytest_cache
Django settings: omega.settings (from ini file)
rootdir: ~/git/ocp8, inifile: pytest.ini
plugins: django-3.4.3, cov-2.6.0
collected 1 item                                                                                                             

ersatz/tests/tests.py::test_search_return_status_is_false FAILED

========================================================== FAILURES ==========================================================
_____________________________________________ test_search_return_status_is_false _____________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff3fa427240>

    def test_search_return_status_is_false(monkeypatch):
        monkeypatch.setattr('ersatz.views._get_search_context', fake__get_search_context)
>       response = views.search('This request will return `{status: False}`')

ersatz/tests/tests.py:54: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
ersatz/views.py:127: in search
    return render(request, 'ersatz/result.html', data)
.venv/lib/python3.6/site-packages/django/shortcuts.py:36: in render
    content = loader.render_to_string(template_name, context, request, using=using)
.venv/lib/python3.6/site-packages/django/template/loader.py:62: in render_to_string
    return template.render(context, request)
.venv/lib/python3.6/site-packages/django/template/backends/django.py:61: in render
    return self.template.render(context)
.venv/lib/python3.6/site-packages/django/template/base.py:171: in render
    return self._render(context)
.venv/lib/python3.6/site-packages/django/test/utils.py:96: in instrumented_test_render
    return self.nodelist.render(context)
.venv/lib/python3.6/site-packages/django/template/base.py:937: in render
    bit = node.render_annotated(context)
.venv/lib/python3.6/site-packages/django/template/base.py:904: in render_annotated
    return self.render(context)
.venv/lib/python3.6/site-packages/django/template/loader_tags.py:150: in render
    return compiled_parent._render(context)
.venv/lib/python3.6/site-packages/django/test/utils.py:96: in instrumented_test_render
    return self.nodelist.render(context)
.venv/lib/python3.6/site-packages/django/template/base.py:937: in render
    bit = node.render_annotated(context)
.venv/lib/python3.6/site-packages/django/template/base.py:904: in render_annotated
    return self.render(context)
.venv/lib/python3.6/site-packages/django/template/defaulttags.py:398: in render
    return strip_spaces_between_tags(self.nodelist.render(context).strip())
.venv/lib/python3.6/site-packages/django/template/base.py:937: in render
    bit = node.render_annotated(context)
.venv/lib/python3.6/site-packages/django/template/base.py:904: in render_annotated
    return self.render(context)
.venv/lib/python3.6/site-packages/django/template/loader_tags.py:62: in render
    result = block.nodelist.render(context)
.venv/lib/python3.6/site-packages/django/template/base.py:937: in render
    bit = node.render_annotated(context)
.venv/lib/python3.6/site-packages/django/template/base.py:904: in render_annotated
    return self.render(context)
.venv/lib/python3.6/site-packages/django/templatetags/static.py:106: in render
    url = self.url(context)
.venv/lib/python3.6/site-packages/django/templatetags/static.py:103: in url
    return self.handle_simple(path)
.venv/lib/python3.6/site-packages/django/templatetags/static.py:118: in handle_simple
    return staticfiles_storage.url(path)
.venv/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py:152: in url
    return self._url(self.stored_name, name, force)
.venv/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py:131: in _url
    hashed_name = hashed_name_func(*args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <whitenoise.storage.CompressedManifestStaticFilesStorage object at 0x7ff3fa3a8ba8>, name = 'img/off-logo-footer.png'

    def stored_name(self, name):
        parsed_name = urlsplit(unquote(name))
        clean_name = parsed_name.path.strip()
        hash_key = self.hash_key(clean_name)
        cache_name = self.hashed_files.get(hash_key)
        if cache_name is None:
            if self.manifest_strict:
>               raise ValueError("Missing staticfiles manifest entry for '%s'" % clean_name)
E               ValueError: Missing staticfiles manifest entry for 'img/off-logo-footer.png'

.venv/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py:419: ValueError
================================================== 1 failed in 0.77 seconds ==================================================
zsh: exit 1     pytest -vv 

Catch DB error if migration is not done

URL : https://ocp8-1664.herokuapp.com/ersatz/favorite/3083680630122/8434702000556/

Displays :

Vous devez être connecté pour utiliser cette fonctionnalité
Produit initial
oups…
Substitut associé

Do not display this :

Produit initial
oups…
Substitut associé

Managing Heroku DB row limit

Free plan offer a DB limited to 10000 rows. That is a good occasion to build a tool to monitor the current status and do maintenance to keep the row number under the ratio.

Local & distant DB conflict

Step to reproduce

  1. A product is stored in DB without nutrition_grades

  2. A new request get this product, now, with a nutrition_grades

  3. Substitute search fails :

    1. search : http://127.0.0.1:8000/ersatz/search/?s=Original
    2. choose Ricoré Original (Nestle) - [E]
    3. Traceback :
Environment:

Request Method: GET
Request URL: http://127.0.0.1:8000/ersatz/candidates/7613032655495

Django Version: 2.1.1
Python Version: 3.6.7
Installed Applications:
['account.apps.AccountConfig',
 'ersatz.apps.ErsatzConfig',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "~/git/ocp8/.venv/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "~/git/ocp8/.venv/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  126.                 response = self.process_exception_by_middleware(e, request)

File "~/git/ocp8/.venv/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  124.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "~/git/ocp8/ersatz/views/views.py" in candidates
  25.     context = ersatz.get_substitute_context()

File "~/git/ocp8/ersatz/views/toolbox.py" in get_substitute_context
  391.             context = self.get_candidate(self.product_val['nutrition_grades'])

File "~/git/ocp8/ersatz/views/toolbox.py" in get_candidate
  411.         ersatz_candidates = {cat: list() for cat in ng_sequence[png]}

Exception Type: KeyError at /ersatz/candidates/7613032655495
Exception Value: ''

Define data model

  • choose kept fields (name, cat, nutriscore, …?)
  • define tables
  • define relations

Page number is not reset when getting back to homepage

Very strange… the page param persists if next request do not use it.

System check identified no issues (0 silenced).
October 10, 2018 - 17:50:39
Django version 2.1.1, using settings 'omega.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=pizza&page=1', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'pizza', 'page': 1}}}
API response page : 1
[10/Oct/2018 17:50:50] "GET /ersatz/search/?s=pizza&p=1 HTTP/1.1" 200 10256
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=pizza&page=2', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'pizza', 'page': 2}}}
API response page : 2
[10/Oct/2018 17:50:57] "GET /ersatz/search/?s=pizza&p=2 HTTP/1.1" 200 10163
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=pizza&page=3', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'pizza', 'page': 3}}}
API response page : 3
[10/Oct/2018 17:51:02] "GET /ersatz/search/?s=pizza&p=3 HTTP/1.1" 200 10165
[10/Oct/2018 17:51:03] "GET / HTTP/1.1" 200 1052
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=foobar&page=3', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'foobar', 'page': 3}}}
API response page : 3
[10/Oct/2018 17:51:18] "GET /ersatz/search/?s=foobar HTTP/1.1" 200 1007
[10/Oct/2018 17:51:21] "GET / HTTP/1.1" 200 1052
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=salade&page=3', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'salade', 'page': 3}}}
API response page : 3
[10/Oct/2018 17:51:33] "GET /ersatz/search/?s=salade HTTP/1.1" 200 10415
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=salade&page=4', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'salade', 'page': 4}}}
API response page : 4
[10/Oct/2018 17:51:39] "GET /ersatz/search/?s=salade&p=4 HTTP/1.1" 200 10194
API traceback : {'context': 'ersatz.api.get_json() method', 'status': False, 'error': {'url': 'https://fr.openfoodfacts.org/cgi/search.pl?search_simple=1&page_size=24&action=process&json=1&search_terms=salade&page=1', 'payload': {'search_simple': 1, 'page_size': 24, 'action': 'process', 'json': 1, 'search_terms': 'salade', 'page': 1}}}
API response page : 1
[10/Oct/2018 17:51:43] "GET /ersatz/search/?s=salade&p=1 HTTP/1.1" 200 10058

User management

  • show account info
  • create an account icon when it is not
  • link to 'My products' when user is logged in
  • add a substitute to favorite

Failover to API if no ersatz found in DB

Use https://world.openfoodfacts.org/category/<cat_name>/(…)/category/<cat_name>.json request pattern.

Example :
https://world.openfoodfacts.org/category/shepherd-s-pie/category/microwave-meals/category/beef-dishes/category/meals-with-meat.json

Stages :

Regex isn't safe enough to parse URL query string

if a key ends by s= (foos= by ex.) 1st check pass, but do not find the key GET['s'] and crash.

Traceback :

Request Method: GET
Request URL: http://127.0.0.1:8000/ersatz/search/?foos=fromage
Exception Type: MultiValueDictKeyError at /ersatz/search/
Exception Value: 's'

'My Products' page

  • list favorites
  • access from menu
  • button under each product in the search
  • detail characteristics of the substitutes

Deploy with Heroku

  • create
  • connects to github/master
  • config PostgreSQL AddOn
  • config staticfiles deploy

`Ersatz` django app

  • basic search product (get 1st page only)
  • basic display product list
  • propose to show the next page (if aivaiable)
  • [optional] result table fields are sortable

Product matching query does not exist

TODO :

  • catch error & display message
  • non regression test
  • checks impact on data already stored
  • FIX

Description

Message :

DoesNotExist at /ersatz/code/0611972117018
Product matching query does not exist.
Request Method:	GET
Request URL:	http://127.0.0.1:8000/ersatz/code/0611972117018
Django Version:	2.1.1
Exception Type:	DoesNotExist
Exception Value:	
Product matching query does not exist.
Exception Location:	~/git/ocp8/.venv/lib/python3.6/site-packages/django/db/models/query.py in get, line 399
Python Executable:	~/git/ocp8/.venv/bin/python
Python Version:	3.6.6
Python Path:	
['~/git/ocp8',
 '/usr/lib/python36.zip',
 '/usr/lib/python3.6',
 '/usr/lib/python3.6/lib-dynload',
 '~/git/ocp8/.venv/lib/python3.6/site-packages']
Server time:	ven, 19 Oct 2018 15:35:50 +0200

Request on DB :

freezed@(none):omega> SELECT p.*, pc.category_id, c.name
                   FROM ersatz_product p
                       JOIN ersatz_product_category pc ON pc.product_id = p.id
                       JOIN ersatz_category c ON c.id = pc.category_id
                   WHERE p.code = '0611972117018'
                   ORDER BY p.id DESC
                   LIMIT 20;
+------+------------------------------------------------------------+--------------------+---------------+---------------+--------+
| id   | name                                                       | nutrition_grades   | code          | category_id   | name   |
|------+------------------------------------------------------------+--------------------+---------------+---------------+--------|
| 3466 | Sardines Écossaises Brisling, dans huile d'olive (Rob-roy) | b                  | 0611972117018 | 145           | fishes |
| 3466 | Sardines Écossaises Brisling, dans huile d'olive (Rob-roy) | b                  | 0611972117018 | 20            | meals  |
+------+------------------------------------------------------------+--------------------+---------------+---------------+--------+
SELECT 2
Time: 0.021s

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.