They have a very specific shape (oval), and very predictable data (that could be Regexed out).
It would be nice to have a bounded crop of the image, as well as the extracted data.

What

• Not an AI task per se, but might yield interesting datasets for actual AI projects:
  Process the Google Cloud Vision results through Google NLP.

Part of

• #76

Automatically detect all text mentions of carbon footprints. Probably adopt a text-based approach

What

• Similar to #11
• A small number of products often feature a QR code. Sometimes, these are part of a "competition", almost always these are URLs.
• By recognising QR codes, simple regex could detect brands from the urls; or automatically link to the product on manufacturer's site.

Part of

• #76

What

• Perhaps a robotoff task is best, to allow human confirmation; but there are a number of categories where a product has certain physical properties that require a common style of packaging.

Examples

• Milk, will come as Bottled or in a Carton, Plastic, Glass or Cardboard
• Eggs, frequently in a cardboard or plastic carton.
• Pasta sauce is typically in a glass jar or bottle.

A lot of this can probably be guessed from looking at the most common packaging for a given (specific) category.

We'd also want a way to exclude some categories, where the category is really broad like "Fruits"

Part of

• #97

What

• http://www.hutter1.net/publ/reflect.pdf describes an approach around detecting reflection, potentially for a lot of transparent glass or plastic containers we could sense bright patches corresponding to reflected light patches

Part of

• #97

What

• Remove backgrounds. We currently have a basic technique but it can made much better

Part of

What

Part of

Detect their category based on their front image

You must enter the characters of the barcode or send a product image when the barcode is visible.

to>

You must enter the characters of the barcode or send a product image where the barcode is visible.

What

• Generate a large labelled dataset of cropped nutrition tables from Open Food Facts data
• This will enable autocropping systems.

Part of

• #75
• #8

Recognize labels and logos on packaging.

NB: we have already used Google Cloud Vision to do that on part of the database. Some interesting results, but we could benefit from a more specialized solution (we have the clean images of the labels on the Wiki, but no brand dataset)

What

• On bottles and tins, the image will be curved.
• Other a way to flatten the image, as if it had been removed from the bottle.
• http://dsp.stackexchange.com/questions/2406/how-to-flatten-the-image-of-a-label-on-a-food-jar

Part of

• #8

What

• Generate a large labelled dataset of food pictures from Open Food Facts data

Part of

• #75

What

• Recognize packaging type (bottle, bag…)
• Closely related to #23

Part of

• #97

Some products have started displaying the ABCDE Nutriscore.
Detect that products have a Nutriscore, and which one it is.

Example: https://world.openfoodfacts.org/product/3596710467815/houmous

What

• A fun experiment could be to try and average product images from categories.
• http://thechive.com/2015/09/13/heres-what-the-average-person-looks-like-in-each-country-11-photos/

Part of

• #76

Many pictures sent to Open Food Facts are not properly rotated. Create a system able to autorotate pictures.

What

Deploy a labelling server

Part of

What

• Create a system able to create 3D models of products from different images

Part of

• #76

Who for

• ML developpers

What

• #14
• #15
• #16
• #17

Why

• Data is needed to train, while being practical for on-laptop training

Part of

• #76

I read somewhere that table recognition is on roadmap...
When this is ready, scanning "bills" or invoices to extract products price by brand/store/date.

With a shared price information, comparators and other apps would be possibles...

• See price over time of a product
• Compare stores margins
• Detect price fluctuation
• Compare categories average prices
• Compare categories average country differences
• An app to calculate the best couple of store to get this gorcery list you have to buy for next dinner, created from my last bills scans recurring products
  ... and so on

Maybe privacy is to be discussed though !
Maybe a mix of anonymous price data, and a way to keep the pictures/OCR/data local in the user device (aka let apps owners use OCR localy)

My 2 cents

What

• Generate a large labelled dataset of cropped ingredient list from Open Food Facts data
• This will enable building autocropping systems

Part of

• #75

Hello,

Firstly, this really is a great service and the API is working fantastically well, so thank you.

When searching using the API, for example for "orange juice" the results come back ordered by unique_scans_n which is great

https://world.openfoodfacts.org/cgi/search.pl?search_terms=Orange%20juice&search_simple=1&action=process&json=1&page_size=100

Searching for "Orange juice" in the Open Food facts website search box brings back different (possibly better?) search results. This is despite the fact that the URL looks identical to mine, for example:

https://world.openfoodfacts.org/cgi/search.pl?search_terms=orange+juice&search_simple=1&action=process

Is there any way to get the same results back from the API? &sort_by=unique_scans_n doesn't make a difference as it appears they are already sorted by this value by default.

Thanks in advance

What

Part of

A more specific version of #7

There are two main variants of "Made in (Country)" labelling which seem common

• Specific logos, such as "Australian Made" (see https://www.australianmade.com.au/why-buy-australian-made/about-the-logo/ for examples)
• Wording such as "Product of" or "Made in" plus a country.

The plain text wording is likely the easiest to do; as it would be simply a pipeline looking for a country name, plus hint text; which would provide a strong indicator of country of origin.

What

• Extract data from nutrition tables, see if we can get to the point where it's reliable enough to offer it as a suggestion to users.
• Should probably be done in conjunction with other things, to provide checks (eg checking based on category, product name…)
• #53
• #17
• #13
• openfoodfacts/robotoff#372

Part of

• #76

What

• Label automatically a product when braille is detected

Part of

Example

example: https://world.openfoodfacts.org/product/8009004901186/cioccolato-extra-fondente-alce-nero

What

• Recognize packaging material from photos (cardboard, plastic…)

Part of

• #76

What

• Enable restaurant/fresh food detection to give average values in case someone taps (This product has no barcode) https://www.tensorflow.org/datasets/catalog/food101
• https://github.com/kumarkan/Food_Detection
• Enable barcode less product recognition using something like Pastec.io (opensource)

Part of

What

• Generate new categories by automatically clustering similar products, break down existing categories in sub categories

Part of

• #98

What

• We could use the generic logos & labels system
• We could do something custom ?
• We have the vector versions of the symbols here: https://www.figma.com/file/Gme79AHqYkupXL5UZuvF0D/Packaging-shapes-from-the-French-CITEO-system
• openfoodfacts/openfoodfacts-server#5627

Part of

• #97

Example

What

• Score whether a contributor makes good or bad contributions (will enable edit scoring)

Part of

What

• Create a system that unblurs images

Part of

• #76

Almost all products feature a numeric quantity and unit, this would predominantly be text extraction and filtering results via regexp of common units.

Importantly, this could be used as a broad indicator for #29 - fluids (ml, L, etc) vs solids (g, kg) require different container mechanisms

https://www.pyimagesearch.com/2015/09/07/blur-detection-with-opencv/
Allows us to prevent overwriting of images by a blurry one.

What

• Not AI per se, but need to flatten images (on bottles for instance) to enable better recognition on other projects

Part of

• #8
• #76

Generate a large labelled dataset of barcodes from Open Food Facts data
The interesting point is to include blurry barcodes, ones not recognized by current libraries…

What

Part of

What

• Currently people have to experiment, or use scientific knowledge to create charts on Open Food Facts.
• See if we can surface charts with interesting shapes, outliers… automatically.
• Humans could potentially then review those charts, how interesting they are…
• We could use past chart usage as training data, or just generate images of results, and compare them to classic scientific distributions (bimodal…)

Potential tasklist

• Generate a dataset of all the charts query ever made on Open Food Facts
• Extract all parameters
• See if some charts can already be suggested easily because they have been shared/recreated several times
• See if suggesting some charts could be problematic
• See if more suggestions can be synthesized
• Create a suggestion engine in Robotoff
• Create thoughtful product opener / mobile integrations

Part of

• #76

What

• We have people taking photos of screens, with copyrighted images.
• Is it possible to detect such cases ?
• https://github.com/dukn/screenshot_detector

Part of

• #99

Example

Many cloud computer vision have face detection algos, but they might find a face on packaging while we're looking for faces of users.
Allow to whitelist "known faces" and to flag actual faces.

What

• Predict if an edit is likely good or bad

Part of

• #99

What

• They have a very specific shape, and very predictable data (if they can be recognized using the zxing library, or using OCR).
• Sometimes, both ZXING and OCR will fail, and being able to recognize it looks like a barcode, but it's not legible, is interesting in many ways.
• Flagging products for human review, nudging the contributor of the photo, quality checks…
• It would be nice to have a bounded crop of the barcode image, as well as the extracted data.

Part of

A simplistic assumption: the name of the product should be the largest text on the front of the product.
Based on this, we could compute a ratio between the area of the bounding boxes (width by height) and the amount of letters inside it .
Based on this ratio, we could have candidates for the product name

Not a AI task per se, but will create even more data for the other AI projects.

Process the whole DB with Google Cloud Vision, with all options enabled. We've done it on part of the DB, we need to redo it again, with all options enabled.

All products have ingredient lists
Many products have ingredient lists in more than one language.

We'd need a system to bound ingredients lists, and ingredient lists in each language. That way we could autoselect an ingredient list image for many products.

Analyse what contributions contributor like most, and propose to contribute more in that direction, or to try something new.