tournesol-app / tournesol-meta Goto Github PK

This ticket would benefit greatly from having data in one's local database #42

• Create a new django application "ml" (name could be changed) that contains the Machine Learning code that can be run with the managment command ml_train.
• Ensure the code can runs and produces expected results

• Edit CSS of index.html page
• Start running Go server and configure nginx

Create pitch videos. Here are the scripts to be reviewed:
EN: https://docs.google.com/document/d/1imc6v6K_-zjDqBZ5ifkwEBDRqKD-Jznm2lsVxOCwioY/edit
FR: https://docs.google.com/document/d/1deA6TzFHW27x2DPndf0Jtir16MxcXzx2_YGfS0HeK10/edit

PeerTube (source) is an open-source alternative to Youtube. Videos are hosted on a decentralized server network and bandwidth is shared between users.

An ethical recommendation system like Tournesol could be an interesting feature for Peertube users, who are usually concerned about Youtube supremacy.

Peertube works with plugins that can be installed by instance administrators. So creating a Tournesol plugin for Peertube is probably the way to do.

Note that it's probably a little more work than #48, because it requires to evaluate non-Youtube videos. In particular, evaluating videos that belongs to a federated network and not a central website is probably not that simple, but SepiaSearch, the new Peertube's global search engine, could be used here.

If the Tournesol organization appreciate this suggestion, it makes sense for the organization to host a Peertube instance dedicated to the Tournesol project, in order to work on this plugin and let users test it and send feedback. To kill two birds with one stone, maybe @lenhoanglnh could also use this instance to share its science videos on a more ethical platform.

Our bank account details:
IBAN CH42 0900 0000 1569 4102 5
BIC POFICHBEXXX

This ticket will be complete when tournesol-backend provides API endpoints for creating users, resetting passwords and authentication.

For crowdfunding.

Code of the current frontend should be added to tournesol-frontend repository. Additionally implement possibility to serve and develop on the app locally

Create a short document aggreed on by @lenhoanglnh @sergeivolodin @aidanjungo and @lfaucon describing Sergei's role

We decided to send alert to a discord channel

Instead of having multiple columns per feature, we want to have separated table with columns (scores x feature_name x weight)

We need to make it easier for users to choose the videos they are rating.

I need to optimize it :)

• Authentication for the wiki should use the same user base as for the backend
• Should reuse the same database as on the previous wiki

• This can be a good starting point (but we should probably adapt it): https://en.wikipedia.org/wiki/Wikipedia:Copyrights
• I think that it would be good to have a pledge for use for good.
• We must list any data reusing entities somewhere on the website. This adds credibility to Tournesol and helps red-flag misuses.
• Finally, we should discuss the incentives caused by an (influential) data reuse: if an algorithm A reuses the Tournesol database in a very influential and non-strategyproof manner, then Tournesol contributors may care about the impact of algorithm A, and may be incentivized to participate strategically, thereby endangering the trustworthiness of the database. We should encourage the use of the Licchavi framework (link to the paper pending...) or of the Tournesol scores only.

As far as I understand, here are the steps to be taken.

1. Obtain a "numéro identification d'entreprise". See here.
2. Register to Caisse Cantonale Vaudoise. Fill this document.
3. LPP and LAA.
4. Taxation at the source (looks like the Caisse Cantonale Vaudoise might offer this already).

1. Create an account on Godaddy with the association's email.
2. Coordinate with Sergei to transfer the tournesol.app domain to the new account

Once the new server is ready to be online. Write a script to import the data from our old database into the new backend db.

The script or django management command should be commited to the repository (probably in a scripts/ folder) and usage should be documented

One-shot donations: https://wemakeit.com/pages/start-a-project

Figure out whether we can:

• distribute scripts using youtube-dl on Github
• distribute files/db dumps containing youtube video metadata on Github
• distribute video's metadata on tournesol.app

Once the backend exposes API enpoints for authentication. Implement such authentication on the frontend. This is best done by reusing the previous implementation, unless there are good reasons to change it.

This ticket should also include:

• Documenting how to run both the frontend and backend locally for development
• A configuration file to enable local development, staging, and live versions

Right now, the wiki accesses the DB using an extension https://github.com/tournesol-app/django_mediawiki_auth_postgres

A better way is to use the Token auth, or the password auth inside the php script. This way, we don't have to have access to the database from the wiki user.

Blocks wiki contributions

For crowdfunding.

We need to upload a paper on ArXiV describing Tournesol overall, so that we can be more easily cited by academic papers (like my upcoming papers 😜).

I'll work on a first draft to upload quickly soon.

(the advantage of ArXiV is that we can be easily cited, while regularly updating the paper)

On tournesol-backend:

• Set-up account on CircleCI
• Set-up pipeline for tests & code syntax on commits to master branch and PRs
• Adds flakify/pylint
• Adds a simple test test that ensures that the server runs properly (requirements + python manage.py runserver does not crash)

(can be discussed)

1. Set a timeout on apache, in case if exceeded, show a message like "tournesol is experiencing difficulties, please stand by and try again later etc"
   • Write the message and the layout @tournesol-app/designers
2. Create monitoring for OOM in case if Munin is not working (or report right after OOM ends)
   

On the figure, Munin is not running because OOM, therefore, there are reports on Telegram
3. Buy a new server and migrate to it (database, website, ...). Freeze the DB for some time (showing "technical difficulties"), copy and run on the new instance. Set up a tunnel for apache traffic to go there before DNS changes

Tournesol contributors should be able to manipulate their rate later list: The need to add, fetch, list, or delete from their rate later list using the api endpoint /rate_later

There are several open-source Youtube clients:

• Invidious (web app) (source);
• FreeTube (desktop app) (source);
• NewPipe (mobile app (source);
• SkyTube (mobile app) (source);

These are front-end alternatives that allow users to watch Youtube videos without sending too much information to Google with interesting features like the ability to follow channels without a Google account.

Since these client have a full control on the UI, they have full control on how to recommend videos to users. Therefore it makes sense here to use an alternative to the Youtube recommendation system, such as Tournesol.

Their users are probably concerned about Youtube supremacy so it could be an efficient way to offer Tournesol to people that want it, and to promote the project. Their developers may accept to add such a feature in their code base if it's well integrated on the UI.

So I wonder if the Tournesol organization and community would want to give some resources and time to contribute to these projects this way.

Edit: having a good recommendation API that allows developers to easily retrieve recommendations and use them in their project (that can be an alternative Youtube client or something else) is probably a good approach.

Create endpoints for a contributor to create, update, delete, fetch, and list his/her own comparisons.

Updates must be possible with partial data for example a contributor submitted comparisons on multiple criteria, but is now sending to the server that he/she updates a single criteria.

The implementation must be careful about handling comparisons between videos X & Y, and comparisons between videos Y & X.

This is best done by keeping the API as similar as possible to the previous implementation, unless there are good reasons to change it.

Details:
The endpoint should be /comparison/

• GET request with parameters video_1 and video_2 returns the single desired comparison or an error if this comparison does not exist. It should return the comparison with opposite ratings if our db contains the comparison between video_2 and video_1 instead (If this is not clear talk to @lfaucon about it)
• GET request with parameters video (optional) offset and limit return the list of comparisons where the requested video is either video_1 or video_2 or just the list of all comparisons if video is not specified
• POST request with parameters video_1, video_2 and any ratings between 0 and 100 for the rating criterias should create the comparison
• PUT with same parameters should update the comparison

• Add e-mail alert to Munin (see the devops repo)
• Decrease number of messages (do not send at each form save)
• Read on how to not get rejected in general

Endpoint /videos, GET request returns sorted list of videos using parameters:

• limit: default 10
• offset: default 0
• date_lte, date_gte
• language
• <CRITERIA_NAME_WEIGHT> value between 0 and 100 (default: 50) the weight of this criteria in the ranking

The video must be sorted based of the sum of scores per criteria multiplied by their weights <CRITERIA_NAME_WEIGHT>

The returned video must contain most of the data about the video (most fields available on the Video and VideoCriteriaScores model)

Optional:
Add search parameter for a text search using title / metadata of the videos. Discuss with @lfaucon and @lenhoanglnh if you want to implement this feature

Using the boilerplate backend django application, ensure that our ansible infra serves the application.

This might be best reviewed by @elineda