kev007 / artful Goto Github PK

words - readd corpus column
rename word table to freq
relative paths for linux
working directory

See: https://github.com/kev007/ARTFUL/blob/frontend/server/src/main/webapp/resources/js/app.js#L240-L270

Structure:
mergedData.features[1].properties.name
mergedData.features[1].properties.frequency

See: http://corpora2.informatik.uni-leipzig.de/ for all languages and the hyperlinked corpora

Create a database structure concept and create an empty implementation.

Calculate and display quotient for each category (see project description)

Normalize frequency entries based on the corpus size

At the moment, we create a layer for every country we have data for. Because of this many countries don't have layers, as you can see here:

But instead we should create a layer for EVERY country and just leave the one white which don't have data.

This has to be done here: https://github.com/kev007/ARTFUL/blob/master/server/src/main/webapp/resources/js/app.js#L179-L201

Add the corpus size to the answer from the /freqs interface

see: http://burnignorance.com/database-tips-and-tricks/physical-mysql-what-are-these-frm-myi-myd-need-this/

Don't know why the error is thrown. Everything still works?

When selecting outgoing references, Zimbabwe has no layer. See:

Map colors are different to the legend colors. See Germany and 5,700,000
Most probably due to style fillOpacity: 0.7

Add Columns:

• Size of the Corporas (100k, 1M, 3M, 10M)
• Source (Web, Wikipedia, News)
• Category (City, Country, Politician, Athlete, ...)

We only have four translations for Russian cities/regions:

22213	614429	Nordwestrussland	deu-ch	Russia
22326	614414	Zentralrussland	deu-ch	Russia
28336	614414	Zentralrussland	deu-at	Russia
28474	301324	Südrussland	deu-at	Russia

Aren't there more in dbpedia?

When selecting a country do not use the references from this country on itself for the legend, just colorize it in an other color.

If a country is clicked a table should be shown listing the countries which reference the clicked country. The task is to write a script that makes this table in the database

(Already partially implemented)
Reevaluate if the linear scale is appropriate for the coloring.

The legend is empty if a selected country has no outgoing references (e.g. China). See:

Add mapping from multiple languages to one country, see: #42 (comment)

e.g. newscrawl, web, wikipedia

• modify database structure to support saving of source

Soon we will be implementing multiple translation categories (Athletes, Politicians, ...) and the parser must be able to distinguish between the categories and then save it in the database.

• Filename = Category
• New Column in the database

As you can see below, there are different frequency values for one country when:

1. Selecting Outgoing References
2. Click on a country
3. Click on another country, without deselecting the first one

Also it seems that the country name is lowercase everytime in this case.

Beautify the Ingoing/Outgoing References Radio Buttons with a toggle switch or something else. Here are some examples: http://demos.jquerymobile.com/1.4.1/flipswitch/ Something else than jquery is also possible.

Find out why only these 2 languages get transferred from csv to db

• compress large files (country-data, country-references, ...)
• spinning loader

• Download the entirety of the Leipzig Corpora: http://corpora2.informatik.uni-leipzig.de/download.html
• Read and Parse the CSV files
• Batch import and read the MySQL dump files
• Fill Database with word frequencies

http://corpora2.informatik.uni-leipzig.de/download.html

Serverside:

• which country refered country X how many times
• list every country country X refered

Save the database path as java opts variable so it can be changed without redeploying the server.

• Aufgabenstellung
• Architektur
  • DB
  • Spring
  • Leaflet
• Features
  • Screenshots der Anwendung
• Live Demo
• Ausblick
  • Quotient anzeigen
  • gerichtete Beziehung zwischen Ländern
  • weitere Kategorien hinzufügen: Städte, Fußballer,...
• Probleme
  • keine vollständigen Korpora
  • Kopora sind Sprachbezogen, nicht Länderbezogen
  • DBPedia
  • SPARQL Einschränkungen

Treffen: Donnerstag 11:00 Uhr

Run the script countryReferences.py to update them.

W_ID, Country references, (for all languages)

• Countries
• Cities / Metropolitan Areas
• States
• Heads of State

Steps to reproduce this bug:

1. Selecting a country
2. change the selected years

After this the selected country has two visual layers (the green one and the one depending on its frequency)

One possible (but maybe not very good) way to solve this issue is to iterate with a python script or whatever over every country and every year we have in the database and create the sql statement like this:

CREATE VIEW top_ten_words AS
  SELECT *
  FROM
    (SELECT
       sum(f.freq) total_freq,
       t.w_id,
       t.word,
       t.located_in,
       f.year
     FROM freq f, translation t
     WHERE f.translation_id = t.id AND t.located_in = 'Germany' AND year = 2014
     GROUP BY t.w_id, t.located_in, f.year
     ORDER BY total_freq
       DESC
     LIMIT 10)
  UNION
  SELECT *
  FROM
    (SELECT
       sum(f.freq) total_freq,
       t.w_id,
       t.word,
       t.located_in,
       f.year
     FROM freq f, translation t
     WHERE f.translation_id = t.id AND t.located_in = 'United States' AND year = 2014
     GROUP BY t.w_id, t.located_in, f.year
     ORDER BY total_freq
       DESC
     LIMIT 10);

If the generation of this view takes too long, we can just create a table instead.