abaumg / covid19-bz-scraper Goto Github PK

If I read http://www.provinz.bz.it/news/de/news.asp?news_action=4&news_article_id=637196 correctly, the column deceased_total from the new detailed data files area not actually the total number of deceased persons, but the number of people deceased in a hospital. Some persons might be counted in this value and the value of deceased_senior_residents.

Probably better rename column deceased_total to deceased_in_hospital (or similar)?

Hi,

First of all thank you a lot for this repo, really saved a lot of work to process the data for municipalities for me.

Though a question, I noticed that the municipality data hasn't been updated for this week. Same seems to have happened to the detailed province data for the last 2 months.
Is this intentional or did some automated job die without you noticing? From what I can tell the data source (https://afbs.provinz.bz.it/upload/coronavirus/CoronavirusPositivi.xlsx) is still getting daily updates. If so, do you plan to fix/update it or should I instead just set up my own import?

Best regards,
Max

It would be nice to have automatic updates for the data files. This could be achieved by a cronjob (shell or python script), but I'm not sure what would be the best way to commit the changes by a script. Any suggestions?

When I run process_municipalities_singleday.py on my computer, I get the following warning:

(venv) andreas@omena:~/covid19-bz-scraper$ python process_municipalities_singleday.py 
/home/andreas/covid19-bz-scraper/venv/lib/python3.6/site-packages/pandas/core/indexing.py:966: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
process_municipalities_singleday.py:63: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_totale['datum'] = date

The data file /data/covid19_bz_municipalities.csv is updated nevertheless. Can I safely ignore the warning?

I would find it interesting to have this data also per municipality. Maybe similar to how the protezione civile publishes the data per province.

This data would be available through the daily press releases, e.g.:

• http://www.provinz.bz.it/news/de/news.asp?news_action=300&news_image_id=1062450 from http://www.provinz.bz.it/news/de/news.asp?news_action=4&news_article_id=637083

@jalaska already has some python code for converting the shreadsheet to more machine-readable data format, which could potentially be used for this.

Unfortunately, these spreadsheets seem to be accessible only through these daily press releases, and not through sites like http://www.provinz.bz.it/sicherheit-zivilschutz/zivilschutz/coronavirus.asp. This would make the automatic scraping a bit cumbersome 🤔 . But maybe there's a secondary source for this, which I don't know?