It would be nice to have an animated map with the new cases per municipality. Is anyone interested in implementing?

It might be interesting to use the shapes of rivm: https://www.rivm.nl/media/bubbleMap/shapes/gemeente-2019.geojson

Hi all,

First of all, thank you all for contributing to this repository. It is a challenge to keep up-to-date with all the daily changes on the RIVM website. Without your contributions, it isn't possible to keep this up-to-date and add new datasets.

There are quite some questions regarding our contacts with RIVM and our offer to help them set up a proper data repository. We, CoronaWatchNL and Utrecht University, had contact with several people of RIVM. Unfortunately, there is no progress yet. More and more (prominent) researchers are teaming up with this project. They help us convincing RIVM to set up a data repository and use our/your expertise.

If there is anything to share, please drop it in this thread or send me an email at [email protected].

Hi Jonathan,

After watching this epic video by 3blue1brown https://www.youtube.com/watch?v=Kas0tIxDvrg I thought you could perhaps fit a sigmoid line as well as the exponential.

Or even better (will probably only make sense if you've seen the video)

1. calculate the change per time step (current value - previous value)
2. calculate the growth rate (ratio of the current and previous change)
3. estimate when the above will be 1 (with another kind of fit?)
4. fit the sigmoid with that time as the inflection point 😲?

Last commit deleted the data from 2020-03-31 (12595)

a450b7c#commitcomment-38181311

Hi all,

When I try to import the data into Python with Pandas, I keep hitting the same error:

ParserError: Error tokenizing data. C error: Expected 1 fields in line 49, saw 2

Any suggestions how I can by pass this? Might be an easy question to answer but I haven't been able to find a solution that actually works :(

Since 3 March, RIVM reports the number of diagnoses with the coronavirus and their municipality of residence on a daily basis. This issue is used to reconstruct the data of March 1 based on RIVM media reports.

Result:

• 1 person from Tilburg
• 1 person from Diemen
• ...

i have made some graphs in ArcGIS Pro, maybe intresting to generate on daily basis (and not in ArcGIS)?

working on a api endpoint that trows a html folium map
todo:

• timeslider
• geojson & style function instead of choropleth map
• chart popup
• splitmap
• data annotation

preview:
http://hazedd.pythonanywhere.com/coronaWatchNL/map

last 3 days:

last 7 days:

last 14 days:

It still shows points until the 13th, should it be included in some python pipe to update it as well? :).

I've discovered RIVM recently published data on COVID-19 cases by sex and age, and also a distribution by GGD district:
https://en.wikipedia.org/wiki/2020_coronavirus_pandemic_in_the_Netherlands

Any plans to incorporate the data in the repository?

The following link does not work ATM: https://github.com/J535D165/CoronaWatchNL/blob/master/data/rivm_NL_covid19_national_by_date.csv If I click on it I get a 404.

It appears in the datasets table on the repository homepage with the name "COVID-19 case counts by date in NL (NEW)"

On 30 March, we had 310 missing municipalities. Based on the RIVM report, there are no missing provinces today. Is this possible? Are they imputing with the GGD region? Or do they no longer include missing locations? That would be interesting. Any thoughts on this?

12595 - (126 + 166 + 130 + 1475 + 161 + 1426 + 3412 + 1845 + 698 + 1046 + 161 + 1949) = 0

Does anyone know what the difference is between the intakeCount in the nice dataset and the number of cases in the lcps dataset? In the media they use the lcps data and say it is the number of corona patients in the icu. The Rivm daily epidemiological report calls it the confirmed covid-19 cases. Does this mean that the lcps data also includes suspected cases?

The maps for hospitalizations and infections look very different. With hospitalizations the big cities light up:

With infections the areas where Corona is most widespread ATM light up:

I first thought that the hospitalizations are recorded for the municipality in which the hospital is located. But this is not possible, because there are municipalities with no hospital that have a non-zero number.

What is the best way to interpret the number of hospitalizations?

The RIVM reports nicely show the data that is added per day, to make it clear that there is a significant lag in the reporting. However I did not find the source data published anywhere, a set that shows the reported cases/deaths/hospitalizations per reporting date and per date.

I took the liberty of extracting the data from the pdf reports. Data is here

Please note that no individual reports are available before march 27th, so this shows in the graphs below.

interactive

interactive

So now I made this, don't really know what to do with it. I can make a PR of course, but not sure how to structure the data for in here. Updating this requires a few minutes to export the data from the pdf graphs. Also I only did this for hospitalizations, not for the deaths/confirmed cases yet

I noticed that test data is last updated on the 20th and have not been updated since then. Also noticed that the format in the pdf was slightly changed to weekly so not sure if this was the cause but might be related. Thanks!

Some of the plot titles, such as "Voorspelling aantal Coronavirus besmettingen" may be misleading when read without context. They aren't actual predictions but rather the result of very simple fits. I've adjusted them in an upcoming pull request.

Since 3 March, RIVM reports the number of diagnoses with the coronavirus and their municipality of residence on a daily basis. This issue is used to reconstruct the data of February 27 based on RIVM media reports.

• https://www.dvhn.nl/extra/Eerste-besmetting-coronavirus-in-Nederland-25399601.html?harvest_referrer=https%3A%2F%2Fwww.google.com%2F

Result:

• 1 person from Tilburg

Can https://raw.githubusercontent.com/J535D165/CoronaWatchNL/master/plots/timeline.png also show 1) the total number of tests (first test positive + first test negative) and 2) the number of people that have recovered and subsequently tested negative (confirmation test negative).

Do you have any suggestions how to improve the simulations of the sigmoid fitting? Currently it samples known data points (with replacement), but the range is almost certainly way too low.

If have also tried inferring data for the two next days, sampled from the distribution of the growth factor of the last five days. But I doubt the resulting image is still useful/readable:

Any suggestions are very welcome.

Since 3 March, RIVM reports the number of diagnoses with the coronavirus and their municipality of residence on a daily basis. This issue is used to reconstruct the data of February 29 based on RIVM media reports.

• https://www.rivm.nl/nieuws/twee-contacten-testen-positief-voor-covid-19
• https://www.rivm.nl/nieuws/ook-enkele-contacten-van-tilburgse-patient-testten-positief-voor-covid-19
• https://www.rivm.nl/nieuws/patient-met-covid-19-in-delft

Result:

• 3 persons from Tilburg
• 3 persons from Diemen
• 1 from Delft

Hi Jonathan,

As discussed by email, you can share my Power BI report for RIVM Corona dataset on your GitHub page, maybe with a reference to my blog-post (where I just uploaded v2 of the report)

https://worktimesheet2014.blogspot.com/2020/03/coronovirus-in-netherlands-power-bi.html
http://tiny.cc/ncwblz

regards,

Maarten van Reek
[email protected]

There is data on COVID-19 case counts on the municipality level available again. We can reactivate https://github.com/J535D165/CoronaWatchNL/blob/master/data/rivm_corona_in_nl.csv. See issue #44. I'm not sure whether this was by mistake or something they will provide on a daily basis. See https://github.com/J535D165/CoronaWatchNL/blob/master/raw_data/peildatum-08-04-2020-13-55.csv

In the Github Actions workflow, the rendering of the maps fails. Unknown error to me, it works locally. It might be an idea to replace this by Python code.

Something went wrong in making the following data files:

• rivm_NL_covid19_total_municipality.csv
• rivm_NL_covid19_hosp_municipality.csv

There are double municipality records for the date 2020-04-17 and 2020-04-18.

Hi,

rivm_corona_in_nl_fatalities.csv data contains two values of data for 27th of April. I guess the last row should contain 28th April data.

Best regards

I think it is quite doable to extract the color values from the maps in an automated way.

Outline:

• Screenshot/image the relevant pdf page.
• Compute the center of mass of each municipality with CBS 2019 Wijk en Buurtkaart maps
• Make this an overlay for the image.
• Extract the color values and map them to the corresponding values.

Is someone interested in giving this one a try?

is het een idee om een website te genereren aan de hand van alle data/grafieken in deze repo?
zoals bijvoorbeeld http://covid19.healthdata.org/

ik zat zelf te denken aan een gatsby site gehost op S3.
met daar op de datasets, grafieken, kaarten etc.

We hebben zelf het domein caard.nl "over" waar vast wel een acronym van te maken is. Maar een ander domein kan natuurlijk ook.

Mocht er interesse zijn kan ik uiteraard een PR sturen.

New datasets have been added. It would be nice to have plots on these numbers as well.

@japhir

On

Compliments with this project!

For many applications that produce maps and/or apply spatial analysis, geocoded COVID-19 data (data with coordinate attributes/columns) would be very helpful. This sounds more complex than it actually is:

• many of the produced data CSVs here, have columns like Municipality (Gemeente)/Province code and or name
• the Dutch government via Kadaster-PDOK provides Open datasets for Administrative Borders (Bestuurlijke Grenzen) with those same names/codes.
• GeoJSON is an ideal format for supplying geospatial data
• there is ample Open Source software to convert/simplify these (GML) datasets. With little search found e.g. this script
• "geocoding" is mainly a matter of JOIN-ing on Municipality/Province names/codes (i.s.o. using Geocoding backends like Nominatim ), possibly GeoPandas can be of help. e.g. https://geopandas.org/mergingdata.html

I hope this triggers interest. Eventually I could foresee some extra/derived GeoJSON files generated from the CSVs under /data/ like under /data/geocoded. Is all data here generated via GitHub Workflows/Actions? Then contributors could also add the required geocoding steps there.

Next step are OpenAPI endpoints from this GeoJSON data. This could be served directly from GitHub. We have a project based on OGC OpenAPI REST standards: pygeoapi where we are working on providing an Open Endpoint for COVID-19 data: https://demo.pygeoapi.io/covid-19/collections?f=html. Some Collections there already serve directly from GitHub repos, like Italy. For NL we use/proxy ESRI Endpoints, but would rather serve directly from a/this GH repo.

In theory I could do the work on this issue, but already quite occupied with the pygeoapi part...

There is a new dataset with fatalities on municipality level. Please see https://github.com/J535D165/CoronaWatchNL/blob/master/data/rivm_NL_covid19_fatalities_municipality.csv

Since 3 March, RIVM reports the number of diagnoses with the coronavirus and their municipality of residence on a daily basis. This issue is used to reconstruct the data of March 1 based on RIVM media reports.

https://www.rivm.nl/nieuws/drie-nieuwe-patienten-met-covid-19

Result:

• 4 x Tilburg
• 3 x Diemen
• 1 x Delft
• 1 x Drenthe (where?)
• 1 x Gorinchem (hospital)

Total 10

Maybe we could write a very short imagemagick script to put all the plots together and resize them?

A couple of records don't have a location (or the sum of all cases doesn't sum up to the reported total). We might want to add an additional row/column to the data with unknown locations.

Thank you all for sending us emails with dashboards, visualizations, and gifs made with CoronaWatchNL data. We love to see them.

At the moment, we are receiving a lot of emails on this and are having a hard time answering these emails. It might be an idea to make a separate page with an overview of these initiatives. (And integrate the Interesting links section of the README. Is there anyone interested in setting this up?

Yesterday Noord-Holland reported 5140 confirmed cases, however today I identify 7459 cases. I think probably something went wrong with data extraction?

For our website (covid-analytics.nl) RIVM and LCPS data trough CoronaWatchNL is used. How would you like me to reference to CoronaWatchNL? Currently all charts that use RIVM and LCPS data trough CoronaWatchNL point to RIVM and LCPS directly. Would a mention in the charts explanation page or in some sort of about us page be acceptable? Or perhaps some sort of CoronaWatchNL logo somewhere on the page?

in file rivm_corona_in_nl.csv i notice some lines which dont specify a "gemeenten". Like e.g. row 18
2020-03-02 |   | -1 |   | 8 -- | -- | -- | -- | --

There a few more of these.

How should i interpret these lines please?

Thanks
PAtrick

In https://raw.githubusercontent.com/J535D165/CoronaWatchNL/master/data/rivm_NL_covid19_municipality_range.csv the number of infections for Amsterdam and Eindhoven is always the same. Even though these numbers are reported in ranges, this still seems a bit suspicious.

These are the rows for the last couple of days, the values in the last two columns are always the same:

Datum,Gemeentenaam,Gemeentecode,Provincienaam,Type,Aantal_min,Aantal_max
2020-04-07,Amsterdam,363,Noord-Holland,Totaal,70.0,115.0
2020-04-07,Eindhoven,772,Noord-Brabant,Totaal,70.0,115.0
2020-04-08,Eindhoven,772,Noord-Brabant,Totaal,70.0,115.0
2020-04-09,Amsterdam,363,Noord-Holland,Totaal,70.0,115.0
2020-04-09,Eindhoven,772,Noord-Brabant,Totaal,70.0,115.0
2020-04-10,Amsterdam,363,Noord-Holland,Totaal,115.0,185.0
2020-04-10,Eindhoven,772,Noord-Brabant,Totaal,115.0,185.0
2020-04-11,Amsterdam,363,Noord-Holland,Totaal,115.0,185.0
2020-04-11,Eindhoven,772,Noord-Brabant,Totaal,115.0,185.0
2020-04-12,Amsterdam,363,Noord-Holland,Totaal,115.0,185.0
2020-04-12,Eindhoven,772,Noord-Brabant,Totaal,115.0,185.0
2020-04-13,Amsterdam,363,Noord-Holland,Totaal,90.0,160.0
2020-04-13,Eindhoven,772,Noord-Brabant,Totaal,90.0,160.0
2020-04-14,Amsterdam,363,Noord-Holland,Totaal,90.0,160.0
2020-04-14,Eindhoven,772,Noord-Brabant,Totaal,90.0,160.0
2020-04-15,Amsterdam,363,Noord-Holland,Totaal,90.0,160.0
2020-04-15,Eindhoven,772,Noord-Brabant,Totaal,90.0,160.0

Seems like RIVM excludes the (single?) person living abroad in their statistics. Therefore, our totals are too high. Need to remove this one from our counts imo.

Hi there,
i created https://corona-map-nl.web.app/
I've separated the new hospital intakes and the new cases, it looks a bit like what you guys made.
You can also click on a municipality for historic data.
And if you click on the country icon you see a complete history nationally.

I've also got quite a lot of more structured data by parsing the RIVM javascript, that way i don't have to deal with the PDF files. I also process the data from NICE and LCPS.

At the moment it takes me about 10 minutes to extract all the data and convert it to the format my app uses. Maybe i can help collecting part of the data?

Also, anybody notice that the chart "Aantal overledenen naar datum van overlijden" at https://www.rivm.nl/coronavirus-covid-19/grafieken has the value 109 for the 25th of march and in the pdf it shows 108 for that date. The total used by RIVM is the total in the PDF, so i assume that's the correct total. I noticed they have a one off error everyday for that date...

Let me know if i can be of help,

Best regards,
Jeroen

Hello Jonathan,

Quick heads up! as you will find on the RIVM website they changed the data from confirmed cases to number of people hospitalized in the municipalities.

Kind regards,
Jim

The readme has become a bit bloated with many plots so it needs some organization. Working on this now.

Hi Jonathan,

First of all: fantastic effort by pulling all of this together. It helps many people from many industries (like mine: publishing) to make the data accessible.

Question about rivm_NL_covid19_total_municipality.csv: the dataset seems to be jumping from 31-mrt to 8-apr. Do you know a way how to get the missing data available? I tried to copy/paste from different dataset but there always seems to be a problem (day count vs. cum. count etc.)

First of all thank you so much for the efforts! It was the best efforts on getting all the numbers together I saw and thank you for that.

Regarding the IC data from NICE... understand you said it was experimental but I noticed that it has stopped updating for 4 days. What's the plan on this? Is there temporary issues on getting the data, or is this dataset being deprecated?

thanks!

Since 3 March, RIVM reports the number of diagnoses with the coronavirus and their municipality of residence on a daily basis. This issue is used to reconstruct the data of February 28 based on RIVM media reports.

• https://www.nu.nl/coronavirus/6033932/tweede-coronabesmetting-in-nederland-patient-in-thuisisolatie-in-diemen.html

Result:

• 1 person from Tilburg
• 1 person from Diemen

Hi,

Before in 'rivm_NL_covid19_hosp_municipality.csv' and 'rivm_NL_covid19_total_municipality.csv' data for unknown municipalities/provinces was reported with a blank row.

For example:
Datum,Gemeentecode,Gemeentenaam,Provincienaam,Provinciecode,Aantal
2020-04-16,-1,,,,431

I identified that in the extract of today (28 April) that this is not reported anymore for both tables.
Datum,Gemeentecode,Gemeentenaam,Provincienaam,Provinciecode,Aantal
2020-04-28,-1,,,,

Is this missing data, or will this the new data format and unknow municipalities won't be reported anymore?

Thanks in advance!

The plotting of the sigmoid fails with the latest data @vmenger

Sys.setlocale(category="LC_ALL", locale = "en_US.UTF-8")
Inflection expected after 15.6 days, at date 13/03/2020 13:00
/Users/jonathan/.pyenv/versions/anaconda3-2018.12/lib/python3.7/site-packages/pandas/plotting/_matplotlib/converter.py:103: FutureWarning: Using an implicitly registered datetime converter for a matplotlib plotting method. The converter was registered by pandas on import. Future versions of pandas will require you to explicitly register matplotlib converters.

To register the converters:
	>>> from pandas.plotting import register_matplotlib_converters
	>>> register_matplotlib_converters()
  warnings.warn(msg, FutureWarning)
Traceback (most recent call last):
  File "/Users/jonathan/Dropbox/Projects/Corona/CoronaWatchNL/python_plots.py", line 287, in <module>
    inflection_y = compute_inflection_cases(df, inflection_x)
  File "/Users/jonathan/Dropbox/Projects/Corona/CoronaWatchNL/python_plots.py", line 158, in compute_inflection_cases
    upper_bound = df[df['Dag'] == math.ceil(inflection_x)].iloc[0]['Aantal']
  File "/Users/jonathan/.pyenv/versions/anaconda3-2018.12/lib/python3.7/site-packages/pandas/core/indexing.py", line 1424, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/Users/jonathan/.pyenv/versions/anaconda3-2018.12/lib/python3.7/site-packages/pandas/core/indexing.py", line 2157, in _getitem_axis
    self._validate_integer(key, axis)
  File "/Users/jonathan/.pyenv/versions/anaconda3-2018.12/lib/python3.7/site-packages/pandas/core/indexing.py", line 2088, in _validate_integer
    raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds
[Finished in 3.5s with exit code 1]