barthoekstra / knmy Goto Github PK

When I try using this example code I get the following error.
Am I doing something wrong or is this script out of date?

Traceback (most recent call last):
  File "W:\Some_map\knmi.py", line 3, in <module>
    disclaimer, stations, variables, data = knmy.get_hourly_data(stations=[209, 235], start=2022061715, end=2022061716, inseason=True, variables=['WIND', 'TEMP'], parse=True)
  File "C:\Users\some_name\AppData\Local\Programs\Python\Python39\lib\site-packages\knmy\knmy.py", line 214, in get_hourly_data
    return get_knmi_data('hourly', stations=stations, start=start, end=end, variables=variables, inseason=inseason,
  File "C:\Users\some_name\AppData\Local\Programs\Python\Python39\lib\site-packages\knmy\knmy.py", line 136, in get_knmi_data
    return parse_raw_weather_data(r.text)
  File "C:\Users\some_name\AppData\Local\Programs\Python\Python39\lib\site-packages\knmy\parser.py", line 31, in parse_raw_weather_data
    stations_raw = next(chunk)
StopIteration

Great tool! I successfully used your package to download daily KNMI data for all avalaible stations using just a few lines of code. I have some suggestions which might offer improvements:

• Add warnings. For example, I tried to download time series including the period 1901-2022, which resulted in the following error:
  

Apparently, the KNMI API does not support retrievals of this length: 'De query geeft te veel resultaten terug. Pas uw parameters aan!' You could add a warning message when this message is returned by the api call.

• Add multi-dimensional array (data cube) support using e.g. xarray:
  

I might be able to contribute if you are interested in adding this functionality.

Best,
Michiel

Your tool is excellent!

There appears to have been a change in the way the data is delivered by KNMI. Date and time columns are repeated, leading the parser to fail.

In the parser, I modified the code as follows (line 61):
# We've reached the header of the CSV section with data
raw_header = line.strip('# ').replace(' ', '')
raw_header = raw_header.split(',')

            # Remove duplicate headers to avoid error
            header = []
            for item in raw_header:
                if item not in header:
                    header.append(item)

An additional change is in the way the headers are specified when creating the dataframe:
data = pd.read_csv(StringIO('\n'.join(records)), names=header)

The split of the string is not required here anymore.

Hope this helps.