Comments (5)
It looks like those columns are not in your data set. Since you're trying to drop them, it shouldn't matter.
You could try replacing the drop code with this:
df.drop(columns=obsolete_columns, inplace=True, errors='ignore')
This will tell pandas to ignore the error that's being raised because the columns are not in the DataFrame.
from pbpython.
Hi Chris
I've run the example code as provided w/o mods. Ran your suggested change (line 28) and got another error as seen below. I'm not an experienced "Pandite", but rely only on the provided code.
Thanks for your help
'Finished file: import\produkt_tu_stunde_20190409_20201231_00096.txt'
'This is file 10'
'Shape of the main_df is: (851261, 1)'
KeyError Traceback (most recent call last)
~\Anaconda3\envs\tide\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
3079 try:
-> 3080 return self._engine.get_loc(casted_key)
3081 except KeyError as err:
pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
pandas_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
KeyError: 'MESS_DATUM'
The above exception was the direct cause of the following exception:
KeyError Traceback (most recent call last)
in
27 # df.drop(columns=obsolete_columns, inplace=True)
28 df.drop(columns=obsolete_columns, inplace=True, errors='ignore')
---> 29 df["MESS_DATUM"] = pd.to_datetime(df["MESS_DATUM"], format="%Y%m%d%H")
30 df = df[df['MESS_DATUM']>= "2007-01-01"]
31 df.set_index(['MESS_DATUM', 'STATIONS_ID'], inplace=True)
~\Anaconda3\envs\tide\lib\site-packages\pandas\core\frame.py in getitem(self, key)
3022 if self.columns.nlevels > 1:
3023 return self._getitem_multilevel(key)
-> 3024 indexer = self.columns.get_loc(key)
3025 if is_integer(indexer):
3026 indexer = [indexer]
~\Anaconda3\envs\tide\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
3080 return self._engine.get_loc(casted_key)
3081 except KeyError as err:
-> 3082 raise KeyError(key) from err
3083
3084 if tolerance is not None:
KeyError: 'MESS_DATUM'
from pbpython.
Hmm. I'm not sure what't going on. It's likely there's an error earlier in the script and the files are downloaded or processed properly. You should try to look at the downloaded files and make sure they are placed in the correct directories and have the right content.
I realize that's a little vague for a new user but I think its likely something changed and the files are stored differently.
from pbpython.
Hi Chris
Here's a snippet from the imported file list. All have a MESS_DATUM column. Is MESS_DATUM format the problem? It is rejected by df["MESS_DATUM"] = pd.to_datetime(df["MESS_DATUM"], format="%Y%m%d%H")
17 2 Dir(s) 434,812,313,600 bytes...
STATIONS_ID MESS_DATUM QN_9 TT_TU RF_TU eor
0 3 1950040101 5 5.7 83.0 eor
1 3 1950040102 5 5.6 83.0 eor
2 3 1950040103 5 5.5 83.0 eor
3 3 1950040104 5 5.5 83.0 eor
4 3 1950040105 5 5.8 85.0 eor
from pbpython.
I re-ran this on my local machine and the file I see looks like yours so I think the date format is ok.
Is it possible that there is an extra file in your import directory? Look at each of the files and make sure they are all formatted the same.
from pbpython.
Related Issues (20)
- Typo causes fail in Ipython-pandas-tips-and-tricks.ipynb HOT 1
- Typos in Forecasting-with-prophet.ipynb causes missing file error HOT 1
- rounding error
- Fix pathlib article based on these comments.
- AttributeError: 'Book' object has no attribute 'extract_formulas' HOT 2
- Handling Missing values
- Slightly idiosyncratic usage of groupby HOT 2
- Needing to consider the case of an empty list HOT 2
- Error in pathlib cheatsheet HOT 1
- df.ix need to replace with df.loc HOT 1
- Syntax error on any command using HOT 3
- Update record-linking.html with new library
- Viewing Notebooks
- Unable to show full path as string HOT 1
- About Monte Carlo Simulation HOT 3
- analyze_ppt(input, output) function not finding all layouts
- Table on pd.DataFrame from List and Dictionaries is wrong
- The very first post from the RSS feed is not available on the website HOT 4
- The Reverse of a Loan Payment Schedule Calculator
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pbpython.