Preproceesing eICU issue about tpc-los-prediction HOT 3 OPEN

Hey Seham - I ran into the same issues and it appears due to missing values in the lab csv... Have you find out a way to trouble shoot it? many thanks!

Update: fixed the issue after downgraded python form 3.9 to 3.6 and panda from higher version to the one specified by author.

Another update: turns still having issues with lower version of python/panda:

==> Processed 144000 patients...
==> Processed 148000 patients...
Traceback (most recent call last):
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'patient'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/why217/Desktop/UIUC/cs598/TPC-LoS-prediction/eICU_preprocessing/run_all_preprocessing.py", line 18, in
timeseries_main(eICU_path, test=False)
File "/Users/why217/Desktop/UIUC/cs598/TPC-LoS-prediction/eICU_preprocessing/timeseries.py", line 229, in timeseries_main
further_processing(eICU_path, test)
File "/Users/why217/Desktop/UIUC/cs598/TPC-LoS-prediction/eICU_preprocessing/timeseries.py", line 205, in further_processing
processed_timeseries.set_index('patient', inplace=True)
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/site-packages/pandas/core/frame.py", line 4178, in set_index
level = frame[col]._values
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/site-packages/pandas/core/frame.py", line 2927, in getitem
indexer = self.columns.get_loc(key)
File "/Users/why217/opt/miniconda3/envs/TPS/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'patient'

Final update:
---finally figured out it's still the python/panda version issues with the new M1 MacBook. Everything resolved after I made a new env of python 3.6 and pip installed all requirements.

from tpc-los-prediction.

@hanyin88 I have downgraded my Python version but getting errors around installing torch=1.5.0. Can you share the final package versions that you have used to fix this issue please? Thanks.

from tpc-los-prediction.

Ok, these versions of packages worked for me(atleast for pre-processing):

requirements.txt:

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
absl-py=1.4.0=pypi_0
aiohttp=3.8.6=pypi_0
aiosignal=1.2.0=pypi_0
apscheduler=3.6.3=pypi_0
async-timeout=4.0.2=pypi_0
asynctest=0.13.0=pypi_0
attrs=22.2.0=pypi_0
backports-zoneinfo=0.2.1=pypi_0
cachetools=4.2.2=pypi_0
captum=0.2.0=pypi_0
certifi=2024.2.2=pypi_0
charset-normalizer=2.0.12=pypi_0
click=8.0.4=pypi_0
cloudpickle=3.0.0=pypi_0
colorama=0.4.6=pypi_0
colorlover=0.3.0=pypi_0
contourpy=1.1.1=pypi_0
cycler=0.10.0=pypi_0
dataclasses=0.8=pypi_0
decorator=4.4.2=pypi_0
dill=0.3.4=pypi_0
flask=2.0.3=pypi_0
fonttools=4.51.0=pypi_0
frozenlist=1.2.0=pypi_0
grpcio=1.48.2=pypi_0
idna=3.6=pypi_0
idna-ssl=1.1.0=pypi_0
imageio=2.15.0=pypi_0
importlib-metadata=7.1.0=pypi_0
importlib-resources=6.4.0=pypi_0
itsdangerous=2.0.1=pypi_0
jinja2=3.0.3=pypi_0
joblib=1.4.0=pypi_0
jsonpatch=1.32=pypi_0
jsonpointer=2.3=pypi_0
kiwisolver=1.4.5=pypi_0
llvmlite=0.36.0=pypi_0
markdown=3.3.7=pypi_0
markupsafe=2.0.1=pypi_0
matplotlib=3.2.2=pypi_0
multidict=5.2.0=pypi_0
multiprocess=0.70.12.2=pypi_0
networkx=2.5.1=pypi_0
numba=0.53.1=pypi_0
numpy=1.18.1=pypi_0
packaging=24.0=pypi_0
pandas=0.24.2=pypi_0
pathos=0.2.8=pypi_0
pillow=10.3.0=pypi_0
pip=21.3.1=pyhd8ed1ab_0
plotly=5.18.0=pypi_0
portalocker=2.7.0=pypi_0
pox=0.3.0=pypi_0
ppft=1.6.6.4=pypi_0
protobuf=3.19.6=pypi_0
pyparsing=3.1.2=pypi_0
python=3.6.15=h39d44d4_0_cpython
python-dateutil=2.9.0.post0=pypi_0
python-graphviz=0.19.1=pypi_0
python-telegram-bot=13.12=pypi_0
python_abi=3.6=2_cp36m
pytz=2024.1=pypi_0
pytz-deprecation-shim=0.1.0.post0=pypi_0
pywin32=305=pypi_0
requests=2.27.1=pypi_0
scikit-learn=0.20.2=pypi_0
scipy=1.4.1=pypi_0
seaborn=0.11.2=pypi_0
setuptools=58.0.4=py36ha15d459_2
shap=0.35.0=pypi_0
six=1.16.0=pypi_0
slackclient=2.9.4=pypi_0
slicer=0.0.7=pypi_0
tb-nightly=1.14.0a20190523=pypi_0
tenacity=8.2.2=pypi_0
threadpoolctl=3.4.0=pypi_0
torch=1.10.2=pypi_0
torchvision=0.11.3=pypi_0
tornado=6.1=pypi_0
tqdm=4.66.2=pypi_0
trixi=0.1.2.2=pypi_0
typing-extensions=4.11.0=pypi_0
tzdata=2024.1=pypi_0
tzlocal=4.2=pypi_0
ucrt=10.0.22621.0=h57928b3_0
umap-learn=0.4.6=pypi_0
urllib3=1.26.18=pypi_0
vc=14.3=hcf57466_18
vc14_runtime=14.38.33130=h82b7239_18
visdom=0.2.4=pypi_0
vs2015_runtime=14.38.33130=hcb4865c_18
websocket-client=1.3.1=pypi_0
werkzeug=2.0.3=pypi_0
wheel=0.37.1=pyhd8ed1ab_0
yarl=1.7.2=pypi_0
zipp=3.18.1=pypi_0

from tpc-los-prediction.