Problem generating bulk trajectories with ERA5 data about pysplit HOT 22 OPEN

Are your meteorology file names properly formatted? monYY*# or monYYYY*#

from pysplit.

from pysplit.

Hi @AngelaAmpuero, sorry to dive in here as I am struggling to properly use ERA5 data as meteo source for running backtrajectories with HYSPLIT/PySPLIT.

I assume you succeeded at this attempt, but I wonder:

• What variables from ERA5 did you download to be able to run HYSPLIT?
• Did you use data from all pressure levels?
• I assume you downloaded the files in GRIB format then converted them to ARL via a conversion tool. Could you point me towards which tool you used?

Thanks a lot! (and if there is any resource/tutorial on how to deal with this specifically with ERA5 data, I'd be really happy to know :) )

from pysplit.

from pysplit.

Hi @AngelaAmpuero!

Thanks a lot for your quick response! And sorry for my late reply, I am only now noticing your response 😅

In the end I managed to work with weekly GDAS1 files, although I would like to be able to use ERA5 data in the future. Since GDAS data is already in ARL format, it was rather easy to get it working once you I had everything properly stored in the meteo folder.

I used exactly the scripts by mscross. This means that I downloaded all
variables within the script and all pressure levels.

However, I was wondering if there'd be any way to specify where the scripts you mention here are, as I don't know which ones you mean!

In any case, thanks and kind regards!

from pysplit.

from pysplit.

I can see it now, I think I should be able to repurpose those scripts to do what I want!

I don't think GitHub supports attachments in comments even when responding through email, so probably they just got lost in the way in the previous comment.

Gracias @AngelaAmpuero 😁

from pysplit.

Hi Alejandro, I used exactly the scripts by mscross. This means that I downloaded all variables within the script and all pressure levels. I downloaded the data in GRIB format and used the ERA52ARL tool to make the conversion. I should point out that I use global data and download each month separately. This can be automated with a bash script. Here I attach the exact files that I use, all of them in the same directory. Finally, there are some things you should consider: -The commented part I run separately as indicated in the script. -I create daily files, then I merge them in groups of 8 days per month (last week has less days depending on the month) -When you generate your final data, avoid the .ARL extension, otherwise you will get an error. I hope this helps, and if anything, let me know, Cheers, Angela El mar, 22 feb 2022 a las 13:00, Alejandro Fontal @.>) escribió:
…
Hi @AngelaAmpuero https://github.com/AngelaAmpuero, sorry to dive in here as I am struggling to properly use ERA5 data as meteo source for running backtrajectories with HYSPLIT/PySPLIT. I assume you succeeded at this attempt, but I wonder: - What variables from ERA5 did you download to be able to run HYSPLIT? - Did you use data from all pressure levels? - I assume you downloaded the files in GRIB format then converted them to ARL via a conversion tool. Could you point me towards which tool you used? Thanks a lot! (and if there is any resource/tutorial on how to deal with this specifically with ERA5 data, I'd be really happy to know :) ) — Reply to this email directly, view it on GitHub <#84 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWJR2ZYAC772H7Q5FG4CPODU4OXINANCNFSM5HD722GA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.>

Hi Angela,

I was wondering how you converted daily .ARL files to weekly / combined 8 files into one?

Thank you,
Ruth

from pysplit.

from pysplit.

Hi Angela,

Thank you so much for your quick reply! I combined 5 daily files as you have done above and wanted to test out to see if they work ok. When I ran the file as a single back trajectory using the GUI I got this error - did you get this too or not?

Model started ...
HYSPLIT - Initialization
HYSPLIT version: hysplit.v5.2.0
Last Changed Date: 2022-01-16
ERROR metset: meteorological data time interval varies. Check input meteo file.
Changed from 60 min to 24540 min
At day/hr 20 0
23

Ruth

from pysplit.

from pysplit.

ok cool thank you - out of interest how big were your ARL daily files?

from pysplit.

from pysplit.

I got it working, thank you ! : )

from pysplit.

from pysplit.

I found this so helpful thank you!!

If the ARL files aren't that big, just wondering if anyone has any converted ERA5 weekly/monthly files that you could share?

I'm currently downloading June 2021 global and after a whole day I'm not even half way through. Eventually I need to have at least a complete year (any complete year between 2016 and 2021 will do), and just hoping to speed things up a little.

Thanks again!

from pysplit.

I got it working, thank you ! : )

How did you get it working? I merged 9th - 16th Jun 2021 and tried to run a trajectory, but I also get the error:

HYSPLIT - Initialization
HYSPLIT version: hysplit.v5.2.1
Last Changed Date: 2022-05-16
ERROR metset: meteorological data time interval varies. Check input meteo file.
Changed from 60 min to -300 min
At day/hr 15 18
23

Also, my week-long merged ARL file is 53.23 GB global..

from pysplit.

I got it working, thank you ! : )

HYSPLIT - Initialization
HYSPLIT version: hysplit.v5.3.0
Last Changed Date: 2023-11-07
ERROR metset: meteorological data time interval varies. Check input meteo file.
Changed from 60 min to -4260 min
At day/hr 15 0
23

FileNotFoundError Traceback (most recent call last)
File ~/anaconda3/lib/python3.11/shutil.py:825, in move(src, dst, copy_function)
824 try:
--> 825 os.rename(src, real_dst)
826 except OSError:

FileNotFoundError: [Errno 2] No such file or directory: 'DY_3_ERA5mar0500spring2018031600' -> '/Users/lixinhao/Desktop/GTRA/trajectories/DY_3_ERA5/DY_3_ERA5mar0500spring2018031600'

During handling of the above exception, another exception occurred:

FileNotFoundError Traceback (most recent call last)
Cell In[10], line 1
----> 1 pysplit.generate_bulktraj(basename, working_dir, storage_dir, meteo_dir,
2 years, months, hours, altitudes, location, runtime,
3 monthslice=slice(15, 21, 1), hysplit='/Users/lixinhao/Desktop/hysplit/exec/hyts_std')

File ~/anaconda3/lib/python3.11/site-packages/pysplit/trajectory_generator.py:176, in generate_bulktraj(basename, hysplit_working, output_dir, meteo_dir, years, months, hours, altitudes, coordinates, run, meteoyr_2digits, outputyr_2digits, monthslice, meteo_bookends, get_reverse, get_clipped, hysplit)
173 _cliptraj(output_cdir, trajname)
175 # Move the trajectory file to output directory
--> 176 shutil.move(trajname, final_trajpath)
178 # Revert current working directory
179 finally:
180 os.chdir(cwd)

File ~/anaconda3/lib/python3.11/shutil.py:845, in move(src, dst, copy_function)
843 rmtree(src)
844 else:
--> 845 copy_function(src, real_dst)
846 os.unlink(src)
847 return real_dst

File ~/anaconda3/lib/python3.11/shutil.py:436, in copy2(src, dst, follow_symlinks)
434 if os.path.isdir(dst):
435 dst = os.path.join(dst, os.path.basename(src))
--> 436 copyfile(src, dst, follow_symlinks=follow_symlinks)
437 copystat(src, dst, follow_symlinks=follow_symlinks)
438 return dst

File ~/anaconda3/lib/python3.11/shutil.py:256, in copyfile(src, dst, follow_symlinks)
254 os.symlink(os.readlink(src), dst)
255 else:
--> 256 with open(src, 'rb') as fsrc:
257 try:
258 with open(dst, 'wb') as fdst:
259 # macOS

FileNotFoundError: [Errno 2] No such file or directory: 'DY_3_ERA5mar0500spring2018031600'

Hello! I encountered the similar problem, could you give me some help, and how do you solve that problem finally.

from pysplit.

Great!!
…
On Thu, Jul 13, 2023 at 12:44 PM Ruth Moore @.> wrote: I got it working, thank you ! : ) — Reply to this email directly, view it on GitHub <#84 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWJR2ZZOMHNU2FDUMRNOSW3XQAJXJANCNFSM5HD722GA . You are receiving this because you were mentioned.Message ID: @.>

I got it working, thank you ! : )

How did you get it working? I merged 9th - 16th Jun 2021 and tried to run a trajectory, but I also get the error:

HYSPLIT - Initialization HYSPLIT version: hysplit.v5.2.1 Last Changed Date: 2022-05-16 ERROR metset: meteorological data time interval varies. Check input meteo file. Changed from 60 min to -300 min At day/hr 15 18 23

Also, my week-long merged ARL file is 53.23 GB global..

Do you solve this problem, I would really appreciate it if you would like to provide some help, thanks.

from pysplit.

I didn't get that error message. But if it helps, my data was daily and every 6 hours.
…
On Tue, Jul 11, 2023, 19:47 Ruth Moore @.> wrote: Hi Angela, Thank you so much for your quick reply! I combined 5 daily files as you have done above and wanted to test out to see if they work ok. When I ran the file as a single back trajectory using the GUI I got this error - did you get this too or not? Model started ... HYSPLIT - Initialization HYSPLIT version: hysplit.v5.2.0 Last Changed Date: 2022-01-16 ERROR metset: meteorological data time interval varies. Check input meteo file. Changed from 60 min to 24540 min At day/hr 20 0 23 Ruth — Reply to this email directly, view it on GitHub <#84 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWJR2Z5LY6EH2IUA7TTKAYLXPXJZBANCNFSM5HD722GA . You are receiving this because you were mentioned.Message ID: @.>

HYSPLIT - Initialization
HYSPLIT version: hysplit.v5.3.0
Last Changed Date: 2023-11-07
ERROR metset: meteorological data time interval varies. Check input meteo file.
Changed from 60 min to -4260 min
At day/hr 15 0
23

FileNotFoundError Traceback (most recent call last)
File ~/anaconda3/lib/python3.11/shutil.py:825, in move(src, dst, copy_function)
824 try:
--> 825 os.rename(src, real_dst)
826 except OSError:

FileNotFoundError: [Errno 2] No such file or directory: 'DY_3_ERA5mar0500spring2018031600' -> '/Users/lixinhao/Desktop/GTRA/trajectories/DY_3_ERA5/DY_3_ERA5mar0500spring2018031600'

During handling of the above exception, another exception occurred:

FileNotFoundError Traceback (most recent call last)
Cell In[10], line 1
----> 1 pysplit.generate_bulktraj(basename, working_dir, storage_dir, meteo_dir,
2 years, months, hours, altitudes, location, runtime,
3 monthslice=slice(15, 21, 1), hysplit='/Users/lixinhao/Desktop/hysplit/exec/hyts_std')

File ~/anaconda3/lib/python3.11/site-packages/pysplit/trajectory_generator.py:176, in generate_bulktraj(basename, hysplit_working, output_dir, meteo_dir, years, months, hours, altitudes, coordinates, run, meteoyr_2digits, outputyr_2digits, monthslice, meteo_bookends, get_reverse, get_clipped, hysplit)
173 _cliptraj(output_cdir, trajname)
175 # Move the trajectory file to output directory
--> 176 shutil.move(trajname, final_trajpath)
178 # Revert current working directory
179 finally:
180 os.chdir(cwd)

File ~/anaconda3/lib/python3.11/shutil.py:845, in move(src, dst, copy_function)
843 rmtree(src)
844 else:
--> 845 copy_function(src, real_dst)
846 os.unlink(src)
847 return real_dst

File ~/anaconda3/lib/python3.11/shutil.py:436, in copy2(src, dst, follow_symlinks)
434 if os.path.isdir(dst):
435 dst = os.path.join(dst, os.path.basename(src))
--> 436 copyfile(src, dst, follow_symlinks=follow_symlinks)
437 copystat(src, dst, follow_symlinks=follow_symlinks)
438 return dst

File ~/anaconda3/lib/python3.11/shutil.py:256, in copyfile(src, dst, follow_symlinks)
254 os.symlink(os.readlink(src), dst)
255 else:
--> 256 with open(src, 'rb') as fsrc:
257 try:
258 with open(dst, 'wb') as fdst:
259 # macOS

FileNotFoundError: [Errno 2] No such file or directory: 'DY_3_ERA5mar0500spring2018031600'

I encountered this problem, and my ERA5 data is 0.25*0.25, hourly data downloaded as you recommended by hysplit_metdata. I also combined them through a similar .sh script as you did. Do you believe my data format may not be good for Pysplit package or any other problems? Hope for your reply, and I would really appreciate it, thank you so much.

from pysplit.