noaa-mdl / gifts Goto Github PK

Tests by a Met Office revealed that the YYGGgg and yyyyMMddhhmmss portions of the <bulletinIdentifier> element can disagree. The former is based on UTC and the latter timestamp can be affected by local time zone settings. This disagreement can be potentially confusing to systems that consume these products. Therefore, the yyyyMMddhhmmss portion of the file name shall always be based on the UTC clock.

Providing elevation always throws an IndexError

GIFTs tagged version 1.3.0 writes to directory in the module installation path which may not be writable by anyone other than the install user.

Steps to reproduce the behavior:

1. Install the GIFTs software.
2. Run a script that invokes the GIFTs library as a different user (at least initially). An error results.

It is expected that the software will complete without error.

Below is example output from our test case. The script 'generate_iwxxm' is our custom command line interface to the GIFTs library.

2021-12-13 19:28:50,837 - gifts.swaDecoder - ERROR - 121 - Unable to load/initialize Skyfield Module.
Traceback (most recent call last):
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/swaDecoder.py", line 112, in init
load = Loader(os.path.join(os.path.dirname(file), '../data'), verbose=False)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/skyfield-1.40-py3.8.egg/skyfield/iokit.py", line 110, in init
os.makedirs(self.directory)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/../data'
Traceback (most recent call last):
File "generate_iwxxm.py", line 75, in
service = generate_iwxxm_service()
File "generate_iwxxm.py", line 39, in init
self.encoders.append((re.compile(r'FN\w\w\d\d\s+[A-Z]{4}\s+\d{6}', re.MULTILINE), gifts.SWA.Encoder()))
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/SWA.py", line 27, in init
self.decoder = swaDecoder.Decoder()
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/swaDecoder.py", line 112, in init
load = Loader(os.path.join(os.path.dirname(file), '../data'), verbose=False)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/skyfield-1.40-py3.8.egg/skyfield/iokit.py", line 110, in init
os.makedirs(self.directory)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/../data'

Setting the directory permissions on the directory 'gifts-1.0.0-py3.8.egg' to writable by other users (e.g., 777) allows the application to complete.

WMO 306 FM 201 makes no restrictions on multiple reports appearing in a MeteorologicalBulletin, this includes advisories.

Amendment 79 goes into effect in November 2020. The VAA and TCA decoders need to be updated to parse changes introduced by amendment without generating errors.

While Annex 3 currently requires a 24-hr forecast of a tropical cyclone/storm/depression's intensity and movement, the case of tropical storm/depression dissipation within 24-hrs is not handled well, either in TAC or in IWXXM 3.0 formats, reference issue #166.

tropicalCycloneAdvisory v3.1 schema permits a more 'graceful' way to depict storm/depression dissipation within the 24-hr forecast window. The TCA encoder will be updated to reflect this change.

Nonsensical date high-lighted

The eruption took place August 31st as reported in the <eruptionDetails> element

Thanks to Dr. Crawford at NOAA Air Resources Laboratory for the report!

Small code change to support Windows environments. Updates to README documentation.

Update config file to create IWXXM documents with the new URI associated with latest IWXXM release Update validation script and documentation.

• Improved error message in such case.

Missing 306/4678 +FC, -SS and -DS references in gifts/data/AerodromePresentOrForecastWeather.rdf file.

The vaaDecoder generates an incorrect error message by placing the '^' character on the wrong line.

Forecasts such as this one are not being correctly represented in XML.
FCST SWX +6 HR: 15/1000Z MNH E015 - E030

The delay of ICAO GRF implementation means that runway conditions will continue to be reported in METAR code forms. XML encoding of runway conditions needs to be restored in the codebase.

Amendment 79 to ICAO Annex 3 introduces a new mandatory parameter, intensity change, to the tropical cyclone advisory. The GIFTs TCA encoder's output will provide this new element in accordance with the v3.1 schema. The GIFTs TCA decoder already handles this new parameter and passes it to the TCA encoder.

Some states report 'CLR' as a sky condition instead of NSC or NCD. IWXXM encoding needs to make it clear when 'CLR' is reported to distinguish between it and NSC and NCD.

(METP/5 - WP/2302, Appendix C, PANS-MET, Appendix 7) Rationale for the proposed revisions to Annex 3 and PANS-MET:
a) Moved the “Intensity” from the “Effect” line to the “Observed” and “Forecast” lines, which allows for the intensity to change within an advisory as well as allows for multiple intensities, thus reduces the number of advisories in effect at one time.
b) Allow for an updated advisory to replace up to four existing advisories.
c) Added “NIGHTSIDE” and renamed “DAYLIGHT SIDE” to “DAY SIDE”
d) Updated the template and examples in the Table.
e) Removed the “minutes” from latitude and longitude since this accuracy is beyond the state of the science.
f) Editorial changes to footnote 1 for TEST and EXER.
g) Clarified that altitude information only applies to radiation events.
h) Clarified that the end point of a polygon is a repeat of the start point.
i) With the agreement that MET products in IWXXM form are not constrained to limits of coordinates, made note that advisories disseminated in abbreviated plain language should keep coordinates to a minimum of seven. This same note has been added to other similar MET products.

Washington VAAC has provided a test case:

 FVXX27 KNES 191645
 VA ADVISORY
 DTG: 20240319/1645Z
 
 VAAC: WASHINGTON
 
 VOLCANO: TEST 999999
 PSN: N3700 W07700
 
 AREA: US
 
 SUMMIT ELEV: 32805 FT (9999 M)
 
 ADVISORY NR: 2024/001
 
 INFO SOURCE: TEST
 
 ERUPTION DETAILS: TEST AT 19/1700Z
 
 OBS VA DTG: 19/1645Z
 
 OBS VA CLD: VA NOT IDENTIFIABLE FROM SATELLITE
 DATA.
 
 FCST VA CLD +6HR: 19/2300Z NO VA EXP
 
 FCST VA CLD +12HR: 20/0500Z NO VA EXP
 
 FCST VA CLD +18HR: 20/1100Z NO VA EXP
 
 RMK: THIS IS A VA ADVISORY TEST MSG. MWO TAHITI
 SHOULD NOW ISSUE A SIGMET TEST MSG FOR VA
 ADVISORY IF RECEIVED. ...KIBLER
 
 NXT ADVISORY:  NO FURTHER ADVISORIES

The VAA TAC decoder halts with a message

2024-03-20 21:52:07,819 - tac_to_xml.vaa.vaaDecoder - INFO -
 VA ADVISORY TEST MSG. MWO TAHITI
            ^
 SHOULD NOW ISSUE A SIGMET TEST MSG FOR VA
 ADVISORY IF RECEIVED. ...KIBLER
 
 NXT ADVISORY:  NO FURTHER ADVISORIES

 Expecting test or exercise or Date/Time'

The regular expression did not stop at the first occurrence of 'VA ADVISORY' but instead halted at the last one, as indicated by the caret (^) position, that occurs in the remark section of the advisory. The decoder then attempted to process the text beginning at that point.

The regular expression in question needs to be not greedy and halt at the first occurrence instead which denotes the beginning of the message.

Describe the bug
Space Weather Advisory and Tropical Cyclone Advisory encoders are using a deprecated GML type, gml:polygonPatches.

Expected behavior
Replace deprecated type with gml:patches instead.

Additional context
Thanks to François for bringing this to my attention.

Provide a little additional flexibility beyond what Annex 3 allows.

According to Section 4.1.4 of the Guidelines for the Implementation of OPMET Data Exchange using IWXXM 4th Edition, the only allowable file extension for XML products are 'xml' and 'xml.gz', as the bulletinIdentifier text.

To facilitate the ease of transmitting the XML products by US OPMET Gateway's current practice, an option was provided to the bulletin.write() method to insert the WMO AHL as the first line in the resulting output file. The bug is that the text for bulletinIdentifier is either 'txt' or 'txt.gz' as file extensions when this option is used. This is contrary to the Guidelines.

xmlUtilities computeLatLon assumes bearing (compass direction) is conformant with mathematical polar coordinates.

Forecasts such as this:

FCST VA CLD +18HR: 21/1600Z SFC/FL210 S0146
W07907 - S0158 W07822 - S0202 W07821 - S0211
W07905 - S0146 W07907 SFC/FL350 NO VA EXP

Are not compliant with Annex 3. Specifically, the
SFC/FL350 layer should be ignored. The SFC/FL210 layer
and its polygon is kept.

Although UTF-8 encoding is the default, it should be explicitly stated in the XML document.

GIFTs uses the ElementTree module to construct and serialize the XML document. Python 3 now makes the distinction between text and byte sequences. In Python 2, there was implicit conversion of byte strings to Unicode text. This is no longer done in Python 3 and the ElementTree tostring method.

Pip reports version 1.0.0 is installed when installed from the tagged version 1.3.0.

Steps to reproduce the behavior:

1. Install GIFTs from the tagged 1.3.0 version according to the README.
2. Execute 'pip list' to see the GIFTs reported version.

pip should report the GIFTs version as 1.3.0.

Below is an example reported by pip after GIFTs v1.3.0 installed.

$ pip list
Package Version

...
gifts 1.0.0
...
skyfield 1.40
...