noaa-mdl / gifts Goto Github PK
View Code? Open in Web Editor NEWGenerate IWXXM From TAC software
License: Other
Generate IWXXM From TAC software
License: Other
Tests by a Met Office revealed that the YYGGgg and yyyyMMddhhmmss portions of the <bulletinIdentifier>
element can disagree. The former is based on UTC and the latter timestamp can be affected by local time zone settings. This disagreement can be potentially confusing to systems that consume these products. Therefore, the yyyyMMddhhmmss portion of the file name shall always be based on the UTC clock.
Providing elevation always throws an IndexError
GIFTs tagged version 1.3.0 writes to directory in the module installation path which may not be writable by anyone other than the install user.
Steps to reproduce the behavior:
It is expected that the software will complete without error.
Below is example output from our test case. The script 'generate_iwxxm' is our custom command line interface to the GIFTs library.
2021-12-13 19:28:50,837 - gifts.swaDecoder - ERROR - 121 - Unable to load/initialize Skyfield Module.
Traceback (most recent call last):
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/swaDecoder.py", line 112, in init
load = Loader(os.path.join(os.path.dirname(file), '../data'), verbose=False)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/skyfield-1.40-py3.8.egg/skyfield/iokit.py", line 110, in init
os.makedirs(self.directory)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/../data'
Traceback (most recent call last):
File "generate_iwxxm.py", line 75, in
service = generate_iwxxm_service()
File "generate_iwxxm.py", line 39, in init
self.encoders.append((re.compile(r'FN\w\w\d\d\s+[A-Z]{4}\s+\d{6}', re.MULTILINE), gifts.SWA.Encoder()))
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/SWA.py", line 27, in init
self.decoder = swaDecoder.Decoder()
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/swaDecoder.py", line 112, in init
load = Loader(os.path.join(os.path.dirname(file), '../data'), verbose=False)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/skyfield-1.40-py3.8.egg/skyfield/iokit.py", line 110, in init
os.makedirs(self.directory)
File "/opt/python3/pythons/python-3.8.5/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/python3/pythons/python-3.8.5/lib/python3.8/site-packages/gifts-1.0.0-py3.8.egg/gifts/../data'
Setting the directory permissions on the directory 'gifts-1.0.0-py3.8.egg' to writable by other users (e.g., 777) allows the application to complete.
WMO 306 FM 201 makes no restrictions on multiple reports appearing in a MeteorologicalBulletin, this includes advisories.
Amendment 79 goes into effect in November 2020. The VAA and TCA decoders need to be updated to parse changes introduced by amendment without generating errors.
While Annex 3 currently requires a 24-hr forecast of a tropical cyclone/storm/depression's intensity and movement, the case of tropical storm/depression dissipation within 24-hrs is not handled well, either in TAC or in IWXXM 3.0 formats, reference issue #166.
tropicalCycloneAdvisory v3.1 schema permits a more 'graceful' way to depict storm/depression dissipation within the 24-hr forecast window. The TCA encoder will be updated to reflect this change.
Validating all XML files in C:\IWXXM_Validators\testData with IWXXM XML schemas and schematron (version: 3.0)
01:58:37.114 INFO | Validating file C:\IWXXM_Validators\testData\taf.xml against XML schema using the following catalog(s): C:\IWXXM_Validators\IWXXM-Validation\catalog-3.0.xml
01:59:12.218 ERROR | Validation FAILED on C:\IWXXM_Validators\testData\taf.xml line 2, col 313: schema_reference.4: Failed to read schema document 'file:C:/IWXXM_Validators/IWXXM-Validation/file%5C:C:%5CIWXXM_Validators%5CIWXXM-Validation/externalSchemas/schemas.wmo.int/collect/1.2/collect.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
01:59:12.219 ERROR | Validation FAILED on C:\IWXXM_Validators\testData\taf.xml line 2, col 313: cvc-elt.1.a: Cannot find the declaration of element 'MeteorologicalBulletin'.
01:59:12.220 ERROR | Validation FAILED on C:\IWXXM_Validators\testData\taf.xml line 2, col 313: schema_reference.4: Failed to read schema document '', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
01:59:12.220 ERROR | Validation FAILED on C:\IWXXM_Validators\testData\taf.xml line 2, col 340: schema_reference.4: Failed to read schema document 'file:C:/IWXXM_Validators/IWXXM-Validation/file%5C:C:%5CIWXXM_Validators%5CIWXXM-Validation/externalSchemas/schemas.wmo.int/collect/1.2/collect.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.```
Small code change to support Windows environments. Updates to README documentation.
Update config file to create IWXXM documents with the new URI associated with latest IWXXM release Update validation script and documentation.
Missing 306/4678 +FC, -SS and -DS references in gifts/data/AerodromePresentOrForecastWeather.rdf file.
The vaaDecoder generates an incorrect error message by placing the '^' character on the wrong line.
Forecasts such as this one are not being correctly represented in XML.
FCST SWX +6 HR: 15/1000Z MNH E015 - E030
The delay of ICAO GRF implementation means that runway conditions will continue to be reported in METAR code forms. XML encoding of runway conditions needs to be restored in the codebase.
Amendment 79 to ICAO Annex 3 introduces a new mandatory parameter, intensity change, to the tropical cyclone advisory. The GIFTs TCA encoder's output will provide this new element in accordance with the v3.1 schema. The GIFTs TCA decoder already handles this new parameter and passes it to the TCA encoder.
Some states report 'CLR' as a sky condition instead of NSC or NCD. IWXXM encoding needs to make it clear when 'CLR' is reported to distinguish between it and NSC and NCD.
(METP/5 - WP/2302, Appendix C, PANS-MET, Appendix 7) Rationale for the proposed revisions to Annex 3 and PANS-MET:
a) Moved the “Intensity” from the “Effect” line to the “Observed” and “Forecast” lines, which allows for the intensity to change within an advisory as well as allows for multiple intensities, thus reduces the number of advisories in effect at one time.
b) Allow for an updated advisory to replace up to four existing advisories.
c) Added “NIGHTSIDE” and renamed “DAYLIGHT SIDE” to “DAY SIDE”
d) Updated the template and examples in the Table.
e) Removed the “minutes” from latitude and longitude since this accuracy is beyond the state of the science.
f) Editorial changes to footnote 1 for TEST and EXER.
g) Clarified that altitude information only applies to radiation events.
h) Clarified that the end point of a polygon is a repeat of the start point.
i) With the agreement that MET products in IWXXM form are not constrained to limits of coordinates, made note that advisories disseminated in abbreviated plain language should keep coordinates to a minimum of seven. This same note has been added to other similar MET products.
Washington VAAC has provided a test case:
FVXX27 KNES 191645
VA ADVISORY
DTG: 20240319/1645Z
VAAC: WASHINGTON
VOLCANO: TEST 999999
PSN: N3700 W07700
AREA: US
SUMMIT ELEV: 32805 FT (9999 M)
ADVISORY NR: 2024/001
INFO SOURCE: TEST
ERUPTION DETAILS: TEST AT 19/1700Z
OBS VA DTG: 19/1645Z
OBS VA CLD: VA NOT IDENTIFIABLE FROM SATELLITE
DATA.
FCST VA CLD +6HR: 19/2300Z NO VA EXP
FCST VA CLD +12HR: 20/0500Z NO VA EXP
FCST VA CLD +18HR: 20/1100Z NO VA EXP
RMK: THIS IS A VA ADVISORY TEST MSG. MWO TAHITI
SHOULD NOW ISSUE A SIGMET TEST MSG FOR VA
ADVISORY IF RECEIVED. ...KIBLER
NXT ADVISORY: NO FURTHER ADVISORIES
The VAA TAC decoder halts with a message
2024-03-20 21:52:07,819 - tac_to_xml.vaa.vaaDecoder - INFO -
VA ADVISORY TEST MSG. MWO TAHITI
^
SHOULD NOW ISSUE A SIGMET TEST MSG FOR VA
ADVISORY IF RECEIVED. ...KIBLER
NXT ADVISORY: NO FURTHER ADVISORIES
Expecting test or exercise or Date/Time'
The regular expression did not stop at the first occurrence of 'VA ADVISORY' but instead halted at the last one, as indicated by the caret (^) position, that occurs in the remark section of the advisory. The decoder then attempted to process the text beginning at that point.
The regular expression in question needs to be not greedy and halt at the first occurrence instead which denotes the beginning of the message.
Describe the bug
Space Weather Advisory and Tropical Cyclone Advisory encoders are using a deprecated GML type, gml:polygonPatches.
Expected behavior
Replace deprecated type with gml:patches instead.
Additional context
Thanks to François for bringing this to my attention.
Provide a little additional flexibility beyond what Annex 3 allows.
According to Section 4.1.4 of the Guidelines for the Implementation of OPMET Data Exchange using IWXXM 4th Edition, the only allowable file extension for XML products are 'xml' and 'xml.gz', as the bulletinIdentifier text.
To facilitate the ease of transmitting the XML products by US OPMET Gateway's current practice, an option was provided to the bulletin.write() method to insert the WMO AHL as the first line in the resulting output file. The bug is that the text for bulletinIdentifier is either 'txt' or 'txt.gz' as file extensions when this option is used. This is contrary to the Guidelines.
xmlUtilities computeLatLon assumes bearing (compass direction) is conformant with mathematical polar coordinates.
Forecasts such as this:
FCST VA CLD +18HR: 21/1600Z SFC/FL210 S0146
W07907 - S0158 W07822 - S0202 W07821 - S0211
W07905 - S0146 W07907 SFC/FL350 NO VA EXP
Are not compliant with Annex 3. Specifically, the
SFC/FL350 layer should be ignored. The SFC/FL210 layer
and its polygon is kept.
Although UTF-8 encoding is the default, it should be explicitly stated in the XML document.
GIFTs uses the ElementTree module to construct and serialize the XML document. Python 3 now makes the distinction between text and byte sequences. In Python 2, there was implicit conversion of byte strings to Unicode text. This is no longer done in Python 3 and the ElementTree tostring method.
Pip reports version 1.0.0 is installed when installed from the tagged version 1.3.0.
Steps to reproduce the behavior:
pip should report the GIFTs version as 1.3.0.
Below is an example reported by pip after GIFTs v1.3.0 installed.
$ pip list
Package Version
...
gifts 1.0.0
...
skyfield 1.40
...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.