networktocode / circuit-maintenance-parser Goto Github PK
View Code? Open in Web Editor NEWPython library to parse circuit maintenances from network service providers.
License: Apache License 2.0
Python library to parse circuit maintenances from network service providers.
License: Apache License 2.0
Use new Pydantic 2.0
Check CI execution from auto PRs
Error
RecursionError: maximum recursion depth exceeded in comparison
Now that we can combine multiple DataParts
we could take Stamp
from the Email Date, and make it mandatory.
This will add extra info to the Maintenance
, about when it was created
Automated Parser/End-to-end test discoverability, so the test cases could be automatically discovered following some pattern
Simplify development time, by dynamically fetching test cases, so no need to manually explicit them,
Only unique Circuit IDs should be in one notification
Some Parsers return repeated Circuit IDs in the same notification
Remove support for Python 3.6 as it is EOL.
Allow updates to various dependencies that have already dropped support for 3.6.
I expected that the invoke tasks would allow for the testing
invoke build
We expect for the Zayo parser to pull the following fields from maintenance emails:
Circuit Id
Expected Impact
A Location Address
Z Location Address
Legacy Circuit Id
The Zayo parser is looking for data from the following fields:
Circuit Id
Expected Impact
A Location CLLI
Z Location CLLI
Legacy Circuit Id
Proposed Fix
diff --git a/circuit_maintenance_parser/parsers/zayo.py b/circuit_maintenance_parser/parsers/zayo.py
index 49e9b68..bfc78b7 100644
--- a/circuit_maintenance_parser/parsers/zayo.py
+++ b/circuit_maintenance_parser/parsers/zayo.py
@@ -64,8 +64,8 @@ class HtmlParserZayo1(Html):
if len(head_row) < 5 or [self.clean_line(line) for line in head_row[:5]] != [
"Circuit Id",
"Expected Impact",
- "A Location CLLI",
- "Z Location CLLI",
+ "A Location Address",
+ "Z Location Address",
"Legacy Circuit Id",
]:
logger.warning("Table headers are not as expected: %s", head_row)
no errors while installing.
when installing napalm-panos we see the lxml version error.
All test data should be tested
Properly parse the email attached.
$ circuit-maintenance-parser --data-file a.eml --data-type email --provider-type zayo
Traceback (most recent call last):
File "/home/jmaslak/.pyenv/versions/3.9.6/bin/circuit-maintenance-parser", line 8, in <module>
sys.exit(main())
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/circuit_maintenance_parser/cli.py", line 47, in main
parsed_notifications = provider.get_maintenances(data)
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/circuit_maintenance_parser/provider.py", line 118, in get_maintenances
if self.exclude_filter_check(data) or not self.include_filter_check(data):
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/circuit_maintenance_parser/provider.py", line 79, in include_filter_check
return self.filter_check(self._include_filter, data, "include")
File "/home/jmaslak/.pyenv/versions/3.9.6/lib/python3.9/site-packages/circuit_maintenance_parser/provider.py", line 97, in filter_check
data_part_content = data_part.content.decode().replace("\r", "").replace("\n", "")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc9 in position 2254: invalid continuation byte
See above. The problem in the email appears to be the =C9's in the raw .EML file, that is part of the email signature. Note I annonymized key aspects of the EML file but that does not restrict reproduction of the bug. I have gzipped and attached the email file (since Github doesn't allow an EML that isn't zipped).
As part of the initial setup, I ran invoke tests --local
I expect all tests to pass
One of the unit test failed
tests/unit/test_e2e.py:516:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Seaborn()
data = NotificationData(data_parts=[DataPart(type='email-header-date', content=b'Mon, 16 Aug 2021 17:29:56 +0100'), DataPart(...px"> <br></div></blockquote></div> <div><br></div></div><div id="m_-8814736200534887510ZDeskInteg"></div><br></div>')])
def get_maintenances(self, data: NotificationData) -> Iterable[Maintenance]:
"""Main entry method that will use the defined `_processors` in order to extract the `Maintenances` from data."""
provider_name = self.__class__.__name__
error_message = ""
related_exceptions = []
if self.exclude_filter_check(data) or not self.include_filter_check(data):
logger.debug("Skipping notification %s due filtering policy for %s.", data, self.__class__.__name__)
return []
for processor in self._processors:
try:
return processor.process(data, self.get_extended_data())
except ProcessorError as exc:
process_error_message = (
f"- Processor {processor.__class__.__name__} from {provider_name} failed due to: %s\n"
)
logger.debug(process_error_message, traceback.format_exc())
related_exc = rgetattr(exc, "__cause__")
error_message += process_error_message % related_exc
related_exceptions.append(exc)
continue
> raise ProviderError(
(f"Failed creating Maintenance notification for {provider_name}.\nDetails:\n{error_message}"),
related_exceptions=related_exceptions,
)
E circuit_maintenance_parser.errors.ProviderError: Failed creating Maintenance notification for Seaborn.
E Details:
E - Processor CombinedProcessor from Seaborn failed due to: SubjectParserSeaborn1 parser was not able to extract the expected data for each maintenance.
E - Raw content: b'Fwd: [rd-notices] Re:[## 99999 ##] Emergency Maintenance Notification\r\n CID: AAA-AAAAA-AAAAA-AAA1-0000-00 TT#11111'
E - Result: [{}]
E - Processor CombinedProcessor from Seaborn failed due to: 5 validation errors for Maintenance
E maintenance_id
E field required (type=value_error.missing)
E circuits
E At least one circuit has to be included in the maintenance (type=value_error)
E start
E field required (type=value_error.missing)
E end
E field required (type=value_error.missing)
E status
E field required (type=value_error.missing)
circuit_maintenance_parser/provider.py:136: ProviderError
invoke tests --local
Equinix maintenance notification is parsed correctly
Email parsing fails with error
Provider processing failed: Failed creating Maintenance notification for Equinix.
Details:
- Processor CombinedProcessor from Equinix failed due to: too many values to unpack (expected 3)
Besides the data representing the Maintenance, the response could incorporate metadata to understand things like:
No significant increase in memory consumption when loading the circuit_maintenance_parser
library after upgrading from 2.0.3 to 2.0.4
Memory usage jumps by over 400 MB after import circuit_maintenance_parser
with 2.0.4.
This appears to be due to #104, specifically due to a Geolocator
object being instantiated at library load time, which in turn calls tzwhere.tzwhere()
, which appears to allocate a large amount of memory.
python
import circuit_maintenance_parser
Verizon maintenance notification is parsed correctly
Email parsing fails with error
Provider processing failed: Failed creating Maintenance notification for Verizon.
Details:
- Processor CombinedProcessor from Verizon failed due to: 'NoneType' object has no attribute 'find_all'
Add CLI command to parse the data from a file
Interact with other tooling than Python code
Return multiple maintenance windows in the event where a vendor sends us multiple activity dates. This feature should allow multiple maintenance windows to be returned as separate fields rather than one big window. Currently, multiple windows from vendor are being clumped into one large window (earliest start time + latest end time found).
The proposed functionality of this would be allowing maintenance windows to only be active for the window that the vendor specifies, not all the time in between the earliest start time and latest end time. This increases availability of network resources in the event where operators move traffic away from network links during maintenance windows.
An example email (sanitized) that we would receive:
# Some Zayo notifications may include multiple activity dates.
# For example, given:
#
# 1st Activity Date
# 01-Nov-2021 00:01 to 01-Nov-2021 05:00 ( Mountain )
# 01-Nov-2021 06:01 to 01-Nov-2021 11:00 ( GMT )
#
# 2nd Activity Date
# 02-Nov-2021 00:01 to 02-Nov-2021 05:00 ( Mountain )
# 02-Nov-2021 06:01 to 02-Nov-2021 11:00 ( GMT )
#
# 3rd Activity Date
# 03-Nov-2021 00:01 to 03-Nov-2021 05:00 ( Mountain )
# 03-Nov-2021 06:01 to 03-Nov-2021 11:00 ( GMT )
By testing the email file against circuit-maintenance-parser, we should be getting multiple maintenance objects rather than just one with the earliest start time and latest end time:
#circuit-maintenance-parser --data-file ~/zayo_test_email.eml --data-type html --provider-type zayo
Circuit Maintenance Notification #0
{
"account": "Test",
"circuits": [
{
"circuit_id": "ZAYO/TEST/CIRCUIT",
"impact": "OUTAGE"
}
],
"end": 1635764400,
"maintenance_id": "TTN-0001234567",
"organizer": "[email protected]",
"provider": "zayo",
"sequence": 1,
"stamp": 1635573660,
"start": 1635746460,
"status": "CONFIRMED",
"summary": "Zayo Third-Party Provider will implement maintenance to perform temporary fiber relocation in order to proactively avoid outages.",
"uid": "0"
}
Circuit Maintenance Notification #1
{
"account": "Test",
"circuits": [
{
"circuit_id": "ZAYO/TEST/CIRCUIT",
"impact": "OUTAGE"
}
],
"end": 1635850800,
"maintenance_id": "TTN-0001234567",
"organizer": "[email protected]",
"provider": "zayo",
"sequence": 1,
"stamp": 1635573660,
"start": 1635832860,
"status": "CONFIRMED",
"summary": "Zayo Third-Party Provider will implement maintenance to perform temporary fiber relocation in order to proactively avoid outages.",
"uid": "1"
}
Circuit Maintenance Notification #2
{
"account": "Test",
"circuits": [
{
"circuit_id": "ZAYO/TEST/CIRCUIT",
"impact": "OUTAGE"
}
],
"end": 1635937200,
"maintenance_id": "TTN-0001234567",
"organizer": "[email protected]",
"provider": "zayo",
"sequence": 1,
"stamp": 1635573660,
"start": 1635919260,
"status": "CONFIRMED",
"summary": "Zayo Third-Party Provider will implement maintenance to perform temporary fiber relocation in order to proactively avoid outages.",
"uid": "2"
}
Add parsing for AWS Direct Connect notifications
EXAMPLE 1
Subject
Reminder: AWS Direct Connect Planned Maintenance Notification [AWS Account: 11111111]
Body
Hello,
Planned maintenance has been scheduled on an AWS Direct Connect endpoint in Equinix SG2, Singapore, SGP from Mon, 13 Sep 2021 19:02:00 GMT to Tue, 14 Sep 2021 02:02:00 GMT for 7 hours. During this maintenance window, your AWS Direct Connect services listed below may become unavailable.
dxvif-XXXXXXXX
This maintenance is scheduled to avoid disrupting redundant connections at the same time.
If you encounter any problems with your connection after the end of this maintenance window, please contact AWS Support[1].
[1] https://aws.amazon.com/support
Sincerely,
Amazon Web Services
Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210.
EXAMPLE 2
Subject
AWS Direct Connect Maintenance Complete [AWS Account: 111111]
Body
Hello,
We would like to inform you that the maintenance that was scheduled for AWS Direct Connect endpoint in Equinix LD5, Slough, GBR from Thu, 9 Sep 2021 03:30:00 GMT to Thu, 9 Sep 2021 04:30:00 GMT has been completed. Please find below your AWS Direct Connect services that would have been affected by this maintenance.
dxvif-xxxx
dxcon-ccccc
If you have any questions regarding the completed work, or if you would like to report a fault or adjust your contact information, please contact AWS support at https://aws.amazon.com/support .
Sincerely,
Amazon Web Services
Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210.
Decouple core and parser functionalities to allow custom import of parser outside of the upstream library.
A test parser for Cogent notification emails with Content-Type: text/plain
A text parser for cogent would provide additional coverage over incoming notifications by including emails of type Content-Type: text/plain
Not sure if this is the right way to bring this issue up. Some of my cogent notifications are not able to be parsed on their own and I believe it is due to cogent's email formatting.
Cogent email of Content-Type: text/plain
are not able to be parsed by the current Cogent provider parsers.
Example provided below:
─➤ circuit-maintenance-parser --data-file "cogent_eml.eml" --data-type email --provider-type cogent 2 ↵
Provider processing failed: Failed creating Maintenance notification for Cogent.
Details:
- Processor CombinedProcessor from Cogent failed due to: 6 validation errors for Maintenance
account
field required (type=value_error.missing)
maintenance_id
field required (type=value_error.missing)
circuits
field required (type=value_error.missing)
start
field required (type=value_error.missing)
end
field required (type=value_error.missing)
status
field required (type=value_error.missing)
Example email:
cogent_eml.eml.txt
Allow user to define a local folder/repository containing additional parser classes that can be used to parse email notifications.
Users can quickly implement parsing extensions to support providers in their environment.
These parsing updates can still be submitted back upstream to the parent project when the user can assign time to work on the upstream PR.
All tests to pass
Equinix Parser test fails. The parsed time stamp appears to be different than the expected timestamp.
Per the BCOP, X-MAINTNOTE-STATUS
can be omitted from an iCal notification (the only example I see of this in the BCOP is for a rescheduled maintenance window), but our Maintenance
class defines status
as a mandatory field. In general how should we handle iCal notifications that omit this field?
I have a real-world example received from a provider for a "completed" notification that omits the X-MAINTNOTE-STATUS, so simply assuming that "no X-MAINTNOTE-STATUS implies "status=re-scheduled"" would not be correct in this case. We could possibly parse the DESCRIPTION
or SUMMARY
fields -- in this particular case the word "completed" is present in both -- but that's probably not desirable as a general solution? Looking for suggestions here.
Colt email subject Colt Service Affecting Maintenance Notification - CRQ1-xxxxx [20/8/2022 20:00:00 GMT - 21/8/2022 01:00:00 GMT] for YYYYY, XYZ
is not properly handle by the parser.
Colt email doesn't start with [ EXTERNAL ]
, this field could have been add by the email platform used during the first tests.
The regex in SubjectParserColt1 and SubjectParserColt2 should be modify to not enforce the presence of brackets at the beginning of the email.
Colt Parser is expecting COLT email to start with [ .* ]
, the 2 brackets should be see as facultative as well.
Add PCCW maintenance parser
Useful for clients PCCW.
Migrate to GitHub Actions
Migrate off of Travis to GitHub Actions.
Test data should not contain any public IP addresses for anonymity purposes.
Some files contains public IPv4 and IPv6 addresses.
Backwards compatibility in minor-version-incrementing releases.
The MaintenanceNotification
model was renamed to Parser
by #26, meaning that any code (such as version 0.1.3 of https://github.com/nautobot/nautobot-plugin-circuit-maintenance) that makes use of this model will fail when attempting to import it:
Traceback (most recent call last):
File "/usr/local/bin/nautobot-server", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/nautobot/core/cli.py", line 62, in main
initializer=_configure_settings, # Called after defaults
File "/usr/local/lib/python3.7/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
management.execute_from_command_line([runner_name, command] + command_args)
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/usr/local/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
app_config.ready()
File "/usr/local/lib/python3.7/site-packages/nautobot_circuit_maintenance/__init__.py", line 68, in ready
super().ready()
File "/usr/local/lib/python3.7/site-packages/nautobot/extras/plugins/__init__.py", line 92, in ready
jobs = import_object(f"{self.__module__}.{self.jobs}")
File "/usr/local/lib/python3.7/site-packages/nautobot/extras/plugins/utils.py", line 44, in import_object
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.7/site-packages/nautobot_circuit_maintenance/jobs.py", line 2, in <module>
from .handle_notifications.handler import HandleCircuitMaintenanceNotifications
File "/usr/local/lib/python3.7/site-packages/nautobot_circuit_maintenance/handle_notifications/handler.py", line 6, in <module>
from circuit_maintenance_parser import ParsingError, init_parser, MaintenanceNotification
ImportError: cannot import name 'MaintenanceNotification' from 'circuit_maintenance_parser' (/usr/local/lib/python3.7/site-packages/circuit_maintenance_parser/__init__.py)
New LLM (Large Language Model) allow a sophisticated language processing that could get data from structures/unstructured notifications and retrieve part or total of the necessary information.
This could avoid having to write custom parser for all Providers
I'm starting to run into dependency hell with some plugins. There are some dependencies in this project (mainly click) which are outdated. Also development dependencies should probably be set to * as to not cause issues down the line but they are development dependencies
Plays nice in the sandbox with plugins dependent on newer versions of dependencies
this plugin is dependent on click 7
Parsing failure should result in a clearly formatted exception and/or logging message to make it easy to troubleshoot and fix.
Due to the logic in https://github.com/networktocode/circuit-maintenance-parser/blob/develop/circuit_maintenance_parser/providers.py#L60 that catches all ParsingError
exceptions raised by individual parser classes and discards them, the only log messages seen are generic, i.e.:
Parser HtmlParserCogent1 for provider Cogent was not successful
and the ultimately raised ParsingError
is similarly generic:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/nautobot_circuit_maintenance/handle_notifications/handler.py", line 193, in process_raw_notification
parsed_notifications = parser.process()
File "/usr/local/lib/python3.7/site-packages/circuit_maintenance_parser/providers.py", line 81, in process
raise ParsingError("None of the parsers was able to parse the notification")
circuit_maintenance_parser.errors.ParsingError: None of the parsers was able to parse the notification
This makes it very difficult to diagnose why the parsing failed.
A Provider that uses a CombinedProcessor should be able to cleanly process multiple NotificationData in series without data bleeding over from one processing to the next
After successfully processing one NotificationData, then processing a second, incomplete NotificationData, the second processing returns a Maintenance that retains fields from the first processed NotificationData instead of raising a ProviderError due to the incomplete data.
Add Crown Castle maintenance parser
Useful for clients of Crown Castle.
Parsers could support different parsing implementations for vendors.
When a provider offers different templates or it is transitioning, the library should provide the best parsed output.
For instance, Telstra has started using iCalendar format: https://twitter.com/NetworkAndrew/status/1402964730285789190
So, a provider parser should be able to parse multiple formats, and for each format, multiple implementations.
When a problem is detected initializing NotificationData
, the user of the init_data
methods should not get the exception
To create a valid Maintenance
object we enforce a minimum set of attributes that it's validated via Pydantic
.
The parser
that is parsing the raw
data from the notification has to get all the necessary information before creating the Maintenance
and there are some identified cases where not all the necessary information is available in the raw
payload that a notification is providing (more in Use Case).
Providing an optional extra_data
attribute(s) together with the raw
data could complement it with data that the provider is not embedding in the raw notification data, or even allow customization of the objects by the library consumer.
This feature is based in, at least, 2 use cases:
extra_data
that would extend the data processed from the main information payload.Add Hurricane Electric maintenance parser
Useful for clients Hurricane Electric.
Remove the Cogent parser dependency on the tzwhere library that is unmaintained (last release 7 years ago).
Allow for adopting newer Python versions.
Cogent parser works well and tests passes
The Cogent parser doesn't provide the Timezone, so we use the Location to deduce the Timezone using an external API that often hits rate limit and it fails
Notification parsed successfully
Notification parsing fails. No maintenance_id
.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/nautobot_circuit_maintenance/handle_notifications/handler.py", line 234, in get_maintenances_from_notification
result = parser_provider.get_maintenances(data_to_process)
File "/usr/local/lib/python3.8/site-packages/circuit_maintenance_parser/provider.py", line 136, in get_maintenances
raise ProviderError(
circuit_maintenance_parser.errors.ProviderError: Failed creating Maintenance notification for Verizon.
Details:
- Processor CombinedProcessor from Verizon failed due to: 1 validation error for Maintenance
maintenance_id
field required (type=value_error.missing)
All parsers should err on the side of including more information in the summary
field versus trying to narrowly extract a single sentence or paragraph from the overall notification.
Notifications often include many additional details beyond what we're currently capturing in most summary
fields. In integrating the output of this parser library with other systems, it's better to preserve information rather than discarding it.
Frankly, I'm not entirely sure if this classifies as bug or feature request.
Sparkle started sending us maintenance notifications with this datetime format: MMM/DD/YYYY HH:mm UTC. For example: Jan/27/2023 03:00 UTC.
Currently circuit-maintenance-parser doesn't support this format and it's not a common/standard format. However, it can very much be parsed:
Python 3.10.9 (main, Dec 15 2022, 17:11:09) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import arrow
>>> a = "Jan/27/2023 03:00 UTC"
>>> arrow.get(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/tdo/Documents/dev/MNA/lib/python3.10/site-packages/arrow/api.py", line 91, in get
return _factory.get(*args, **kwargs)
File "/Users/tdo/Documents/dev/MNA/lib/python3.10/site-packages/arrow/factory.py", line 254, in get
dt = parser.DateTimeParser(locale).parse_iso(arg, normalize_whitespace)
File "/Users/tdo/Documents/dev/MNA/lib/python3.10/site-packages/arrow/parser.py", line 208, in parse_iso
raise ParserError(
arrow.parser.ParserError: Expected an ISO 8601-like string, but was given 'Jan/27/2023 03:00 UTC'. Try passing in a format string to resolve this.
>>> arrow.get(a, 'MMM/DD/YYYY HH:mm UTC')
<Arrow [2023-01-27T03:00:00+00:00]>
We expected a successfully parsed Maintenance object.
We get an error as shown here:
(CMP) tdo@PlayStation [16:02:13] [~]
-> % circuit-maintenance-parser --data-file ~/Downloads/NETENG-22869.eml --data-type email --provider-type sparkle
Provider processing failed: Failed creating Maintenance notification for Sparkle.
Details:
- Processor CombinedProcessor from Sparkle failed due to: Unknown string format: Jan/27/202303:00 UTC
circuit-maintenance-parser --data-file /path/to/sparkle.eml --data-type email --provider-type sparkle
circuit_maintenance_parser version: v2.2.2
Similar to #210, Lumen notifications are not accurately pulling all maintenance windows. In Lumens maintenances, Circuit Maintenance Parser (CMP) is actually only currently able to export one start/end time and in all cases, this is the last table that CMP scans. See my example in the Use Case.
Allows operators to schedule in accurate maintenance windows for Lumen Maintenances.
See the attached Lumen email in the comments of this issue.
Currently with this example, we extract the following start and end time:
Start: 2023/03/23-04:00 UTC
End: 2023/03/23-12:00 UTC
As you can see in the email, this is only the second window for the "Alternate Night".
We need to be able to capture both windows and have them scheduled as two separate events.
When running the circuit-maintenance-parser test on it, this should return something similar to the below:
✗ circuit-maintenance-parser --data-file ~/Downloads/Lumen_redacted.eml --data-type email --provider-type lumen
Circuit Maintenance Notification #0
{
"account": "Company",
"circuits": [
{
"circuit_id": "111111111",
"impact": "OUTAGE"
},
{
"circuit_id": "222222222",
"impact": "OUTAGE"
}
],
"end": 1679479200,
"maintenance_id": "123456789",
"organizer": "[email protected]",
"provider": "lumen",
"sequence": 1,
"stamp": 1678379003,
"start": 1679457600,
"status": "IN-PROCESS",
"summary": "Lumen intends to carry out internal maintenance within its network. This has been designated as ESSENTIAL. The nature of this work is to repair fiber and is required in order to avoid unplanned outages from damages related to natural causes.",
"uid": "0"
}
Circuit Maintenance Notification #1
{
"account": "Company",
"circuits": [
{
"circuit_id": "111111111",
"impact": "OUTAGE"
},
{
"circuit_id": "222222222",
"impact": "OUTAGE"
}
],
"end": 1679565600,
"maintenance_id": "123456789",
"organizer": "[email protected]",
"provider": "lumen",
"sequence": 1,
"stamp": 1678379003,
"start": 1679544000,
"status": "IN-PROCESS",
"summary": "Lumen intends to carry out internal maintenance within its network. This has been designated as ESSENTIAL. The nature of this work is to repair fiber and is required in order to avoid unplanned outages from damages related to natural causes.",
"uid": "0"
}
Today, a Provider
processing a notification can only return either the maintenance objects or raise a ProviderError
when it was not possible to get, at least, a maintenance object from the notification.
Some times the notification parsed is not actually containing any relevant information and it's not actually needed to be parsed and raise an error, just signal that it was not containing any relevant info.
Being able to differentiate between parsing errors and non relevant notification would make the library clients able to handle each one in a different way.
E2E testing for multiple maintenances should pass
Timestamps for Lumen PR #216 are not enable for e2e testing.
Do not depend fail hard on external dependencies. Maybe catch results or maybe just provide some default info.
Flaky Cogent parser depending on external geolocation API:
FAILED tests/unit/test_utils.py::test_city_timezones[North Bergen, NJ-America/New_York] - geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=North+Bergen%...
Enable the client of the library to initalize the Provider
with a default provider
type and organizer
to overwrite the default one that every Provider
has.
This enable better customization, for instance, if for a specific client a network service provider uses a custom email or the client wants to use a custom type name.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.