Builds can pass the YAML format check but still fail if the file isn't in the format expected by serde in src/stricts.rs and src/main.rs

For example: https://github.com/beyarkay/eskom-calendar/runs/7703427563?check_suite_focus=true

To complete this issue, create another workflow that will only pass it manually specified is in a format that will be accepted by serde.

City Power *.ics files were part of a previous release, but I see that they are now gone…

Originally posted by @zeorin in #9 (comment)

I think these issues would have been caused by this build on commit a0d6128.

Comment from @beyarkay:

Not entirely sure what happened there though, they shouldn't have been gone in the first place. I just had to recompile the calendars in order to get them to appear again.

What's probably happened is that the first build failed silently for only one or two areas, so the fox will likely include some more workflows to ensure everything got converted to an ICS file properly.

Great tool, would be awesome if it takes into account CPT special levels

I think adding an FAQ would be helpful, and a nice place to send people for common questions. Some FAQs

Can you add [feature]?

How do I add this to my calendar?

How is this different to EskomSePush, Loadshedding Notifier, etc?

Help! I can't find my area

How often does this update?

Currently there isn't much formal documentation, and none of it is HTML so it's tricky to see the project at a glance. This issue would be DONE when there is documentation for every member and struct. Also look into hosting the documentation on docs.rs

Hi there,

I hope you are well. First of all - I absolutely love the calendar. It is such a brilliant idea and it is already working so well for my personal needs.

I have a team of 10 people and would love for them to use it too - however, we are struggling to locate all of the areas that we need for the team to be fully using this calendar. Are areas still being added, or is this something that you would like me to raise with you?

Please let me know - and thank you again for the great work.

Matthew

Check the python script is properly formatted with black and deny a push if it is not properly formatted.

Github releases are great, but they're a bit intimidating and unintuitive for those who don't know code. Make a frontend (probably via github pages) on which to host the links. This should include:

• some method of searching or filtering the calendars so that the user doesn't have to scroll through 300+ calendars
• Preferably the links should be clickable, and result in the user being subscribed to the correct calendar
• A more user-friendly description of the different calendars, explaining which human-readable suburbs are included in each area

First, thanks for this project, it's a great initiative.

I think it'd be great if the latest ICS files could be published online somewhere (maybe GitHub pages?), deployed automatically with CI when there's a new release, so that they'd always be up to date. Then users could subscribe to them with their calendar apps (most support subscribing to an ICS file ("feed") online), or even with e.g. Google Calendar

Area name: Tshwane

Link to loadshedding schedule: https://www.tshwane.gov.za/sites/Departments/Public-works-and-infrastructure/Pages/Load-Shedding.aspx

Possible technical issues:

None anticipated

Many areas are not shed by eskom and so don't have areas specified by eskom. Currently these areas just result in a log line like Cannot parse: Some("https://www.capetown.gov.za/Family%20and%20home/Residential-utility-services/Residential-electricity-services/Load-shedding-and-outages") "City of Cape Town" and there are no CSV files defining the national LS schedule for these areas.

A scraper should be created to download data and format them as CSVs in the same format as the existing CSVs

Additionally, these areas often have a different schedule than the rest of the country. Changes should be made to src/main.rs to enable manually_specified.yaml to accept a parameter like "shedder" or similar, so the schedules for the country and for the regionally-shed areas can be kept separate

Information sources for non-eskom shed areas:

• Lexity (for 2nd hand information in HTML tables) https://www.lexity.co.za/
• Emfuleni: https://www.eskom.co.za/distribution/wp-content/uploads/2021/06/emfuleni-load-shedding-schedule.pdf
• Tshwane: https://www.tshwane.gov.za/sites/Departments/Public-works-and-infrastructure/Pages/Load-Shedding.aspx
• citypower: https://www.citypower.co.za/customers/Pages/Load_Shedding_Downloads.aspx added in d766ab4
• Ekurhuleni: https://www.ekurhuleni.gov.za/for-me/load-shedding-schedules/ Added in 9aafeef

Possible complications:

• Stellenbosch shedding schedule might be more complicated than originally thought: https://stellenbosch.gov.za/eskom-schedule/

Interested parties:

• Tswane:
  • https://www.reddit.com/r/southafrica/comments/w3hq7v/get_loadshedding_schedules_in_your_calendar/igw933g/

Currently building the rust dependencies is about 3 minutes of the 9 minute total build time (the other 6 minutes are mostly replacing the release assets)

We can use https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows to cache the build assets and significantly improve time to release.

This will also help #23

In general it is impossible to automate loadshedding announcements since eskom has no rhyme nor reason to how they are structured. However it might be able to automate ~50% of them.

Examples of announcements:

• #POWERALERT1 Breakdowns and persistent shortages of generation capacity necessitate the continuous
  implementation of Stage 4 loadshedding until 05:00 on Sunday, thereafter Stage 3 until 05:00 on Monday
• #POWERALERT1 Stage 4 loadshedding will continue to be implemented throughout Thursday and Friday, with a
  possibility of lower stages from Saturday morning
• #POWERALERT1 Stage 4 loadshedding will regretfully be implemented from 10:00 this morning until 05:00 on Monday morning
• #POWERALERT1 Loadshedding has been increased to Stage 4 until 05:00 on Thursday as three Kendal generation
  units tripped. Thereafter it will revert to Stage 2 until midnight Friday.
• #Loadshedding #Stage4 A sudden tripping of three Kendal Power Station units (1 920MW maximum generating capacity) has forced the implementation of Stage 4 loadshedding starting at 10:00 THIS morning until 05:00 on Thursday morning. A full statement will be published shortly
• #Stage2 #loadshedding will be implemented from 16:00 until 22:00 this evening Schedules can be downloaded from https://loadshedding.eskom.co.za Municipal customers can obtain schedules from their local municipalities
• Load Reduction Notice Date: 05 September 2022 #EskomMpumalanga will implement load reduction tomorrow morning, from 05:00 to 07:00, and in the evening from 17:00 to 19:00. Please refer to the attached statement for areas that will be affected.
• #POWERALERT1 Eskom cautions the public that the generation system remains constrained and Stage 2 loadshedding might be implemented at short notice during the evening peaks this week
• #POWERALERT1 As previously communicated, Stage 2 loadshedding will be implemented at 16:00 - 24:00 this afternoon and at the same time on Thursday afternoon

So it might be possible to do something quite simple that would watch for tweets matching a pattern, and then automatically suggest the PR that corresponds to these changes. So a tweet announcing load shedding stage 4 from Aug-18 to Aug-19 would automatically adjust manually_specified.yaml to represent those changes.

Note that not all tweets will be automatable, as sometimes the formatting goes out the window.

Idea inspired by conversation in #56

A really powerful option would be to have bots that replied to messages or sent messages automatically on various platforms. This would provide the user with more customisation, and also allow business teams to be more tightly integrated with the current loadshedding schedule.

The platforms which allow bot accounts are (to the best of my knowledge):

• Slack
• MS Teams
• Telegram

These should be developed in separate repositories, to keep a separation of concerns.

Nice features for the bots to have

• Message in a certain channel/group/team when loadshedding in certain areas is starting (or ending)
• Some level of customisation in the above message, so team members know who's in which area (ie "CityPower2 loadshedding starting in 30 minutes, power off for Jess and Julia"
• Be able to toggle the status of a user (after being granted permission) so the bot could make it obvious to your team members if you're without power (and/or without internet). Something like
  • "Boyd (status: 🕯)" - No power, no internet
  • "Boyd (status: 🔦)" - Loadshedding, but still some power (battery packs/inverter/laptop battery/UPS/etc)
  • "Boyd (status: ⚡️)" - No loadshedding

If there's a high enough level of loadshedding, then it's possible that two loadshedding slots could overlap, like:

• Loadshedding stage 5: 02:00 -> 04:30
• Loadshedding stage 5: 04:00 -> 06:30

And currently these will be displayed as two events, when they really should be one event, like:

• Loadshedding stage 5: 02:00 -> 06:30

This shouldn't be too tricky to fix, although some thought should be given to how this will impact the description of the event.

This should be implemented on the data-input level, so that the CSV files will have the unduplicated information (making it easier for end-users to use the CSVs)

Note: the release links redirect to amazon S3 links with expiring credentials in them.

When I added the calendar to Thunderbird it automatically replaced the calendar's URL with the S3 link, which upon expiry of the credentials would cause an error. In the Thunderbird UI it is not possible to change this URL back to the original, neither when adding the calendar nor by editing it once it has been added.

I am trying a workaround where I've edited the URL using the about:config settings editor in Thunderbird that gives one direct access to the settings values. Search for calendar.registry.*.uri in the search bar of the config editor and replace the S3 uri with the original.

Originally posted by @zeorin in #14 (comment)

Currently the eskom-shed data is collected from the PDFs available here, but there's also an API (see forum description here) and a file-download facility available by pinging URLs like : https://loadshedding.eskom.co.za/LoadShedding/DownloadFile?ID=1058819&Name=Macassar.

The API wasn't found to be accurate, but it's possible that the file-download facility might be more accurate than the PDFs (although so far the PDFs haven't been inaccurate)

Create a git-hook or github action that checks the formatting of all yaml and csv files in the repository, and ensures they are correctly formatted. If the formatting is incorrect, the commit should be denied and an error message displayed to the client.

The aim is to reduce the chance of any failed builds such as this one, introduced by 72b525b and fixed by a0d6128.

The calendar events currently does not have configured alerts. Alerts will make this solution a lot more useful

Are stages updated manually?

How about:
Scrape the info (shortly after) each new tweet from eskom, from pages like this.

I would be more than happy to help with this (email me).

There's currently a python script that downloads and parses the Eskom PDFs. This isn't great, but I couldn't find anything in Rust to do the same. This script is entirely untested. This issue would be considered DONE when the python script has a bunch of unit tests against various PDFs and various URLs to ensure it's hardy against the strange formatting eskom can pull sometimes.

Looks like yours is the first open source project to implement Eskom's national stage schedule into the results, Congrats & really nice work. I also prefer the idea of representing loadshedding schedules as a visualized time series like a calendar, as opposed to just time tables, so kudos from me there too

When it comes to updating manually_specified.yaml I see you would need to accept a pull request ?

I'm trying to explore options of how we can make this dataset available for all open source projects to use, in way that is secure, easily accessible, and easily/quickly updatable by the public, without needing to be moderated by a single person who might not be available at the time a new schedule is released... but it would also need to be secure enough that it can not be easily manipulated by vandalism or malicious intent. I was thinking something along the lines of a blockchain or wiki ? Realistically this would be a distinct, purpose-build project, and hosted as some kind of API on a private web server independently of github.

A bit of an over-engineered approach? Then how about a Google Sheets document maintained by a group of volunteers with hierarchical ACL ?

Hope this isn't going too off-topic but I'll let you decide on that one!

Thanks

This idea comes from Cara van Coller on LinkedIn:

...imagine if you could get suggestions about places in your area to work at when load shedding hits...

And they linked to remotable: "Remotable helps you filter for spaces that suit your work needs no matter where you are. Download the app and find good wifi, coffee and parking" who I'm sure would appreciate knowing the loadshedding schedule.

Maybe an interesting thing for students would be to be able to connect with your friends, so that you can see when each of you have loadshedding and then go to study at whoever's flat has power.

Technical details

This will be dependant on the website #16 and possibly also a mobile app, which doesn't have an issue open yet. But definitely a very good idea that needs to have a solution nonetheless.

Commit 5857b2a incorrectly specified the month of the change, and was fixed by 80dc75e. However 80dc75e also incorrectly specified the date of the change, and was fixed by 5ba1e40.

Having a human-friendly summary which auto-replies to PR comments would help mitigate these errors, and remove the need for a human to manually try parse ISO datestrings.

Something like:

Proposed new loadshedding dates:

• Tuesday September 06 from 16:00 to 22:00
• Wednesday September 07 from 05:00 to 22:00
• Thursday September 08 from 05:00 to 22:00
• Friday September 09 from 05:00 to 22:00
• Saturday September 10 from 05:00 to 22:00

Would be very helpful. Possibly with links to the original tweet

Useful links:

https://stackoverflow.com/questions/58066966/commenting-a-pull-request-in-a-github-action

Request from: https://twitter.com/byronpbrummer/status/1570286440323366913?s=20&t=cPkCtM4dQ08G0M1WQDnaag

Source URL: https://www.durban.gov.za/pages/residents/load-shedding

Hi,

I don't see a loadshedding schedule for Paarl. Can you please help?

I think it would be useful to compile a list of suburb names and their corresponding loadshedding area names (as they appear on the calendar list). Something as simple as this:

This would be good to aid location lookups for each calendar since it's often difficult to know what schedule your suburb belongs to. Plus it would be cool to have a a single open source data file for this information (which is usually displayed on separate municipal websites such as this one)

Compiling such a .csv file would be a once off task and not too tedious. Most suburb block information is posted in PDFs or spreadsheets in some form.

I'd be happy to manually compile one and put it on this repo if it would be useful (kinda want to do it for myself anyway)

Related to #56 and #58

Currently the events are just named "Stage X Loadshedding" but that's not super informative. A better option might be to include the area name in the event name like "Stellenbosch Loadshedding Stage X". This raises two problems:

1. The name of the event is quite a bit longer, which could obscure a lot of the information on some phones/calendars
2. It's not immediately obvious what the area name should be. City of cape town area 42 can probably be shortened to CoCT42, but I'm hesitant to make long lists of abbreviations that people need to remember. And having the event name be "City Of Cape Town Area 42 Loadshedding Stage X" is definitely too much of a mouthful.

This might be a case of having to solve the problem 90% by just using the area name without the suburb (western-cape-stellenbosch.ics would become Stellenbosch Loadshedding Stage X) and then deal with the remainders on a case-by-case basis.

Currently, the build calendars workflow will run even if the formatting and compilation checking workflows fail. At best this is bad practice, at worst this could lead to the 45m of build-calendars runtime (and API calls) being wasted because they could fail due to the issues found in the format& compilation checking workflows.

Create a dependency (as described in this SO post https://stackoverflow.com/a/68078768/14555505) that will only build the calendars if all the checks pass.

Currently it's only possible to specify one loadshedding schedule for the entire country, but this isn't ideal because quite often cape town will be on a different schedule. This change would change the specification of the manually_specified.yaml file to allow an optional "include_regex: XXX" and "exclude_regex: XXXX" to specify which areas the change would apply to. For example:

  - stage: 6
    start: 2022-09-18T04:16:00
    finsh: 2022-09-20T23:59:00
    source: https://twitter.com/Eskom_SA/status/1571328907005120513
    include_regex: city-of-cape-town-area-.*

  - stage: 4
    start: 2022-09-18T04:16:00
    finsh: 2022-09-20T23:59:00
    source: https://twitter.com/Eskom_SA/status/1571328907005120513
    exclude_regex: city-of-cape-town-area-.*

This could get a bit tiresome, so a shorthand include: XXX and exclude:XXX would also be nice. So instead of specifying the full regex as above, you could just do:

  - stage: 6
    start: 2022-09-18T04:16:00
    finsh: 2022-09-20T23:59:00
    source: https://twitter.com/Eskom_SA/status/1571328907005120513
    exclude: coct

  - stage: 4
    start: 2022-09-18T04:16:00
    finsh: 2022-09-20T23:59:00
    source: https://twitter.com/Eskom_SA/status/1571328907005120513
    include: coct

But then it also gets a bit tiresome to have to specify both the inclusion and the exclusion, so maybe there's a way to only use the most specific schedule?

Currently there's no way to keep track of which calendars are being downloaded and which are not, or even how many times any calendar has been downloaded.

Having this knowledge might be important in the future for knowing which calendars are less used and therefore less critical to having higher uptime. Also having some visibility into how many people are using the calendars would be motivating for continuous improvements.

kefir500/github-release-stats or Somsubhra/github-release-stats might be able to solve this, although the download stats look to reset every time the assets are updated so there would have to be some way to cache the release stats in builld_calendars.yaml so that they don't get lost.

This issue is considered DONE when all unit tests are run as github actions every time there is a pull request, to ensure that the PR doesn't break anything.

mich-a-b

Congrats for being on the radio.
Could you also have a look at adding Stellenbosch/Cloetesville ?

Currently struggling to find a source of information about the loadshedding schedule in Cloetesville. There's this page, although I'm not sure when Stellenbosch central is shed by stellenbosch and when it's shed by Eskom

@mich-a-b could you confirm that you use the timetable in the link?

Alternatively, EskomSePush reference this link as their source of information for Cloetesville, although it doesn't seem to load:

http://stellenbosch.gov.za/news/notices/notices-engineering/6330-loadshedding-stage-1-4-schedule-2018/file

IA there a calendar for Ekhurleni?

Hi there, do you perhaps have an ics file for Milnerton WesternCape?

Thanks

Something I've noticed so far using your load shedding calendar feed is that some events were created (for Cape Town area 15) at a time that load shedding was projected but when that gets revised, the events are still present. For example, at some point, load shedding was scheduled today for 18:00-20:30 but at present, no load shedding is scheduled.

I'm not sure if it is possible to periodically "clean" out events where no load shedding is actually taking place?

IFTTT provides simple automation capabilities, and it looks like it's free to setup once you have an API (#10 ) so this would enable easier home automation

See this reddit suggestion: https://www.reddit.com/r/southafrica/comments/w3hq7v/get_loadshedding_schedules_in_your_calendar/igxyst0/

It would be great to have an API that developers could query to get the current loadshedding schedule for different areas. It's gonna take a while to get there though. And I'm not sure how it would be hosted.

But for now, we can always just curl github:

$ curl https://raw.githubusercontent.com/beyarkay/eskom-calendar/main/generated/western-cape-stellenbosch.csv
start_time,finsh_time,date_of_month,stage
00:00,02:30,1,0
00:00,02:30,2,0
00:00,02:30,3,0
00:00,02:30,4,0
00:00,02:30,5,1
00:00,02:30,6,2
00:00,02:30,7,3
00:00,02:30,8,4
00:00,02:30,9,0
...

Great job on this project @beyarkay!

I noticed that my Cape Town area calendar events are updated as national stages change, whereas I'd expect the events to be updated if and when the Cape Town stages change

Currently the rust script is entirely untested. This isn't great and makes it tricky for new users to submit PRs since they won't know if their changes break things. This issue should be considered DONE when there are rust unit tests over all working aspects of the code base.

Please can you add Madibeng for the North West province as I see it's not on the list.

Currently the GH actions scripts aren't automating the builds. There's a bunch of issues from obscure ones to API rate limits.

The deployment should be as follows:

1. A change is pushed to manually_specified.yaml
2. GitHub action starts, checking out the repo, downloading the python packages and rust crates
3. The project is built with cargo run --release, causing ics calendar files to be saved in calendars/
4. The tag latest should be moved to the latest commit.
5. All files under the relase corresponding to the latest tag should be deleted.
6. The ics calendar files from this build must be uploaded to the release corresponding to the latest tag

This should fully automate the updating of the calendar files. The trickery with the latest tag is so that github can maintain one URL (something like https://github.com/beyarkay/eskom-calendar/releases/tag/latest that points to the most recent and up-to-date calendars. This means that users won't have to update their calendar's URL every time there's a change from eskom.

Hey!

Would you be able to add the Raithby schedule as well. It falls under Stellenbosch.

EskomSePush has it as Raithby(8) with description Eskom Direct, Stellenbosch, Western Cape. Raithby doesn't fall under Stellenbosch's main schedule. It seems to fall under the Stellenbosch Farmers block 8 which includes:

Block 8: Cloetesville, De Zalze, Franschhoek, Jamestown, Raithby, Koelenhof, Kylemore, Kayamandi, Vlottenburg, Plankenburg, La Motte, Lanquedoc (Dwarsrivier), Lynedoch and Pniel

This is the best problem solved sinced sliced cheese! Well done champ!

Check that the rust code is properly formatted with rustfmt on a push, and deny the push if it isn't properly formatted.

Hi Boyd
Thank you for your app. Imported into Google Calander.

Seems to be a "duplication" error for some stage 5, CPT region 7.
Today the power was off 12:00-14:00 (first pic)
Second "duplication" tomorrow as exaple

Hi there,

Thank you for a great product.

It seems there is a discrepancy for city-power-11 or I am doing something wrong.

If I compare CityPowers schedule, there are four blocks of load shedding for this date, but only one block appears in the calendar.

https://www.citypower.co.za/customers/Pages/Load_Shedding_Downloads.aspx

Is there anything I can do?

The stages vary from one area to another. For example:

• National is currently at Stage 5,
• City of Cape Town is currently at Stage 3.

Can this be taken into account?

Hi Boyd

I see allot of potential for your code to be imported into MS Teams for team productivity management. Two updates would add great value and set you product above the rest:

1. NEW Custom Name/Area String directly below current Title
   Requirement: Several files would need to be imported and renamed with the Team members names that it affects. e.g.

• for Me and Team member X in CPT 7;
• you and team members Y and Z in City Power 1
• team member K, V, W in CPT 2

IT loads 3 files into MS Team and renames the title of each to:

• CPT reg 7 Stephan,X
• City Power 1 Boyd, Y, Z
• CPT 2 K,V,W

2. Custom Emoji/Icon
   Requirement: Some team members may have limited and other no access during loadshedding times. These can be indicated by assigning different Emojis/Icons.
   Example, you have limited connection on mobile device, use a data signal icon. I have no connection, use a No icon

Proposed solution:

1. For the quickest fix:
   a. both can be addressed by making the Title("Stage X Loadshedding" ) customisable
   b. Title should always default to your current: "Stage X Loadshedding" if no custom option has been set. But if a custom name has been set, it should NOT update with every refresh.
   c. Title length should be determined by the app it is used in (keep it default)
   d. adding a HOW TO manage teams availably, in your HelpGuide.

2. Future Improvements:
   a. You can later then develop two more customisable fields, separate from the title at a later stage, if more people request it. ;)

Hope this helps and once again thank you!

Currently there are workflows which attempt to catch formatting or compilation errors quickly before they cause the (API limiter) build-calendars workflow to fail.

This is a partial solution, as it requires one workflow for every type of error, and it's impossible to create a workflow for an error if you don't know the error even exists. Therefore this method can never catch all the errors that would cause the workflow to fail prematurely.

This issue requires the that the build-calendars workflow be split into two workflows, with the first one feeding into the second. The second workflow should contain only the process of actually uploading the files to GitHub and making the expensive API requests. The first workflow should contain everything else. This will include actually building the ICS files, which should then get passed as an input to the workflow which uploads those files. The second workflow should NOT rebuild the ICS files.

The first workflow should be run on every PR and commit to main, so that we can be sure the PR doesn't break anything.

this SO post has some useful information about setting up dependencies between workflows.

This comes from #56 (comment)

Problem statement

Currently every event's name is just based on the stage, which is fine if you've only got one area in your calendar:

But only gets more confusing if you've got multiple areas you want to keep track of (as is the case with a manager and team members):

So this change would allow the end user to customize the name of the events to suit their workflow.

Technical challenges

The ICS format (which describes the calendars) doesn't allow for the subscriber of a calendar (ie the users of eskom-calendar) to modify the details of the subscribed calendar. So to support this, we would need to generate a custom calendar for everyone who wanted custom names for their calendars. This would be an issue because making the Releases page even longer will frighten even more potential users.

This would become a non issue however if the project was properly hosted on a website. If that were the case, then users wanting a custom calendar can just toggle some options which get stored on the server and looked up every time the calendars are generated. And new users wouldn't have to be frightened because they could just search for their areas, and not have to see all the custom generated calendars.