Zimfarm at youzim.it doesn't show schedule names about zimfarm HOT 15 CLOSED

I'm ok with @rgaudin proposition, so since the decision is on us, we will implement it:

• store original_schedule_name in requested_tasks and tasks
• expose the original_schedule_name to the API along the schedule_name which will remain empty in youzimit. This way, we'd expose a truthful situation: there is no schedule_name (because there is no schedule) but there is an original one.
• adapt UI to a missing schedule_name and use the original one.

from zimfarm.

Tentative UI, feedbacks welcomed:

• new_recipe schedule has been deleted so it is not anymore a link
• targetedjustice.com_5f8aa391 schedule is still there
• this situation where both cases are present at the same time will usually never occur, since we either have all schedules still present (zimfarm) or no schedules present (youzimit)

from zimfarm.

Indeed, when we migrated from mongo to psql, we linked the [requested]tasks with the schedule that created them.
What was once a string recording the schedule name now points to the name of the linked-schedule.

This has no impact on Zimfarm because schedules are present but on youzim.it one, there's no link anymore as soon as you delete the schedule (which we do as soon as we request a task from the schedule).

We figured loosing the schedule name was a minor inconvenience because this zimfarm UI is solely used for debugging and was only helping when browsing the pipeline.

We could bring back that feature by either:

• storing the schedule_name in those tables and link via it instead of the ID
• storing both the schedule_id and the schedule_name. We'd have to maintain consistency on renames.
• adding a blank schedule_name that only gets filled on schedule removal

@benoit74 what do you think?

from zimfarm.

Thanks for the explanation. I agree the schedule name is somewhat useless, especially since clicking on it always just gave a 404 anyways (since the schedule was deleted). On the other hand, it gave a way to discriminate tasks when looking at a long list.

from zimfarm.

A fourth solution ;-)

Add an original_schedule_name field on requested_task and task that is populated only on task creation and used in the API to compute schedule_name when schedule is missing (i.e. has been deleted).

Benefits :

• more straightforward to populate this field at requested_task and task creation (rather than "not forgetting to set this at schedule deletion" or "maintain consistency")
• name with a lot of meaning
• no change on IDs / relational fields
• no need to maintain this value on schedule renaming, it is clearly the original name, and on youzimit the schedule is immediately deleted so the name won't change, and on the "regular" farm we usually keep the schedule so we can continue to use it

from zimfarm.

OK, works for me

from zimfarm.

Should we add another field schedule_missing in the task and requested_tadk API so that the UI knows about it and does not display a non-working link ?
I can work on it on Thursday at the latest.

from zimfarm.

We could add the schedule_id which is an actual information that might be useful later.
Hi could react to it to toggle link display.

from zimfarm.

Makes sense.
Btw, I realized that when filtering by schedule name, it will now necessary to also take into account the original_schedule_name data when there is no more schedule.

from zimfarm.

An other solution would be to delete the schedule somehow later (at the same time like the task?).

from zimfarm.

Indeed, this would probably make a lot of things easier.

We could :

• add a marked_for_deletion field on schedules
• refuse to delete a schedule if there are still associated task
• allow to mark a schedule for deletion
• when deleting tasks, also delete their schedule marked for deletion if there are no more linked task
• bring back integrity constraints on the database (a task must have an associated schedule)

I don't know how this would interact with the fact that we need more auditability of actions performed on the zimfarm, which could also lead to the need to keep older schedule configurations to check how the schedule has been configured.

from zimfarm.

Makes sense. Btw, I realized that when filtering by schedule name, it will now necessary to also take into account the original_schedule_name data when there is no more schedule.

We don't want to do that.
The point of having something clearly named original_schedule_name is to avoid things like this. But indeed as suggested above, that would have been just on the model and not on the API.
I suggest we expose the original_schedule_name to the API along the schedule_name which will remain empty. This way, we'd expose a truthful situation: there is no schedule_name (because there is no schedule) but there is an original one.
UI would adapt to a missing schedule_name and use the original one.

The filters thus needs not to be updated. We're not using it in UI anyway so it would not be visible.

An other solution would be to delete the schedule somehow later (at the same time like the task?).

The thing is that we don't delete the tasks and we don't want to. In particular for youzim.it which is mostly unsupervised, those tasks are useful for statistics or inquiry.

Also, it makes more semantic sense to delete the schedule because its role is just to repeatedly create tasks. So if it's a unique task, schedule has no reason to stay.

from zimfarm.

This regression is annoying and makes really difficult to find tasks at farm.youzim.it. I would appreciate if this is fixed soon.

I have no strong opinion about the way to fix it. But from high level I see that:

• The root recipe to make a task (we call it a schedule) is pretty much useless to keep over time for farm.youzim.it
• Unfortunately this use case has not been though 100% through (therefore this regression)
• Either we support/make it right OR we stop creating an "excpetional" situation (and we stop to delete shedules).
• I prefer to keep things simple and create exceptions only if absolutly necessary. Here we can have millions of rows in the DB without problems.

@benoit74 @rgaudin Please make a decision and fix this regression.

from zimfarm.

TODO is probably the least useful view for this. DOING/DONE/FAILED would have both the not-linked schedule name as well as the linked task ID

from zimfarm.

Yes, I just wanted to ensure that we are all aligned on this "worst case" (I'm fine with it).

from zimfarm.