Delete shared constants about scoop HOT 5 OPEN

I thought a bit about it. I guess deleting a constant on all workers might be difficult, because you need to ensure that the deletion is propagated to all workers in a reasonable amount of time. But (for me) it would already be sufficient to be able to delete a constant on the current worker.
In order to do that would it suffice to add the following function to the shared module?

def deleteConst(name):
    """ Deletes a constant with a particular `name` on the **current** worker. """
    for key in elements:
        elements[key].pop(name, None)

If so, I can open a pull request with these changes, including a unit test, of course.

from scoop.

Well, an alternative proposal is the following function:

def deleteConst(name):
    """Deletes a constant with a particular `name` on the current worker.

    Deletion does **not** propagate to other workers.
    Deleted variables can also not be reset by `setConst`.

    """
    for key in elements:
        var_dict = elements[key]
        if name in var_dict:
            var_dict[name] = None

To avoid potential side effects by the user trying to reset a deleted constant via shared.setConst.

from scoop.

Hello,
The issue with constant deletion is not exactly the propagation assertion, but the synchronization between the tasks. If an API offered constant deletion, a user could want to perform variable sharing (or "updating") through constant setting and deletion. In this case, there is no way to determine if the worker will use version 1 or version 2 of the variable when it executes. I guess this will wreck havoc in the code's logic and/or bewilder most of the non-acquainted with parallel programming.

I had planned integrating shared variables using vector clocks or playing around with the operational transformation, but lacked the resources to do so. The API that should be proposed is also not completely trivial, at least it is not evident to me.

I'm not quite sure of the use case for your second proposal? If I understand correctly, a worker can set a constant, and it is the job of every other to delete their constant when they don't need it anymore? This kind of asymmetry is something to think about in the API coherence... Aside from very large variables that could be dropped to free some memory, is there any other use cases?

from scoop.

Well, the use case I had in mind is where you have repeated but independent batches of tasks. So the first batch with multiple tasks is distributed among SCOOP workers with shared variables. These variables could be big. So after a full batch ended, all variables can be deleted to free memory and make space for the next batch.

from scoop.

I understand your use case, and I like your last example of deleteConst. It is indeed important not to allow the reset of a variable, otherwise non-aware users may simply fall into problems easily...
The only strange behaviour I see is that setConst applies to every worker, while deleteConst only applies to a the current worker.

Well, I think it could be interesting. Let me sleep over this decision and I'll keep you posted on any API change.

from scoop.