Performance Issue - Clingo in concurrent threads about clingo HOT 4 CLOSED

Hi,
As Roland pointed out, continuously adding new program increments for new time steps won't scale. We noticed this problem too and to address it, we introduced in this paper a technique to implement a time-based sliding window. The primary idea is as follows:

• The discrete time steps are arranged in a cycle (modulo-ring) which loops after a fixed number of steps.
• The problem is encoded with rules containing only regular facts and a fixed horizon = the modulo-ring size
• At each time point t, arriving streamed data represented as external facts are continuously assigned (via equivalence) to corresponding regular facts for timestamp t modulo window-length.

With that, only the assignment rules have to be continuously grounded and added as new modules but not the whole problem encoding. Naturally, this won't scale ad infinitum either but should provide a notable improvement to your original approach.

from clingo.

Hi, I do not know much about stream reasoning but can tell you what the limitations of the clingo system are.

• Continuously adding new facts to a program and then ground and solve will certainly cause slow downs/failure at some point because the system is accumulating more and more memory.
• What can be done is to write an encoding that takes a fixed set of inputs (*). The inputs can then be assigned before solving. This approach will not consume unbounded amounts of memory but is harder to implement. Also if the domains of your inputs are too large the approach might be infeasible.
• If you can solve you problem in reasonable time when solving each problem from scratch, why bother with anything more complicated (except as a research topic).

(*) It is also possible to extend the inputs dynamically but this is getting complicated.

from clingo.

Pobermei and rkaminsk, thanks a lot for your answer.
I read the paper and understood the sliding window approach. Given streams of sensor data (as external predicates), the clingo solver (plus a python scripts handling the incremental behavior) in my application is used as a stream reasoner in order to recognize activities in a smart home. It means that for some activities the solver needs to keep track of some events (as the precondition of other upcoming events) for a long period of time e.g., 4 hours equivalent to timestamp t = 4*3600 (the resolution of timestamp is in second). I think for such situations, the sliding window solution doesn't seem effective. Right?

My second question: Is it possible to remove some obsolete predicates from the grounding set of the solver during the solving process?

Best

from clingo.

It means that for some activities the solver needs to keep track of some events (as the precondition of other upcoming events) for a long period of time e.g., 4 hours equivalent to timestamp t = 4*3600 (the resolution of timestamp is in second). I think for such situations, the sliding window solution doesn't seem effective. Right?

Representing so many time points with atoms will certainly not scale. You might want to have a look into clingcon, which can handle variables over (finite) numbers. But note that it does not support multi-shot solving yet.

My second question: Is it possible to remove some obsolete predicates from the grounding set of the solver during the solving process?

To remove something from a logic program, externals can be used. Releasing a. in

#external a.
b :- a.

will remove a and b from the program. By calling Control.cleanup(), both atoms will also be removed from clingo's domains and not used for grounding in later steps. Actually, anything the solver determined to be false is removed. But note that this does not free all memory associated with the atoms but will keep the symbolic representation in memory. I thought about also collecting such kind of memory but this is difficult to implement and won't happen anytime soon. The solving backend will also keep a small amount of memory because even though all constraints over a variable will be removed the variable as such is not.

from clingo.