logic-and-learning-lab / popper Goto Github PK

View Code? Open in Web Editor NEW

185.0 19.0 29.0 22.56 MB

An inductive logic programming system

License: MIT License

Python 85.88% Prolog 14.12%

program-synthesis program-induction inductive-logic-programming

popper's People

Contributors

Stargazers

Watchers

popper's Issues

Add description to the examples

It would be great if the examples contains a description of the example. In earlier versions of Popper, most examples was quite clear (to me), but some of the newer iggp-* examples are not that clear.

The examples are surely described a little more in some of the Popper papers, but a (perhaps short) description would clarify what problem the example is solving.

Remove renaming redundancies in the hypothesis space

The Alan ASP program generator is non-optimal. For instance, Alan will generate this program, where the last two rules are equivalent after renaming the variables:

f(A) :- head(A,B),even(B),odd(B)
f(A) :- tail(A,B),f(B)
f(A) :- tail(A,B),tail(B,C),f(C)
f(A) :- tail(A,C),tail(C,B),f(B)

Likewise in the robots-linear problem, Alan generates this solution:

f(A,B):-right(A,D),right(C,E),right(D,C),right(E,B).
f(A,B):-right(A,D),right(C,B),right(D,E),right(E,C).
f(A,B):-right(A,E),right(C,B),right(D,C),right(E,D).
f(A,B):-right(A,E),right(C,D),right(D,B),right(E,C).
f(A,B):-right(A,C),right(C,D),right(D,E),right(E,B).
f(A,B):-right(A,C),right(C,E),right(D,B),right(E,D).

In the sorted problem, Alan generates this solution:

f(A) :- tail(A,B),empty(B).
f(A) :- head(A,C),tail(A,D),head(D,B),gt(B,C),f(D).
f(A) :- head(A,B),tail(A,D),head(D,C),gt(C,B),f(D).

In the trains problem, Alan generates this solution:

f(A):-circle(B),circle(C),has_load(A,B),has_load(A,C).

We need to improve the ALAN encoding to reduce the number of redundant programs.

max_vars(7) don't generate a solution with variable G

Here is a problem where Popper don't generate/test the combination that should be the solution, or more specific: the variable G is not generated even though max_vars is set to 7.

The setup is to solve the following puzzle:

1 + 4 = 5
2 + 5 = 12
3 + 6 = 21
5 + 8 = ??

(A variant that works as expected is here: http://hakank.org/popper/facebook_puzzle/ ).

In this variant the problem is represented by a 6 arity predicate f(A,B,C,D,E,F) with these positive examples:

%% exs.pl
%% pos(f( A, B, C, D, E, F)).
pos(f( 0, 0, 0, 1, 4, 5)).
pos(f( 1, 4, 5, 2, 5,12)).
pos(f( 2, 5,12, 3, 6,21)).

There are several answers to the puzzle, but I'm now looking for the solution where ?? is 34 by the following

F = C + D + E   % 21 + 5 + 8 = 34

This should be - unless I'm missing something - the predicate

f(A,B,C,D,E,F) :- plus(C,D,G), plus(E,G,F).

But Popper don't find this - or any - solution. Here's the encoding.

The BK as

%% bk.pl
plus(X,Y,Z) :-
        integer(X),
        integer(Y),
        Z is X + Y.

%% Also tested this:
%% :- use_module(library(clpfd)).
%% plus(X,Y,Z) :- 
%%      Z #= X + Y.
%%

and the bias as:

%% bias.pl
max_vars(7).
max_body(2).
max_clauses(1).

head_pred(f,6).
body_pred(plus,3).

Popper generates 360 programs, but does not generate the candidate predicate plus(C,D,G). In fact, it doesn't generate any solution with the variable G. Given that max_vars allow 7 variables, I had expected that Popper should try some program with variable G.

Have I missed how max_vars/1 works?

Interestingly, when changing to max_body(3) (from max_body(2)), then there are several candidate programs that includes the variable G for bodies with 3 predicates, but no solution is found (with a timeout 45 minutes and over 65 000 generated programs).

bkcons filepath

bkcons.py assumes that popper is being invoked from the root directory of the repo, here:

with open('popper/lp/cons.pl') as f:

possible fix:

from pathlib import Path

cons_file = Path(__file__).parent / "lp" / "cons.pl"
with open(cons_file) as f:

Wrong pruning during the search?

Hello,

I have not read the codes, but according to the description in the paper, the learner should give the best solution. During my test, the learned predicate is not stable for the same setting. The test case and the results are as follows.

%%bias.pl
max_vars(5).
max_body(5).
max_clauses(2).
enable_recursion.

head_pred(f,2).
body_pred(nullptr,1).
body_pred(delete,3).
body_pred(my_min_list,2).
body_pred(empty,1).
body_pred(value,2).
body_pred(pointer,2).

type(f,(element,list)).
type(nullptr,(element,)).
type(delete,(list,integer,list)).
type(my_min_list,(list,integer)).
type(empty,(list,)).
type(value,(element,integer)).
type(pointer,(element,element)).

%%bk.pl
use_module(library(lists)).

empty([]).
zero(0).
one(1).

value(n1,1).
value(n2,2).
value(n3,3).
value(n4,4).

pointer(n1, n2).
pointer(n2, n3).
pointer(n3, n4).
pointer(n4, null).
nullptr(null).

my_min_list(List, Num):-
    ground(List), min_list(List, Num).

%%exs.pl
pos(f(n1,[4,3,1,2])).
pos(f(n2,[4,3,2])).
pos(f(n3,[3,4])).
pos(f(n4,[4])).
pos(f(null,[])).

neg(f(n1,[4,1,2])).
neg(f(n1,[4,3,5,2])).
neg(f(n1,[4,3,2])).
neg(f(n2,[4,3,2,7])).
neg(f(n3,[3,4,5])).
neg(f(n4,[])).

The program won't run for long when it returns the correct result. (with --info flag)

Thread 1 (main): foreign predicate system:is/2 did not clear exception: 
        time_limit_exceeded
% NEW BEST PROG 1:
f(A,B):-my_min_list(B,C),value(A,C).
% Precision:0.57, Recall:0.80, TP:4, FN:1, TN:3, FP:3

% NEW BEST PROG 309:
f(A,B):-empty(B),nullptr(A).
f(A,B):-pointer(A,C),my_min_list(B,E),delete(B,E,D),f(C,D).
% Precision:0.83, Recall:1.00, TP:5, FN:0, TN:5, FP:1


% BEST PROG 891:
f(A,B):-empty(B),nullptr(A).
f(A,B):-pointer(A,C),my_min_list(B,D),delete(B,D,E),value(A,D),f(C,E).
% Precision:1.00, Recall:1.00, TP:5, FN:0, TN:6, FP:0

Wrong result

Thread 1 (main): foreign predicate system:is/2 did not clear exception: 
        time_limit_exceeded
% NEW BEST PROG 1:
f(A,B):-value(A,C),my_min_list(B,C).
% Precision:0.57, Recall:0.80, TP:4, FN:1, TN:3, FP:3

% NEW BEST PROG 261:
f(A,B):-nullptr(A),empty(B).
f(A,B):-nullptr(C),value(A,D),my_min_list(B,D),pointer(A,C).
% Precision:1.00, Recall:0.40, TP:2, FN:3, TN:6, FP:0


% BEST PROG 16022:
f(A,B):-nullptr(A),empty(B).
f(A,B):-nullptr(C),value(A,D),my_min_list(B,D),pointer(A,C).
% Precision:1.00, Recall:0.40, TP:2, FN:3, TN:6, FP:0

Noise handling

Thanks for making Popper!

An extra feature i would find useful is the ability to handle noise. I tried this but i get no theory.

bias.pl

max_vars(3).
max_body(2).

head_pred(grandparent,2).
body_pred(parent,2).

bk.pl

parent(abe,homer).
parent(homer,lisa).
parent(homer,bart).
parent(clancy,marge).
parent(marge,maggy).
parent(marge,lisa).
parent(marge, bart).

exs.pl

pos(grandparent(abe,bart)).
pos(grandparent(abe,lisa)).
pos(grandparent(clancy,lisa)).
pos(grandparent(clancy,bart)).
neg(grandparent(homer,bart)).
neg(grandparent(abe,homer)).
neg(grandparent(homer,bart)).
neg(grandparent(bart,lisa)).
neg(grandparent(clancy,maggy)). % noise

Order stats by total running time

It is only a minor change but it would be useful to order the stats output by the total duration:

Popper/popper/util.py

Line 235 in 46fe4dc

def show(self):

What does max_clauses(N) do exactly?

I am trying to learn a program that does addition by succession, and I am trying to limit the number of rules generated (which is supposedly what max_clauses(N) is for) to 2 clauses. I have added 'max_clauses(2).' to my bias file, but I am still getting programs that are more than 2 rules. Is there a way to limit this? or am I misusing the max_clauses(N) parameter?

ex prog generated:

y_plus(A,B,C):- y_succ(A,C),y_prec(B,C).
y_plus(A,B,C):- y_succ(B,C),y_prec(A,C).
y_plus(A,B,C):- y_prec(B,D),y_succ(D,C),y_prec(A,D).
y_plus(A,B,C):- y_prec(B,D),y_succ(D,B),y_succ(A,C).
y_plus(A,B,C):- y_prec(B,D),y_succ(D,C),y_succ(A,D).
y_plus(A,B,C):- y_succ(B,D),y_plus(D,A,C).
y_plus(A,B,C):- y_succ(A,D),y_plus(B,D,C).

my bias file is as follows:

max_clauses(2).
max_vars(5).
max_body(3).
enable_recursion.

head_pred(y_plus,3).
body_pred(y_plus,3).
body_pred(y_succ,2).
body_pred(y_prec,2).

direction(y_plus,(in,in,out)).
direction(y_succ,(in,out)).
direction(y_prec,(in,out)).

type(y_plus,(integer,integer,integer)).
type(y_succ,(integer,integer)).
type(y_prec,(integer,integer)).

Windows compatibility

Windows does not allow the use of singal.SIGALRM when enforcing timeout. Would need to be changed in order to allow Window's users to run Popper.

Popper/popper/util.py

Line 44 in 4611db5

signal.signal(signal.SIGALRM, handler)

Timeout

We need to implement a timeout feature. The original Popper version has one, but it is more complicated than it sounds because we need to interrupt Clingo.

The reason to force ordering in predicate invention?

Hello,

I find the current code forbids the learning of mutual recursive predicates by "force ordering" in the alan.pl.

Popper/popper/lp/alan.pl

Line 591 in 91256b2

%% FORCE ORDERING

I don't get the reason for adding it, and some learning can be done if removing the restriction. Could you tell me something about that?

BTW, may I know the status of the issue #25? Has the bug been located? Currently, I am using some tricks to skip the direction. (As in issue #36, I use the ground predicate rather than the direction, and run multiple instances of learning. The correct one can be learned if lucky enough.) I would appreciate it if you can offer some details of the bug.

Predicate invention does not work when directions are not given.

There is a problem with predicate invention. When directions are given for background relations, predicate invention does not work.

The example iggp-minimal-decay don't work anymore. Shows "No program returned."

The example examples/iggp-minimal-decay don't work anymore. After 1 minute it shows "No program returned"; same result with and without the --test-all flag.

(I know that It worked as of 2021-04-29.)

ERROR: Syntax error: Operator expected

When running some of the examples, e.g. examples/reverse, the following error is thrown:

$ popper.py --info examples/reverse
ERROR: Syntax error: Operator expected
ERROR: assertz((pos_index(3,f([Ato
ERROR: ** here **
ERROR: m('324485'), Atom('443525'), Atom('544261')], [Atom('544261'), Atom('443525'), Atom('324485')])))). 
Traceback (most recent call last):
  File "popper.py", line 180, in <module>
    timeout(popper, (settings, stats), timeout_duration=int(settings.timeout))
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/util.py", line 51, in timeout
    result = func(*args, **kwargs)
  File "popper.py", line 122, in popper
    tester = Tester(settings)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 17, in __init__
    self.load_basic()
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 52, in load_basic
    self.load_examples()
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 36, in load_examples
    self.prolog.assertz(f'pos_index({i},{x})')
  File "/usr/local/lib/python3.7/site-packages/pyswip-0.2.10-py3.7.egg/pyswip/prolog.py", line 152, in assertz
  File "/usr/local/lib/python3.7/site-packages/pyswip-0.2.10-py3.7.egg/pyswip/prolog.py", line 129, in __call__
pyswip.prolog.PrologError: Caused by: 'assertz((pos_index(3,f([Atom('324485'), Atom('443525'), Atom('544261')], [Atom('544261'), Atom('443525'), Atom('324485')])))).'. Returned: 'error(syntax_error(operator_expected), string(b"assertz((pos_index(3,f([Atom('324485'), Atom('443525'), Atom('544261')], [Atom('544261'), Atom('443525'), Atom('324485')])))). ", 27))'.

I think it's a very recent error since I'm quite sure I tested all the examples some days ago without any problems.

Show full hypothesis space

I often use Clingo + Alan to check the full hypothesis space, with the rather convoluted approach of calling clingo alan.pl bias.pl -n0 | alanp where alanp is a script I have that turns answer sets into Prolog programs.

It might be useful to add this feature to Popper, so the user could simply call something like:

python popper.py folder -hspace 100 (or something similar)

Info does not show the last program

When running Popper with the --info flag, the final program is not included in the output

Partial directions

Popper currently does not support partial directions in the bias file. All or none of the directions must be provided, providing only some of them might result in failure to find a solution.

For examples, adding the following directions to the minimal decay example prevents to find any model:

%direction(next_value,(in,out)).
direction(does,(in,out,out)).
direction(my_true_value,(in,out)).
direction(my_succ,(out,out)).
direction(c_pressButton,(in,)).
direction(c_noop,(in,)).
direction(c_player,(in,)).
direction(c1,(in,)).
direction(c2,(in,)).
direction(c3,(in,)).
direction(c4,(in,)).
direction(c5,(in,)).

Context Dependent Problems

Hi!

Lets say I am working with a graph structure, but in every instance the nodes and edges are different.
The problem would remain the same, discovering some relation, but the background info changes from instance to instance.
Is it possible to define this kind of problem in Popper?

invoking popper from another python program

it looks like this line in Settings will never get hit, because it's within the if kbpath == False: condition. for this reason, it doesn't seem currently possible to run Popper from another Python program (as in the README). would it be okay to move the line to outside the condition?

Add ability to test programs using Clingo rather than Prolog

Self-explanatory. The only tricky part is continually adding and removing rules from the Clingo instance, but @rolfmorel knows how to do this by disabling rules.

PrologError: error(resource_error(stack)...) in Popper v2.0.0

The following examples throws an error.

bias.pl

max_vars(6).
max_body(4).
max_clauses(4).

% head
head_pred(target,2).
type(target,(element,element)).
direction(target,(in,out)).


% body
body_pred(target,2).
type(target,(element,element)).
direction(target,(in,out)).

body_pred(succ,2).
type(succ,(element,element)).
direction(succ,(in,out)).

bk.pl (empty)

exs.pl

pos(target(1,3)).
pos(target(2,5)).
pos(target(3,7)).
pos(target(4,10)).
pos(target(5,9)).
pos(target(6,8)).
pos(target(7,9)).
pos(target(8,10)).
pos(target(9,10)).


neg(target(3,1)).
neg(target(7,1)).
neg(target(2,2)).
neg(target(8,2)).
neg(target(4,3)).
neg(target(9,3)).
neg(target(4,0)).
neg(target(10,4)).
neg(target(5,5)).
neg(target(6,5)).

The error:

$ time python3 popper.py hakank/less_than
23:10:08 Num. pos examples: 9
23:10:08 Num. neg examples: 10
23:10:08 Searching programs of size: 2
23:10:08 ********************
23:10:08 New best hypothesis:
23:10:08 tp:1 fn:8 size:2
23:10:08 target(A,B):- succ(A,B).
23:10:08 ********************
23:10:08 Searching programs of size: 3
23:10:08 ********************
23:10:08 New best hypothesis:
23:10:08 tp:5 fn:4 size:5
23:10:08 target(A,B):- succ(A,B).
23:10:08 target(A,B):- succ(C,B),succ(A,C).
23:10:08 ********************
23:10:08 Searching programs of size: 4
23:10:08 ********************
23:10:08 New best hypothesis:
23:10:08 tp:6 fn:3 size:9
23:10:08 target(A,B):- succ(A,B).
23:10:08 target(A,B):- succ(C,B),succ(A,C).
23:10:08 target(A,B):- succ(C,D),succ(A,C),succ(D,B).
23:10:08 ********************
Traceback (most recent call last):
  File "popper.py", line 8, in <module>
    prog, score, stats = learn_solution(settings)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/loop.py", line 198, in learn_solution
    timeout(settings, popper, (settings,), timeout_duration=int(settings.timeout),)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/util.py", line 65, in timeout
    result = func(*args, **kwargs)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/loop.py", line 83, in popper
    pos_covered, inconsistent = tester.test_prog(prog)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 83, in test_prog
    pos_covered = frozenset(self.query('pos_covered(Xs)', 'Xs'))
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 14, in query
    return set(next(self.prolog.query(query))[key])
  File "/usr/local/lib/python3.7/site-packages/pyswip/prolog.py", line 129, in __call__
    "Returned: '", str(term), "'."]))
pyswip.prolog.PrologError: Caused by: 'pos_covered(Xs)'. Returned: 'error(resource_error(stack), Atom('27909')(stack_overflow, 6340089, choicepoints, 6340095, depth, 6340092, environments, 49582, globalused, 990639, localused, [Functor(565645,3,6340095,:(user, target(6340092, 9)),[]), Functor(565645,3,6340094,:(user, target(6340091, 9)),[])], non_terminating, 1048576, stack_limit, 0, trailused))'.

(Perhaps this is related to #43?)

Index Out of Range Error

Hi, I'm testing a simple program, but, as the title says, I get an index out of range error.
These are my files, stored in a folder called test:

bk.pl

bird(alice).
bird(betty).
can(alice,fly).
can(betty,swim).
ability(fly).
ability(swim).

not_can(X):- \+can(X).

exs.pl

pos(penguin(betty)).
neg(penguin(alice)).

bias.pl

head_pred(penguin,1).

body_pred(bird(bird), 1).
body_pred(can(bird,ability), 4).

type(penguin,(bird)).
type(bird,(bird)).
type(can(bird,ability)).

Here the command I run

> python3 p3 popper.py examples/test/ --debug
15:35:27 Max rules: 1
15:35:27 Max vars: 6
15:35:27 Max body: 6
15:35:27 Num. pos examples: 1
15:35:27 Num. neg examples: 1
Traceback (most recent call last):
  File "popper.py", line 8, in <module>
    prog, score, stats = learn_solution(settings)
  File "/mnt/c/Users/damia/Desktop/Ricerca/Repos/Popper/popper/loop.py", line 206, in learn_solution
    timeout(settings, popper, (settings,), timeout_duration=int(settings.timeout),)
  File "/mnt/c/Users/damia/Desktop/Ricerca/Repos/Popper/popper/util.py", line 63, in timeout
    result = func(*args, **kwargs)
  File "/mnt/c/Users/damia/Desktop/Ricerca/Repos/Popper/popper/loop.py", line 59, in popper
    generator = Generator(settings, grounder)
  File "/mnt/c/Users/damia/Desktop/Ricerca/Repos/Popper/popper/generate.py", line 48, in __init__
    solver.ground([('base', [])])
  File "/home/damiano/anaconda3/envs/deepstochlog/lib/python3.7/site-packages/clingo/control.py", line 339, in ground
    self._rep, c_parts, len(parts), c_cb, c_data), data)
  File "/home/damiano/anaconda3/envs/deepstochlog/lib/python3.7/site-packages/clingo/_internal.py", line 65, in _handle_error
    raise RuntimeError(msg)
RuntimeError: Traceback (most recent call last):
  File "/home/damiano/anaconda3/envs/deepstochlog/lib/python3.7/site-packages/clingo/script.py", line 202, in _pyclingo_call
    _ffi.new_handle(_PYTHON_SCRIPT))
  File "/home/damiano/anaconda3/envs/deepstochlog/lib/python3.7/site-packages/clingo/script.py", line 165, in _pyclingo_script_call
    ret = script.call(_py_location(location), py_name, py_args)
  File "/home/damiano/anaconda3/envs/deepstochlog/lib/python3.7/site-packages/clingo/script.py", line 139, in call
    return fun(*arguments)
  File "<string>", line 3, in pytype
IndexError: list index out of range

Problem running v2.0.0 (pyswip related)

When running Popper v2.0.0 I got this error:

$ python3 popper.py examples/alleven

Traceback (most recent call last):
  File "popper.py", line 4, in <module>
    from popper.loop import learn_solution
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/loop.py", line 4, in <module>
    from . tester import Tester
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 6, in <module>
    from pyswip import Prolog
  File "/usr/local/lib/python3.7/site-packages/pyswip/__init__.py", line 29, in <module>
    from pyswip.prolog import Prolog
  File "/usr/local/lib/python3.7/site-packages/pyswip/prolog.py", line 28, in <module>
    from pyswip.core import *
  File "/usr/local/lib/python3.7/site-packages/pyswip/core.py", line 849, in <module>
    PL_mark_string_buffers = _lib.PL_mark_string_buffers
  File "/usr/local/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /usr/lib/libswipl.so.7.6: undefined symbol: PL_mark_string_buffers

Environment:

Popper v2.0.0 (commit 88a945d)
Linux Ubuntu 18.04
Python v3.7.6 (it's the same problem with v3.9.4)
pyswip: git master version (installed via )pip install git+https://github.com/yuce/pyswip@master#egg=pyswip). Note that the simple test program at https://github.com/yuce/pyswip works without any problems.
SWI-prolog v7.6.4. This is the version installed via apt (Note: I downgraded from the PPA stable version of SWI-Prolog which throws the errors mentioned in #28 )

Can Popper performs incrementally learning?

Dear developers,

I am trying to find a state-of-the-art ILP system for incremental learning. As far as I know, Popper is a state-of-the-art system. But I cannot understand whether it can perform incremental learning from the manual.

If it cannot, would you like to suggest me another state-of-the-art ILP system for incremental learning?

Crash using v1.0.2 (problem with swi-prolog?)

I just updated to the brand new Popper version v1.0.2 and got the following errors when running any examples:

python3 popper.py examples/alleven 

[Thread 1 (main) at Thu Dec 16 09:42:29 2021] /build/swi-prolog-367kVb/swi-prolog-8.5.3-1-g8692fbe0e/src/pl-fli.c:2556: PL_put_chars: Assertion failed: 0
C-stack trace labeled "assert_fail":
  [0] PL_advance_hash_table_enum() at ??:? [0x7f826fc927a6]
  [1] PL_unify_mpq() at ??:? [0x7f826fc52f80]
  [2] PL_put_chars() at ??:? [0x7f826fc75f08]
  [3] ffi_call_unix64() at ??:? [0x7f8272c46dec]
  [4] ffi_call() at ??:? [0x7f8272c45f55]
  [5] _call_function_pointer() at /home/hakank/python/Python-3.7.6/Modules/_ctypes/callproc.c:850 [0x7f8271cea079]
  [6] PyCFuncPtr_call() at /home/hakank/python/Python-3.7.6/Modules/_ctypes/_ctypes.c:4017 [0x7f8271ce70c3]
  [7] python3(_PyObject_FastCallKeywords+0x92) [0x55aaf779dea2]
  [8] python3(_PyEval_EvalFrameDefault+0x4935) [0x55aaf780ee75]
  [9] python3(+0x1c4658) [0x55aaf7868658]
  [10] python3(+0x16526a) [0x55aaf780926a]
  [11] python3(_PyMethodDef_RawFastCallKeywords+0xe9) [0x55aaf779d529]
  [12] python3(_PyCFunction_FastCallKeywords+0x20) [0x55aaf779d420]
  [13] python3(_PyEval_EvalFrameDefault+0x4643) [0x55aaf780eb83]
  [14] python3(_PyEval_EvalCodeWithName+0x2de) [0x55aaf7809afe]
  [15] python3(_PyFunction_FastCallKeywords+0x1bc) [0x55aaf779f07c]
  [16] python3(_PyEval_EvalFrameDefault+0x49eb) [0x55aaf780ef2b]
  [17] python3(_PyFunction_FastCallKeywords+0xf2) [0x55aaf779efb2]
  [18] python3(_PyEval_EvalFrameDefault+0x517) [0x55aaf780aa57]
  [19] python3(_PyObject_Call_Prepend+0x2db) [0x55aaf779f8fb]
  [20] python3(+0x13aea9) [0x55aaf77deea9]
  [21] python3(+0x137b94) [0x55aaf77dbb94]
  [22] python3(_PyObject_FastCallKeywords+0x92) [0x55aaf779dea2]
  [23] python3(_PyEval_EvalFrameDefault+0x4935) [0x55aaf780ee75]
  [24] python3(_PyFunction_FastCallDict+0x164) [0x55aaf7703464]
  [25] python3(_PyEval_EvalFrameDefault+0x1bc7) [0x55aaf780c107]
  [26] python3(_PyEval_EvalCodeWithName+0x6a3) [0x55aaf7809ec3]
  [27] python3(_PyFunction_FastCallKeywords+0x248) [0x55aaf779f108]
  [28] python3(_PyEval_EvalFrameDefault+0x13ad) [0x55aaf780b8ed]
  [29] python3(_PyFunction_FastCallKeywords+0xf2) [0x55aaf779efb2]
  [30] python3(_PyEval_EvalFrameDefault+0x64d) [0x55aaf780ab8d]
  [31] python3(_PyEval_EvalCodeWithName+0x2de) [0x55aaf7809afe]
  [32] python3(PyEval_EvalCode+0x23) [0x55aaf7809813]
  [33] python3(+0x208f22) [0x55aaf78acf22]
  [34] python3(PyRun_FileExFlags+0x9d) [0x55aaf78ad2ed]
  [35] python3(PyRun_SimpleFileExFlags+0x1b6) [0x55aaf78ad0f6]
  [36] python3(+0x1dfe44) [0x55aaf7883e44]
  [37] python3(_Py_UnixMain+0x3c) [0x55aaf7883adc]
  [38] __libc_start_main() at /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:344 [0x7f82743d1bf7]
  [39] python3(_start+0x2a) [0x55aaf78839ba]
[1]    12676 abort (core dumped)  python3 popper.py examples/alleven

What I can interpret of this is that is a problem with SWI-Prolog. I'm using the current verson 8.5.3. Would that be a problem?

My environment:

Linux Ubuntu 18.05
Python3.7.6
Popper is installed via pip3 install -U popper
Clingo is installed via pip3 install -U clingo
SWI-Prolog vv8.5.3

Too many stacked strings

When running the minimal decay problem, there is the exception:

[FATAL ERROR: at Wed Apr 21 09:44:38 2021
    Too many stacked strings]

This error is to do with pyswip. I can replicate it. It does not happen with pylo2. We will therefore migrate from pswip to pylo2.

Problem with SWI Prolog's new stable PPA version 9.0.0 (Linux Ubuntu)

Yesterday, SWI Prolog's stable PPA version was updated to version 9.0.0 (from v8.4.3) and now Popper does work:

$ python3 popper.py examples/alleven
[Thread 1 (main) at Fri Nov 25 10:52:17 2022] /build/swi-prolog-hvyroz/swi-prolog-9.0.0-1-g0757fd884/src/pl-fli.c:2637: PL_put_chars: Assertion failed: 0
C-stack trace labeled "assert_fail":
  [0] PL_scan_options() at ??:? [0x7f692f994a98]
  [1] PL_unify_mpq() at ??:? [0x7f692f951fdf]
  [2] PL_put_chars() at ??:? [0x7f692f97582d]
  [3] ffi_call_unix64() at ??:? [0x7f6941588dec]
  [4] ffi_call() at ??:? [0x7f6941587f55]
  [5] _call_function_pointer() at /home/hakank/python/Python-3.7.6/Modules/_ctypes/callproc.c:850 [0x7f693dce0079]
  [6] PyCFuncPtr_call() at /home/hakank/python/Python-3.7.6/Modules/_ctypes/_ctypes.c:4017 [0x7f693dcdd0c3]
  [7] python3(_PyObject_FastCallKeywords+0x92) [0x561a29bf4ea2]
  [8] python3(_PyEval_EvalFrameDefault+0x4935) [0x561a29c65e75]
  [9] python3(+0x1c4658) [0x561a29cbf658]
  [10] python3(+0x16526a) [0x561a29c6026a]
  [11] python3(_PyMethodDef_RawFastCallKeywords+0xe9) [0x561a29bf4529]
  [12] python3(_PyCFunction_FastCallKeywords+0x20) [0x561a29bf4420]
  [13] python3(_PyEval_EvalFrameDefault+0x4643) [0x561a29c65b83]
  [14] python3(_PyEval_EvalCodeWithName+0x2de) [0x561a29c60afe]
  [15] python3(_PyFunction_FastCallKeywords+0x1bc) [0x561a29bf607c]
  [16] python3(_PyEval_EvalFrameDefault+0x49eb) [0x561a29c65f2b]
  [17] python3(_PyObject_Call_Prepend+0x2db) [0x561a29bf68fb]
  [18] python3(+0x13aea9) [0x561a29c35ea9]
  [19] python3(+0x137b94) [0x561a29c32b94]
  [20] python3(_PyObject_FastCallKeywords+0x92) [0x561a29bf4ea2]
  [21] python3(_PyEval_EvalFrameDefault+0x4935) [0x561a29c65e75]
  [22] python3(_PyEval_EvalCodeWithName+0x6a3) [0x561a29c60ec3]
  [23] python3(_PyFunction_FastCallDict+0x300) [0x561a29b5a600]
  [24] python3(_PyEval_EvalFrameDefault+0x1bc7) [0x561a29c63107]
  [25] python3(_PyEval_EvalCodeWithName+0x6a3) [0x561a29c60ec3]
  [26] python3(_PyFunction_FastCallKeywords+0x248) [0x561a29bf6108]
  [27] python3(_PyEval_EvalFrameDefault+0x13ad) [0x561a29c628ed]
  [28] python3(_PyFunction_FastCallKeywords+0xf2) [0x561a29bf5fb2]
  [29] python3(_PyEval_EvalFrameDefault+0x64d) [0x561a29c61b8d]
  [30] python3(_PyEval_EvalCodeWithName+0x2de) [0x561a29c60afe]
  [31] python3(PyEval_EvalCode+0x23) [0x561a29c60813]
  [32] python3(+0x208f22) [0x561a29d03f22]
  [33] python3(PyRun_FileExFlags+0x9d) [0x561a29d042ed]
  [34] python3(PyRun_SimpleFileExFlags+0x1b6) [0x561a29d040f6]
  [35] python3(+0x1dfe44) [0x561a29cdae44]
  [36] python3(_Py_UnixMain+0x3c) [0x561a29cdaadc]
addr2line: DWARF error: section .debug_info is larger than its filesize! (0x93ef57 vs 0x530ea0)
  [37] __libc_start_main() at ??:? [0x7f6942d77083]
  [38] python3(_start+0x2a) [0x561a29cda9ba]
[1]    44398 abort (core dumped)  python3 popper.py examples/alleven

So it seems that there is a problem with the pyswip package (again), and I'm trying to find a way to downgrade to the previous SWI Prolog stable PPA version.

(I also tried to downgrade the pyswip version, but all the versions I tried - 0.2,9, 0..2.10, 0.2.11 - yield some error.)

Some of the predicate invention examples don't work in latest version

Here is the output of the classic kinship example. I have checked that no PI is performed during the search.

Precision:1.00 Recall:1.00 TP:19 FN:0 TN:0 FP:0 Size:13
ancestor(A,B):- father(A,B).
ancestor(A,B):- mother(A,B).
ancestor(A,B):- father(C,B),mother(A,C).
ancestor(A,B):- father(A,C),ancestor(C,B).
ancestor(A,B):- father(C,B),ancestor(A,C).

If this is not easy to fix, could you tell me which commit is the most recent one whose PI works?

Adding direction to unneccessary predicate seems to cause stopping finding a solution

If I have bk
edge(1,2). edge(2,3). edge(3,4). edge(5,5). geq(A,B):- nonvar(A), nonvar(B), A >= B.

with bias
head_pred(path,2). body_pred(edge,2).

and examples
pos(path(1,2)). pos(path(1,3)). pos(path(2,4)). neg(path(1,5)). neg(path(2,5)).

then Popper will find that desired fact that path is the transitive closure of edge.
If I add to bias body_pred(gte,2). then it will find something else, due to the small number of example.
But if I also add direction(gte,(in,in)). then it will halt straight away, exploring no hypotheses, saying "NO SOLUTION FOUND"

Presumably I'm setting up something wrong here, but I can't see at all what it is.

A PrologError raise in function success_set when using build-in predicates

Hello. I tried using the system to learn some properties (expressed by predicates), and the python program returned some error messages. For simplicity, I attempted to locate the problem, and found the issue was accompanied with the use of build-in predicates.

Example1 (with error)

%% The expected predicate is 
%% f(Num, Lst) :- succ(A, Num), last(Lst, A).

%% bias.pl as follows
max_vars(3).
max_body(3).
max_clauses(2).

head_pred(f,2).
body_pred(last,2).
body_pred(succ,2).

%% no content needed in bk.pl, exs.pl as follows
pos(f(3,[4,3,1,2])).
pos(f(3,[4,3,2])).
pos(f(5,[3,4])).

neg(f(1,[4,1,2])).
neg(f(1,[4,3,5,2])).
neg(f(2,[4,3,2])).
neg(f(5,[4,3,2,7])).
neg(f(2,[3,4,5])).

returns the following messages.

Traceback (most recent call last):
  File "popper.py", line 11, in <module>
    _prog, stats = learn_solution(settings)
  File "/home/ydl/Popper/popper/loop.py", line 181, in learn_solution
    timeout(popper, (settings, stats), timeout_duration=int(settings.timeout))
  File "/home/ydl/Popper/popper/util.py", line 47, in timeout
    result = func(*args, **kwargs)
  File "/home/ydl/Popper/popper/loop.py", line 142, in popper
    conf_matrix = tester.test(program)
  File "/home/ydl/Popper/popper/tester.py", line 90, in test
    covered.update(self.success_set([rule]))
  File "/home/ydl/Popper/popper/tester.py", line 83, in success_set
    self.seen_prog[prog_hash] = set(next(self.prolog.query('success_set(Xs)'))['Xs'])
  File "/home/ydl/.local/lib/python3.8/site-packages/pyswip/prolog.py", line 126, in __call__
    raise PrologError("".join(["Caused by: '", query, "'. ",
pyswip.prolog.PrologError: Caused by: 'success_set(Xs)'. Returned: 'error(type_error(integer, [4, 3, 1, 2]), context(:(system, /(succ, 2)), _1654))'.

Example2 (worked)

Based on Example1, by replaceing body_pred(succ,2). with body_pred(my_succ,2). and adding

my_succ(1,2).
my_succ(2,3).
my_succ(3,4).
my_succ(4,5).
my_succ(5,6).

in the background file, the correct predicate f(A,B):-my_succ(C,A),last(B,C). was learned.

Example3 (with error)

Then based on Example2, I tried replacing the predicate last with min_list. One modify is needed in the example file.

%% The expected predicate is 
%% f(Num, Lst) :- my_succ (A, Num), min_list(Lst, A).
%% bias.pl as follows
max_vars(3).
max_body(3).
max_clauses(2).

head_pred(f,2).
body_pred(min_list,2).
body_pred(my_succ,2).
%% exs.pl as follows
pos(f(2,[4,3,1,2])).
pos(f(3,[4,3,2])).
pos(f(4,[3,4])).

neg(f(1,[4,1,2])).
neg(f(1,[4,3,5,2])).
neg(f(2,[4,3,2])).
neg(f(5,[4,3,2,7])).
neg(f(2,[3,4,5])).

then returns the following messages.

Traceback (most recent call last):
  File "popper.py", line 11, in <module>
    _prog, stats = learn_solution(settings)
  File "/home/ydl/Popper/popper/loop.py", line 181, in learn_solution
    timeout(popper, (settings, stats), timeout_duration=int(settings.timeout))
  File "/home/ydl/Popper/popper/util.py", line 47, in timeout
    result = func(*args, **kwargs)
  File "/home/ydl/Popper/popper/loop.py", line 142, in popper
    conf_matrix = tester.test(program)
  File "/home/ydl/Popper/popper/tester.py", line 90, in test
    covered.update(self.success_set([rule]))
  File "/home/ydl/Popper/popper/tester.py", line 83, in success_set
    self.seen_prog[prog_hash] = set(next(self.prolog.query('success_set(Xs)'))['Xs'])
  File "/home/ydl/.local/lib/python3.8/site-packages/pyswip/prolog.py", line 126, in __call__
    raise PrologError("".join(["Caused by: '", query, "'. ",
pyswip.prolog.PrologError: Caused by: 'success_set(Xs)'. Returned: 'error(existence_error(matching_rule, :(lists, min_list(3, [4, 3, 2]))), context(:(lists, /(min_list, 2)), _1026))'.

The environment is SWI-Prolog version 8.4.1 for x86_64-linux (on WSL). I am not sure it is a bug or due to violation of some constraints of the system.

Thanks. (Edited a typo)

example/reverse/exs.pl includes the lines of the solution

The file example/reverse/exs.pl includes the solution of the problem, i.e.

f(A,B):-empty(A),empty(B).
f(A,B):-head(A,C),tail(A,D),f(D,E),my_append(E,C,B).

which makes Prolog to complain about No permission to modify static procedure 'f/2'.

A strange behavior when adding extra ASP constraints to pruning by `clause`

I understand that the pruning may make the whole search slower in the DCC version, but here the issue is about a possible bug.
For example, in the examples/filter example, if I want to force the program contains 3 sub-clauses, adding

:-
    not clause(2).

to
https://github.com/logic-and-learning-lab/Popper/blob/8e132faa0f1a7554275f70b0bb7546cc79cce901/examples/filter/bias.pl
should prune the learned programs with less than 3 sub rules. However, a resource_error raises.

pyswip.prolog.PrologError: Caused by: 'pos_covered(Xs)'. Returned: 'error(resource_error(stack), Atom('27909')(stack_overflow, 3355306, choicepoints, 6710615, depth, 3355312, environments, 235922, globalused, 524267, localused, [Functor(565645,3,6710615,:(user, f([], [1])),[]), Functor(565645,3,6710614,:(user, f([1], [1])),[])], non_terminating, 1048576, stack_limit, 0, trailused))'.

One more strange thing is that, if adding

:-
    clause(2).

to the bias file, the learning is accelerated (with the correct result outputted).

I also test the same things in Popper 1.0, then expected pruning works.

% BEST PROG 4354:
f(A,B):-empty(A),empty(B).
f(A,B):-head(A,C),tail(A,D),odd(C),f(D,B).
f(A,B):-head(A,E),even(E),tail(A,C),f(C,D),prepend(E,D,B).
% Precision:1.00, Recall:1.00, TP:5, FN:0, TN:5, FP:0

% BEST PROG 13759:
f(A,B):-empty(B),empty(A).
f(A,B):-tail(A,D),f(D,B),head(A,C),odd(C).
f(A,B):-tail(A,E),f(E,C),head(A,D),even(D),prepend(D,C,B).
% Precision:1.00, Recall:1.00, TP:5, FN:0, TN:5, FP:0

I failed to find the modification of clause's meaning in alan.pl. Could you please help?

Timeout error on dataset

Hello I ran Popper on a Knowledge Graph dataset which is a pretty large dataset. I turned recursion on. I got the following error. Could you please suggestion some changes so that Popper can be run on large datasets

`explain` always true when invoked from commandline

Looking here:

parser.add_argument('--explain', default=True, action='store_true', help='explain')

Since default=True, args.explain will always be True, I think, even if --explain is not given.

I noticed this because of a discrepancy between running popper from the command line vs. running from another python program. The former was much faster than the latter, and it was because explain was true in the former but not the latter.

Btw, what does explain mean?

"PL_put_chars: Assertion failed: 0" probably due to an automatic upgrade of swi-prolog

Hi,

my Popper code now crashes with the error "PL_put_chars: Assertion failed: 0" . I suspect the pyswip module and its dependency to swi-prolog, which I think has been automatically upgraded on my ubuntu PC. This python script fails with the same error:
from pyswip import Prolog
prolog = Prolog()
prolog.assertz("father(joe,john)")
Did somebody face the same issue, and do you know how to solve the problem?

Thank you in advance for your help :)

Here are the versions of my environment:
Ubuntu 20.04.5 LTS
pyswip: 0.2.10
swi-prolog: 9.0.0 for x86_64-linux
popper-ilp: git+https://github.com/logic-and-learning-lab/Popper@d85f5b906402d1cc62d75f5da6e48153727732ea

fyi, I've just reported this issue to yuce/pyswip#93 as well

Usage of the extra bias for specifing the clause

Hello. I want to define some more specific properties in the bias file. I learned from the file alan.pl, but things don't work as I guess. For instance, I want to make sure that

Two clauses.
Only one of them contains the predicate q in the body.

So I write the following things in the bias.pl.

:-
    not clause(1).

:-
    head_literal(1,_,_,_), %%allowing the 0 clause to contain q.
    body_literal(_,q,_,_).

But the result seems to show that neither of them contains q predicate. Obviously, the second clause I write makes it failed, but what is the reason? And is there a way to express what I want to restrict?

Thank you.

Number of clauses are larger than max_clause/1

In my krki example (http://hakank.org/popper/krki/) the bias.pl contains max_clause(7), i.e. allow up to 7 clauses.

max_vars(8).
max_body(4).
max_clauses(7).

head_pred(illegal,6).
body_pred(adj,2).
body_pred(eq,2).

Here are the bk.pl and exs.,pl

Popper v2.0.0 finds this solution after 2min48s which has 27 clause (>> 7).

********** SOLUTION **********
Precision:1.00 Recall:0.59 TP:201 FN:140 TN:655 FP:0 Size:135
illegal(A,B,C,D,E,F):- adj(C,E),adj(G,B),adj(G,A),eq(D,F).
illegal(A,B,C,D,E,F):- adj(F,B),adj(D,A),adj(C,E),adj(E,A).
illegal(A,B,C,D,E,F):- adj(D,G),adj(G,A),adj(B,F),eq(C,E).
illegal(A,B,C,D,E,F):- adj(G,C),adj(G,D),adj(A,E),adj(B,F).
illegal(A,B,C,D,E,F):- adj(A,E),adj(C,B),adj(B,F),adj(A,D).
illegal(A,B,C,D,E,F):- eq(F,D),adj(B,E),adj(G,C),adj(A,G).
illegal(A,B,C,D,E,F):- adj(A,E),adj(G,C),adj(G,B),eq(F,D).
illegal(A,B,C,D,E,F):- adj(B,G),eq(C,E),adj(F,G),eq(A,D).
illegal(A,B,C,D,E,F):- adj(A,F),adj(C,D),eq(D,E),adj(A,B).
illegal(A,B,C,D,E,F):- adj(C,F),adj(A,B),eq(E,C),adj(D,B).
illegal(A,B,C,D,E,F):- adj(F,G),eq(C,E),adj(A,B),adj(D,G).
illegal(A,B,C,D,E,F):- adj(E,D),adj(A,B),adj(C,B),eq(F,D).
illegal(A,B,C,D,E,F):- adj(G,E),adj(C,G),eq(F,D),adj(A,B).
illegal(A,B,C,D,E,F):- eq(C,A),eq(D,B),adj(E,G),adj(F,G).
illegal(A,B,C,D,E,F):- eq(D,F),adj(E,G),adj(C,B),adj(G,A).
illegal(A,B,C,D,E,F):- adj(A,C),adj(E,G),eq(F,D),adj(B,G).
illegal(A,B,C,D,E,F):- adj(C,D),eq(C,E),adj(F,A),adj(B,E).
illegal(A,B,C,D,E,F):- adj(C,B),adj(F,A),adj(A,D),eq(C,E).
illegal(A,B,C,D,E,F):- eq(D,F),adj(F,A),adj(C,B),adj(C,E).
illegal(A,B,C,D,E,F):- adj(F,A),eq(A,C),adj(B,E),adj(D,B).
illegal(A,B,C,D,E,F):- adj(B,E),adj(C,D),eq(F,D),adj(E,A).
illegal(A,B,C,D,E,F):- adj(C,A),adj(A,E),adj(D,E),adj(F,B).
illegal(A,B,C,D,E,F):- adj(G,B),eq(C,E),adj(A,G),adj(D,F).
illegal(A,B,C,D,E,F):- adj(E,A),adj(F,B),adj(D,F),adj(C,A).
illegal(A,B,C,D,E,F):- adj(B,D),adj(C,A),eq(E,C),adj(D,F).
illegal(A,B,C,D,E,F):- adj(D,F),adj(E,A),adj(F,B),adj(E,C).
illegal(A,B,C,D,E,F):- adj(G,F),adj(G,A),eq(C,E),adj(D,B).
******************************
python3 popper.py hakank/krki  168,68s user 1,15s system 100% cpu 2:48,67 total

The program is run with default command line values:

$ python3 popper.py hakank/krki

Has the logic of max_clauses/1 in bias.pl changed in v2.0.0?

Showing covered/not covered instances

When debugging a model it would be helpful to see which instances that are covered/not covered after the (tentative) programs.

Here is a (fake) example for examples/alleven to show what I mean:

% NEW BEST PROG 1:
f(A):-empty(A).
% Precision:n/a, Recall:0.00, TP:0, FN:4, TN:5, FP:0

TP:
-

FN:
-

TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).

FP:
pos(f([34, 62, 10])).
pos(f([62, 10])).
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).





% NEW BEST PROG 7:
f(A):-last(A,B),even(B).
% Precision:0.67, Recall:1.00, TP:4, FN:0, TN:3, FP:2

TP:
pos(f([34, 62, 10])).
pos(f([62, 10])).


FN:
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).


TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).

FP:
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).


% BEST PROG 125:
f(A):-empty(A).
f(A):-head(A,B),tail(A,C),even(B),f(C).
% Precision:1.00, Recall:1.00, TP:4, FN:0, TN:5, FP:0

TP:
pos(f([34, 62, 10])).
pos(f([62, 10])).
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).


FN:
-

TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).

FP:
-

Get all positive and negative examples

First of all, thank you for your great library.
I think the function "query" in the class Tester should return a list instead of a set cause the set only contains the first positive and negative example.
However, I don't get your result in both cases.

Test example
`settings = Settings(kbpath='examples/trains1')
prog, score, stats = learn_solution(settings)
if prog != None:
print_prog_score(prog, score)
else:
print('NO SOLUTION')

Before
class Tester():
def query(self, query, key):
return set(next(self.prolog.query(query))[key])

Num. pos examples: 1
Num. neg examples: 1
SOLUTION
Precision:1.00 Recall:1.00 TP:1 FN:0 TN:1 FP:0 Size:4
f(A):- has_car(A,C), has_load(C,B), three_load(B).

After
class Tester():
def query(self, query, key):
return list(next(self.prolog.query(query))[key])

Num. pos examples: 394
Num. neg examples: 606
SOLUTION
Precision:1.00 Recall:1.00 TP:394 FN:0 TN:606 FP:0 Size:4
f(A):- has_car(A,B), three_wheels(B), roof_closed(B).

The new --info flag sometimes throws ZeroDivisionError

The new --info flag sometimes throws ZeroDivisionError: division by zero . For example running example/dropk:

$ time popper.py --info  examples/dropk 
NEW BEST PROG 1:
f(A,B,C):-tail(A,C),head(C,B)
Traceback (most recent call last):
  File "popper.py", line 197, in <module>
    timeout(popper, (settings, stats, args), timeout_duration=int(settings.timeout))
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/util.py", line 43, in timeout
    result = func(*args, **kwargs)
  File "popper.py", line 175, in popper
    print_conf_matrix(conf_matrix)
  File "popper.py", line 128, in print_conf_matrix
    precision = tp / (tp+fp)
ZeroDivisionError: division by zero

I like the new flag by the way; it's neater than running with --debug.

Add percentage time to --stats

If would be good if --stats showed the percentage of overall time spent doing each step, in addition to the absolute value.

Possible optimization (if not a bug): should the sub programs in the explainer also follow the constraint?

Hi Andrew,

Not sure if I understand correctly, but currently the subprogs generated by the explainer contain the ones which don't satisfy things like non_datalog. Is it true that they should be pruned?

pos/neg examples must be on a single line, otherwise Popper is confused

If the positive and negative examples in exs.pl are not each in a single line, Popper get confused.

Here's an example of this. I changed the exs.pl from examples/length example by splitting each of the pos's and neg's into two lines:

pos(f([3,3,1],
      3)).
pos(f([4],
      1)).
pos(f([4,3],
      2)).
pos(f([4,3,2,2,3,5,2,7],
      8)).
neg(f([3,3,1],
      2)).
neg(f([4],
      0)).
neg(f([4],
      2)).

This is a silly example, but for longer lines it's not unreasonable for a user to split the lines.

As a result Popper give the first program as a solution:

Program 1:
f(A,B) :- empty(A),one(B).
TP: 0, FN: 0, TN: 0, FP: 0

f(A,B) :- empty(A),one(B).

It's 0's for TP and TN, which is weird. The culprit is in popper/tester.py where the method load_basic/1 reads the exs.pl file:

        # Read example file
        with open(kbpath + 'exs.pl') as f:
            for line in f:
               # ...

The for loop slurps the examples line by line which explains the behavior. It's a little surprising that there's no error when asserting the incomplete examples to Prolog.

Since I'm an old Perl program, regular expressions to the rescue. :-)
In the program
http://hakank.org/popper/exs_with_splitlines.py
I do some tests on a regex approach that seems to solves these issues. Here's the regex used:
r'(^[\s%]*(?:pos|neg)\([^\.]+?\)\.\s*)'

The program tests for both splitted lines as well as comments in the lines and works for the test cases, but using regexes for this is error prone and might not be the best idea. It might be better to check with some sort of a grammar.

Here's a suggestion to fix tester.py using this regex approach, but I have not tested it extensively.

diff --git a/popper/tester.py b/popper/tester.py
index 4a9c0ad..8942b39 100644
--- a/popper/tester.py
+++ b/popper/tester.py
@@ -21,17 +21,22 @@ class Tester():
         self.prolog.consult(os.path.dirname(os.path.realpath(sys.argv[0])) + '/popper/test.pl')
 
         # Read example file
+        regexp = r'(^[\s%]*(?:pos|neg)\([^\.]+?\)\.\s*)'
+        flags = re.M
         with open(kbpath + 'exs.pl') as f:
-            for line in f:
-                if line.startswith('%'):
-                    continue
-                # Assert negative and positive examples
-                for x in re.findall("pos\((.*)\)\.", line):
-                    self.prolog.assertz(f'pos({x})')
-                    self.num_pos += 1
-                for x in re.findall("neg\((.*)\)\.", line):
-                    self.prolog.assertz(f'neg({x})')
-                    self.num_neg += 1
+            slurp = f.read().replace("\n"," ")
+            lines = [x.group() for x in re.finditer(regexp, slurp,flags=flags)]
+            # remove commented lines
+            lines = [line for line in lines if not line.startswith('%')]
+            pos = [line for line in lines if line.startswith('pos(')]
+            for x in pos:
+                self.prolog.assertz(x.replace(".",""))
+            neg = [line for line in lines if line.startswith('neg(')]
+            for x in neg:
+                self.prolog.assertz(x.replace(".",""))
+
+            self.num_pos = len(pos)
+            self.num_neg = len(neg)            
 
         self.prolog.assertz(f'num_pos({self.num_pos})')
         self.prolog.assertz(f'num_neg({self.num_neg})')

However, perhaps it might be better to simply state that each pos/neg must be written on a single line.

Forcing the number of clauses?

Is there a way to force that Popper should start with - say - three clauses instead of one? A use case for this is to skip the long road of testing all the 1 and 2 clause programs that I know don't work (since a previous run didn't found any program with just max_clauses(2)).

I assume that this can be done with ASP clauses in bias.pl, but I haven't found out how it's done.

More general, perhaps is would be an idea to have min_clauses(N) (as well as min_vars(N) and min_body(N))?

Poor redundant literal check

Suppose Popper generates this rule:

f(A):-
    has_car(A,B),
    has_car(A,C),
    long(B),
    long(C).

Then Popper's subsumption check will detect that two of the body literals are logically redundant.

Popper will then generate a generalisation constraint to eliminate any generalisations of this rule. However, Popper will then happily generate this rule:

f(A):-
    has_car(A,B),
    has_car(A,C),
    has_car(A,D),
    long(B),
    long(C),
    long(D).

And then this rule:

f(A):-
    has_car(A,B),
    has_car(A,C),
    has_car(A,D),
    has_car(A,E),
    long(B),
    long(C),
    long(D),
    long(E).

In fact, Popper will happily generate rules with more body literals without enforcing that the larger rule is sufficiently specialised as to not contain redundancy.

We need to fix this issue.

The fix is something along the lines of a constraint that identifies the redundant part and then asserts that it needs to be specialised to be useful in a rule.

Error 'error(type_error(integer,...)` in v2.0.0

I'm testing my old instances and got an error when running the following:

bias.pl (see below for a comment on the commented lines)

max_vars(5).
max_body(10).
max_clauses(2).

head_pred(f,2).
% type(f,(list,list)).
% direction(f,(in,out)).

body_pred(f,2).

body_pred(tail,2).
% type(tail,(list,list)).
% direction(tail,(in,out)).

body_pred(head,2).
% type(head,(list,element)).
% direction(head,(in,out)).

body_pred(last,2).
% type(last,(list,element)).
% direction(last,(in,out)).

body_pred(length,2).
% type(length,(list,int)).
% direction(length,(in,out)).

body_pred(sum,3).
% type(sum,(int,int,int)).
% direction(sum,(in,in,out)).

body_pred(cons,3).
% type(cons,(element,list,list)).
% direction(cons,(in,in,out)).

body_pred(empty,1).
% type(empty,(list,1)).
% direction(empty,(in,)).

body_pred(zero,1).
% type(zero,(int,)).
% direction(zero,(in,)).

bk.pl

cons(A,B,C):-
    append([A],B,C).
tail([_|T],T).
head([H|_],H).
sum(A,B,C):-
    C is A+B.
empty([]).
zero(0).

exs.pl

pos(f([65, 32, 87, 84, 25, 91, 36, 43, 64, 78, 51, 65, 90],[65, 32, 87, 84, 25, 91, 36, 43, 64, 78, 51, 65])).
pos(f([65, 32],[65])).
pos(f([1,2,3],[1,2])).
pos(f([65, 32,23],[65,32])).

neg(f([1,2,3],[1,2,3])).
neg(f([1,2,3],[1])).

Running the program:

$  time python3 popper.py --debug hakank/droplast
20:47:46 Max rules: 2
20:47:46 Max vars: 5
20:47:46 Max body: 10
20:47:46 Num. pos examples: 4
20:47:46 Num. neg examples: 2
Traceback (most recent call last):
  File "popper.py", line 8, in <module>
    prog, score, stats = learn_solution(settings)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/loop.py", line 198, in learn_solution
    timeout(settings, popper, (settings,), timeout_duration=int(settings.timeout),)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/util.py", line 65, in timeout
    result = func(*args, **kwargs)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/loop.py", line 83, in popper
    pos_covered, inconsistent = tester.test_prog(prog)
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 83, in test_prog
    pos_covered = frozenset(self.query('pos_covered(Xs)', 'Xs'))
  File "/home/hakank/inductive_programming/andrewcropper/popper/Popper/popper/tester.py", line 14, in query
    return set(next(self.prolog.query(query))[key])
  File "/usr/local/lib/python3.7/site-packages/pyswip/prolog.py", line 129, in __call__
    "Returned: '", str(term), "'."]))
pyswip.prolog.PrologError: Caused by: 'pos_covered(Xs)'. Returned: 'error(type_error(integer, [1, 2]), context(/(length, 2), _54))'.

Originally bias.pl contained the commented lines, but then Popper found this solution directly and then hanged. For some reason it didn't timeout after 10 minutes (I stopped it manually after 12 minutes).

20:20:36 New best hypothesis:
20:20:36 tp:1 fn:3 size:5
20:20:36 f(A,B):- cons(E,C,B),tail(D,C),head(A,E),tail(A,D).
20:20:36 ********************
20:20:36 Searching programs of size: 6

With the --debug flag it hangs here:

...
20:51:52 f(A,B):- cons(C,A,B),f(B,A),head(A,C).
20:51:52 Program 64:
20:51:52 f(A,B):- tail(A,B).
20:51:52 f(A,B):- last(A,C),f(B,A),cons(C,A,B).
20:51:52 Program 65:
20:51:52 f(A,B):- tail(A,B).
20:51:52 f(A,B):- f(B,A),tail(A,C),tail(C,B).

Learning more than one head.

Is it possible?

Generating all solutions

When solving problems with Constraint Programming, I'm often looking for all solutions, or at least checking if there are more than one solution. So I wondered if this is possible in Popper as well.

Here is a simple way to do that (in popper.py):

diff --git a/popper.py b/popper.py
index 325f533..a36ee1a 100755
--- a/popper.py
+++ b/popper.py
@@ -49,7 +49,7 @@ def popper(experiment):
             # 2. Test
             with experiment.duration('test'):
                 (outcome, (TP,FN,TN,FP)) = tester.test(program)
-
+                
             if experiment.debug:
                 print(f'Program {experiment.total_programs}:')
                 pprint(program)
@@ -64,7 +64,8 @@ def popper(experiment):
                     experiment.show_stats(True)
                 print('SOLUTION:')
                 pprint(program)
-                return
+                outcome = (Outcome.NONE,Outcome.NONE)
+                # return

The alternative solutions are not unique, since it's a lot of symmetries (renaming variables etc) and there might be nonsensical/whimsical solutions. For examples here are some of the first solutions for examples/robots-functional:

SOLUTION:
f(A,B):- move_up(A,B).
f(A,B):- move_up(A,C),f(C,B).
SOLUTION:
f(A,B):- move_left(A,C),move_right(C,B).
f(A,B):- move_up(A,C),f(C,B).
SOLUTION:
f(A,B):- move_down(A,C),move_up(C,B).
f(A,B):- move_up(A,C),f(C,B).
SOLUTION:
f(A,B):- move_up(A,C),move_up(C,B).
f(A,B):- move_up(A,C),f(C,B).
SOLUTION:
f(A,B):- move_right(A,C),move_left(C,B).
f(A,B):- move_up(A,C),f(C,B).
SOLUTION:
f(A,B):- move_left(A,C),move_right(C,D),move_up(D,B).
f(A,B):- move_up(A,C),f(C,B).
...

And we now know that this is the only solution (given the bias parameters) for http://hakank.org/popper/member/:

SOLUTION:
mem(A,B):- head(B,A).
mem(A,B):- tail(B,C),mem(A,C).
NO SOLUTION

I'm not sure how useful this is for the general Popper user, but I think it's a great feature, for example to see if there are any interesting alternative solutions or fine tune the decoding.

Can Popper count?

Would it be possible to let popper count?

Specifically, I am trying to do a sentiment analysis using popper on a twitter dataset. I am using a word list, which defines some words as positive or negative words as background logic. I am trying to let popper find a rule which might look like this:

f(A):- pos_count(A, 10)

Popper does find the rule, if I am including a statement like this:

ten(10)

Then it creates a rule in the following format:

f(A):- pos_count(A,B), ten(B)

It is just tedious to define all numbers. Is there a better way to do it?

Fix Popper output

Popper prints:

BEST PROG 1492:
f(A,B,C):-tail(A,C),one(B)
f(A,B,C):-decrement(B,D),f(E,D,C),tail(A,E)
Precision:1.00, Recall:1.00, TP:10, FN:0, TN:10, FP:0

It should print

% BEST PROG 1492:
f(A,B,C):-tail(A,C),one(B).
f(A,B,C):-tail(A,E),f(E,D,C),decrement(B,D).
%  Precision:1.00, Recall:1.00, TP:10, FN:0, TN:10, FP:0

needs . at the end of each rule, and extra % symbols, and correct ordering of literals.

logic-and-learning-lab / popper Goto Github PK

popper's People

Contributors

Stargazers

Watchers

Forkers

popper's Issues

Example1 (with error)

Example2 (worked)

Example3 (with error)

Recommend Projects

Recommend Topics

Recommend Org