Rule::unify_source method execution takes about 30% of ForwardChainer execution time about ure HOT 6 OPEN

Not to hijack the conversation, but the flame-graph also points at opencog::SourceSet which looses huge amounts of time in boost which calls operator== which calls pthread stuff. Over the last decade, I've repeatedly discovered that boost is a performance nightmare -- rip it out of your code, everything will run 2x or 3x faster. I mean, it seems odd to me that pthread-anything should be more than 1% of the total performance; that indicates thundering herds in operator==

Also: what's __GI__exp and __ieee7? These seem to be eating lots of time...

Also: as a general rule, you want to run any/all atomspace things with HUGETLB_MORECORE=yes LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libhugetlbfs.so.0 as this will in general give you a 50% performance boost.

from ure.

If you haven't configed hugepages, see below; its cut-n-paste from atomspace-postgres; you have to adjust these instructions for your app:

Tuning HugePages

Using 2MB-sized HugePages can also offer a large performance boost,
both for Postgres, and for the atomspace, especially when working with a
large atomspace. The procedure to set these up is a bit complicated.
First, add a hugepages user-group, and add postgres to it:

   sudo groupadd hugepages
   sudo gpasswd -a postgres hugepages

Then you need to find out what the group id (gid) was:

   id postgres

Suppose that this shows group id 1234 for hugepages. This needs to
be added to /etc/sysctl.conf as well. So edit, and add:

   vm.nr_hugepages = 16384       # 32GB of hugepages, 25% of RAM.
   vm.hugetlb_shm_group=1234

Don't forget to sudo sysctl -p /etc/sysctl.conf again.
Finally, the ability to use those pages. Add to /etc/security/limits.conf:

    @hugepages      soft    memlock         unlimited
    @hugepages      hard    memlock         unlimited

from ure.

actually, you may as well set hugepages to 80% of RAM, or whatever the typical working-set size for the atomspace is.

from ure.

the flame-graph also points at opencog::SourceSet which looses huge amounts of time in boost which calls operator== which calls pthread stuff

Linas thank you for spotting this. I believe pthread stuff is a mutex lock at:

Also: what's __GI__exp and __ieee7? These seem to be eating lots of time...

I believe it is exponent calculation here:

from ure.

Hmm. Given that exp is fast (should be less than 50-100 clock cycles) the fact that its showing up a lot means it's being called a lot. Probably too much. Similar remarks for the mutex -- it should be no more than dozens of clocks, unless there's a huge amount of lock contention...

from ure.

Thanks. Great findings!

Regarding the locks, these are just crude half-way steps towards parallel URE. Once we move to C++17 I'll use more refined (unique/shared) mutexes. Maybe some thread pool, etc, not sure yet.

Regarding exp, it is used to calculate the probability distribution over sources, obviously this should be cached.

from ure.