memory fault with incremental collection about bdwgc HOT 7 CLOSED

Hi David,

1. try recent gc7.3alpha3 snapshot
2. if you turn off incremental collection, it works, right?
3. what's your target CPU and OS?

Regards,
Ivan

Wed, 21 Aug 2013, 23:24 -07:00 from David Terei [email protected]:

I'm trying to use the GC in the memcached code base and running into the following issue. Note, this is a modified memcached setup so that the connection objects and various buffers per-connection for reading and writing to the network are managed by the GC. The storage engine for the key-value store is left alone, still uses malloc and reference counting.

• Enable incremental collection
• Connect to memcached server with one client and start generating load -- works fine.
• Connect as above with a second client -- first client will have its connection shutdown as during a read system call an EFAULT will occur, so the memcached code kills that client.
  I'm not sure why an EFAULT is occurring for the first client. The buffer that it is trying to read into is one allocated by GC_malloc and so I assume something is going wrong with the page protection for incremental collection. I'm not able to figure out more than that.
  This is with 7.3alpha2 .
  —
  Reply to this email directly or view it on GitHub .

Иван Майданский

from bdwgc.

1. Trying now.
2. Yes.
3. x64 - Xeon E5620, Arch Linux.

You can see the changes I made to memcached to integrate the GC here. I believe I'm doing everything correct but perhaps not.

from bdwgc.

Sorry but where do I find the gc7.3alpha3 snapshot? Can't see any tag in git or mention in logs, nor find on gc homepage... What is the git hash?

from bdwgc.

Hi David,

Please use BDWGC mailing list for the discussion ([email protected]). Please also keep history in the mail otherwise it's hard to follow what your "2. Yes" means.

Sorry but where do I find the gc7.3alpha3 snapshot? Can't see any tag in git or mention in logs, nor find on gc homepage... What is the git hash?

Just use latest commit from master branch (e.g., 4872305) .

Thu, 22 Aug 2013, 0:02 -07:00 from David Terei [email protected]:

• Trying now.
• Yes.
• x64 - Xeon E5620, Arch Linux.
  You can see the changes I made to memcached to integrate the GC here . I believe I'm doing everything correct but perhaps not.
  I've glanced your changes:
• I think you need to call GC_INIT.
• Have you verified that all pthread_create calls are intercepted by GC?
• Just performance concern: Why not to use GC_MALLOC_ATOMIC for buffers?

—
Reply to this email directly or view it on GitHub .
Regards,
Ivan

from bdwgc.

On 22 August 2013 02:46, Ivan Maidanski [email protected] wrote:

Hi David,

Please use BDWGC mailing list for the discussion ([email protected]). Please also keep history in the mail otherwise it's hard to follow what your "2. Yes" means.

OK.

Sorry but where do I find the gc7.3alpha3 snapshot? Can't see any tag in git or mention in logs, nor find on gc homepage... What is the git hash?

Just use latest commit from master branch (e.g., 4872305) .

OK, tried this, still same error.

Thu, 22 Aug 2013, 0:02 -07:00 from David Terei [email protected]:

• Trying now.
• Yes.
• x64 - Xeon E5620, Arch Linux.
  You can see the changes I made to memcached to integrate the GC here . I believe I'm doing everything correct but perhaps not.
  I've glanced your changes:
• I think you need to call GC_INIT.

Already calling GC_INIT, see here:
dterei/memcached@d9c1634#L0R4759

1. Have you verified that all pthread_create calls are intercepted by GC?

I just went and changed all pthread_create calls to
GC_pthread_create calls and still got the same issue. I've actually
been running memcached with just 1 thread so as to remove that
complexity and I still get the bug.

1. Just performance concern: Why not to use GC_MALLOC_ATOMIC for buffers?

Sure, I'm just trying to get something stable first, plan to try
allocation hints after that.

—
Reply to this email directly or view it on GitHub .
Regards,
Ivan

—
Reply to this email directly or view it on GitHub.

from bdwgc.

The discussion goes to BDWGC mailing list...

from bdwgc.

Copying Hans answer here in case other people come across this when searching.

It sounds to me like you're getting EFAULT during a read() system call which is writing to the
garbage-collected heap.  That's unfortunately an expected problem with incremental collection. 
Incremental collection has to protect the heap and catch write faults to track stores into the heap.  That
works find for store instructions, but doesn't work correctly for Linux system calls.

I suspect the easiest correct way out is to make sure that the read system call writes to an atomic
(pointer-free) section of the heap, and call GC_incremental_protection_needs() to check whether it is
safe to call GC_enable_incremental.  It should be on normal x86 systems.

As an alternative, it should be possible to wrap the read system call so as to avoid this problem.  Last I
checked, there was sample code to do so in os_dep.c (grep for __wrap_read).

Hans

from bdwgc.