casIORef failing (i.e. returning False) for no apparent reason in GHCi 7.8 (rc2) about haskell-lockfree HOT 21 CLOSED

We did entertain versions that permitted false negatives (for example on GC) but not positives.

However, the current version should not have any such thing! This definitely sounds like a bug.

This is presumably without profiling or HPC mode turned on? What machine type is this on?

I'm trying to test all the different modes more thoroughly now, and some of the 7.8.1 configurations pass tests. (But on both 7.6 and 7.8 I'm having problems with HPC. And I haven't successfully installed 7.8.1 on the mac connected to this jenkins setup):

http://tester-lin.soic.indiana.edu:8080/job/Haskell-LockFree_master/463/

from haskell-lockfree.

Ah, also, it looks like your using a release candidate? Could you try the 7.8.1 official release?

from haskell-lockfree.

Ah, ok I can confirm on linux with 7.8.1 the following file runs as expected when compiled with GHC 7.6.3, but prints False when run under 7.8.1 either through GHCI or when compiled.

import Data.IORef
import Data.Atomics

main = do
  r <- newIORef "hi"
  t <- readForCAS r
  pr <- casIORef r t "bye"
  print pr

from haskell-lockfree.

The worst part is that you can confirm the pointer returned from the CAS /is/ the same pointer (reallyUnsafePtrEquality#) and yet the CAS failed anyway.

from haskell-lockfree.

Thanks for looking into this, and let me know if I can help. Also FWIW I'm really glad false negatives aren't allowed.

from haskell-lockfree.

Note that the test I just added is completely redundant with some of the tests already there. For example, from the testing/ directory, after installing with --enable-tests, you can run tests selectively like this and see their debugging output:

DEBUG=4 dist/dist-sandbox-*/build/test-atomic-primops/test-atomic-primops  -t create_and_mutate -j1

In this case, the create_and_mutate test from Test.hs does the same thing as Issue28.hs. However, the test running inside test-framework passes fine. The difference appears to be contextual and strange. Switching on optimization and threading doesn't seem to affect it:

ghc-7.8.2 -threaded -O2 -funbox-strict-fields Issue28.hs -fforce-recomp

... will still produce an Issue28 executable that fails using GHC 7.8.2.

@jberryman, one thing I'd be curious about is whether the larger test suite (test-atomic-primops executable) passes for you on GHC 7.8.2. It does for me. @tibbe, would you mind reproducing this too?

from haskell-lockfree.

Here's a gist corresponding to the Core I'm getting for this busted test:

https://gist.github.com/rrnewton/10564888

from haskell-lockfree.

Update: I haven't exactly found it yet, but this is almost certainly to do with changes in type checking, the Any type, and unsafeCoerce.

This is the most delicate part of the "Ticket" trick. We must make sure that GHC never sees our tickets as anything other than Any type (Ticket is just a type alias for Any).

Back in issue #5, the solution was more NOINLINE annotations, to limit where GHC optimizations can peek below the "Any" lie. More NOINLINEs might be necessary here, or a different strategy for performing unsafeCoerce's inside the atomic-primops library.

from haskell-lockfree.

Re: the above referenced commit -- @tibbe, the noCMM branch I just set up is where I plan to try your suggestion of cutting out CMM entirely. Incidentally, I think this will also be an opportunity to fix this issue.

I'm sorry to say that it might not be possible to use the builtin primops in 7.8 at all. They may need to have the ticketed-style Any kinds in their types, which they don't in 7.8. (I was trying to treat that as a higher lvl issue concerning the atomic-primops lib, not GHC.)

from haskell-lockfree.

one thing I'd be curious about is whether the larger test suite (test-atomic-primops executable) passes for you on GHC 7.8.2. It does for me. @tibbe, would you mind reproducing this too?

Yeah a cabal test test-atomic-primops (on v0.6, I didn't try HEAD) passed for me. I couldn't immediately figure out how to get make full to work for me so I couldn't run all the test variations.

The difference appears to be contextual and strange. Switching on optimization and threading doesn't seem to affect it:

I noticed when I was testing in GHCi a library function foo which uses a CAS, it seemed to fail consistently (returning False) when I did foo 0, but worked correctly when I did map foo [0..100] and failed intermittently on map foo [0]. I'll see if I can get a test case.

from haskell-lockfree.

Still chasing this (on the debug branch now). Interestingly, it won't fail for IORef Int, but it fails for IORef String. Trying to pinpoint the difference in the Core.

from haskell-lockfree.

Here's another gist, containing the relevant snippets from the Int and String versions: https://gist.github.com/rrnewton/10603049 . If you download that gist, I made the whitespace in the files match so you can switch back and forth between them and "animate" the differences.

Maybe @goldfirere could advise as to whether a Role annotations problem should be something we consider among our hypothesis here.

Actually, the answer probably is later in the compiler than those .dump-simpl files in the gist. The relevant bits of them look identical, unless I'm missing something. Both version take the ticket directly from readMutVar2zh and pass it into the second argument of casMutVar2zh, like so:

           (tick1_
             `cast` ((Any (UnivCo nominal Int ()))_R :: Any Int ~# Any ()))
            (tick1_
             `cast` ((Any (UnivCo nominal [Char] ()))_R :: Any [Char] ~# Any ()))

And yet on the debug branch, you can see the printfs from the primops themselves show that the ticket argument is messed up in the string version (should stay as 0x1042240f0, but turns into a new object 0x1046122f2):

TEMPDBG: inside readMutVar2 primop, reading value 0x1042240f0
TEMPDBG: inside CAS primop, loc 0x1046122a8, current 0x1042240f0, ticket 0x1046122f2, new 0x10461234a

Footnote: All the business with casting to/from Any () is because (at least as of 7.6.3), GHC wouldn't let me use polymorphic Any types in the FFI.

from haskell-lockfree.

In the STG for the String version, it remains the case that "tick1" is passed straight through between the readMutVar and casMutVar (foreign) primops:

case newMutVar# [old eta_s7zU] of _ [Occ=Dead] {
  (#,#) ipv_s7zW [Occ=Once] ipv1_s7zX ->
      case
          __primcall atomic-primops-0.6.0.4 stg_readMutVar2zh [ipv1_s7zX
                                                               ipv_s7zW]
      of
      _ [Occ=Dead]
      { (#,#) ipv2_s7A0 [Occ=Once] ipv3_s7A1 [Occ=Once] ->
            case ipv3_s7A1 of tick1_s7A2 {
              __DEFAULT ->
                  case unpackCString# "bye"# of new1_s7A3 {
                    __DEFAULT ->
                        case
                            __primcall atomic-primops-0.6.0.4 stg_casMutVar2zh [ipv1_s7zX
                                                                                tick1_s7A2
                                                                                new1_s7A3
                                                                                ipv2_s7A0]
                        of
                        _ [Occ=Dead]
                        { (#,,#) ipv4_s7A6 [Occ=Once*] ipv5_s7A7 [Occ=Once] _ [Occ=Dead] ->
                              case ==# ipv5_s7A7 0 of _ [Occ=Dead] {
                                __DEFAULT -> (#,#) [ipv4_s7A6 ()];
                                0 -> failIO main2 ipv4_s7A6;
                              };
                        };
                  };
            };
      };

Again, there's no notable difference between the (broken) String version and (working) Int version, except that the extra unpackCString# slips in inbetween the two primop calls..

from haskell-lockfree.

It might be a problem all the way down here in the CMM:

       c7B7:
           _s7zX::P64 = R1;
           I64[(young<c7B9> + 8)] = c7B9;
           R1 = _s7zX::P64;
           call stg_readMutVar2zh(R1) returns to c7B9, args: 8, res: 8, upd: 8;
       c7B9:
           _s7A1::P64 = R1;
           I64[(young<c7Bb> + 8)] = c7Bb;
           R1 = _s7A1::P64;
           call stg_ap_0_fast(R1) returns to c7Bb, args: 8, res: 8, upd: 8;
       c7Bb:
           _s7A2::P64 = R1;
           I64[(young<c7Be> + 8)] = c7Be;
           R2 = c7Bd_str;
           call unpackCString#_info(R2) returns to c7Be, args: 8, res: 8, upd: 8;
       c7Be:
           _s7A3::P64 = R1;
           I64[(young<c7Bj> + 8)] = c7Bj;
           R3 = _s7A3::P64;
           R2 = _s7A2::P64;
           R1 = _s7zX::P64;
           call stg_casMutVar2zh(R3,
                                 R2,
                                 R1) returns to c7Bj, args: 8, res: 8, upd: 8;

Or in the opt-cmm:

      c7B7:
          I64[Sp - 8] = PicBaseReg + block_c7B9_info;
          _s7zX::P64 = R1;
          // nop
          P64[Sp] = _s7zX::P64;
          Sp = Sp - 8;
          call (I64[PicBaseReg + stg_readMutVar2zh@GOTPCREL])(R1) returns to c7B9, args: 8, res: 8, upd: 8;
      c7B9:
          I64[Sp] = PicBaseReg + block_c7Bb_info;
          // nop
          call stg_ap_0_fast(R1) returns to c7Bb, args: 8, res: 8, upd: 8;
      c7Bb:
          I64[Sp - 8] = PicBaseReg + block_c7Be_info;
          R2 = PicBaseReg + c7Bd_str;
          P64[Sp] = R1;
          Sp = Sp - 8;
          call unpackCString#_info(R2) returns to c7Be, args: 8, res: 8, upd: 8;
      c7Be:
          _s7zX::P64 = P64[Sp + 16];
          I64[Sp + 16] = PicBaseReg + block_c7Bj_info;
          R3 = R1;
          R2 = P64[Sp + 8];
          R1 = _s7zX::P64;
          Sp = Sp + 16;
          call (I64[PicBaseReg + stg_casMutVar2zh@GOTPCREL])(R3,
                                                             R2,
                                                             R1) returns to c7Bj, args: 8, res: 8, upd: 8;

from haskell-lockfree.

I'm happy to take a look, but I'm very very lost here. Can someone point me where I should be looking and what for?

from haskell-lockfree.

[I'll send context in an email.]

from haskell-lockfree.

Ok, some progress -- after wondering about where the stg_ap_0_fast was coming from exactly, I ended up removing the strictness annotations from casMutVar.

This bug is potentially fixed by 7fa6d66. (All 3/3 test suites pass on GHC 7.8.2 now.) But I don't really consider this finished until I understand why it was happening.

These strictness annotations should have been fine and were fine for GHC 7.6.3! I assume a strictness annotation on an argument that is an Any type should just do nothing. (The compiler shouldn't enter an Any type.) So, yes, it was a typo to have it there, but why should it screw anything up?

from haskell-lockfree.

By the way, even though that patch removed all bang patterns, I confirmed that it was the !tick strict ticket arguments that were causing the breakage. And of course tickets are just a type alias for Any right now:

type Ticket a = Any a

The fact that a ! on an Any argument causes any observable effect seems like a GHC bug to me. Correct?

from haskell-lockfree.

Thanks for the context -- I understand your problem better. But, I don't think I can help here, as I don't think that roles have much to do with it. It's true that the implementation of unsafeCoerce now uses role-aware UnivCo instead of the old UnsafeCo, but that's actually a minor refactoring just to avoid having two different coercions that do essentially the same thing, but with different roles.

And, as for the more recent question about !, I'm afraid I don't know there either -- haven't looked at the implementation for ! in GHC -- sorry!

from haskell-lockfree.

@jberryman, the 0.6.0.5 release is now up on Hackage and hopefully this workaround will work for you:
http://hackage.haskell.org/package/atomic-primops-0.6.0.5

from haskell-lockfree.

Thanks, I'll be playing with it! Sorry I can't be much help.
On Apr 13, 2014 10:30 PM, "Ryan Newton" [email protected] wrote:

@jberryman https://github.com/jberryman, the 0.6.0.5 release is now up
on Hackage and hopefully this workaround will work for you:
http://hackage.haskell.org/package/atomic-primops-0.6.0.5

Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-40328977
.

from haskell-lockfree.