Coercion of GET / Binary data about carmine HOT 7 CLOSED

Hi Chen, thanks for the detailed issue!

What you want here is:

(wcar* (car/set "bin-key" (car/raw (byte-array 50)))
       ;; bit ops go here
       (car/parse-raw (car/get "bin-key")))

There's two problems you're running into with your current approach:

1. (car/set "bin-key" (byte-array 50)) [i.e. w/o the car/raw] is actually going through the Carmine serializer which is prepending a 2-byte identifier to the binary data. This is used so that later calls to get can identify the type automatically, but it's not what you want here since you're going to be doing binary ops via Redis.
2. You're getting the content back as a string because the Redis-side binary ops have munged the 2-byte bin prefix, so the type is no longer identifiable and defaults to string.

The solution above instructs Carmine to send and receive in raw binary mode.

Hope that helps, cheers! :-)

from carmine.

Thanks, @ptaoussanis.

It doesn't seem to work with lua.

(wcar* (car/parse-raw (car/lua "return redis.call('get', _:key)" {:key "..."} [])))

return value is java.lang.String

Any idea?

from carmine.

It doesn't seem to work with lua

Ahh, yes - that's an interesting edge case. Going to need a little time to think about the best way of fixing this, there's a couple possibilities with different tradeoffs.

If you need a temporary workaround in the meantime:

(defn lua-workaround
  [script keys args]
  (let [[key-vars arg-vars var-vals] (#'car/script-prep-vars keys args)]
    (apply car/eval (#'car/script-subst-vars script key-vars arg-vars)
      (count keys) var-vals)))

This is exactly the same as the normal lua fn except that it calls eval rather than eval* (which handles the opportunistic eval).

Will get back to you when there's a proper fix in master.

Cheers :-)

from carmine.

Was just looking at the code and found that on first lua call, when evalsha returns NOSCRIPT then car/parse-raw is working. On any subsequent call it fails.

Why is there:

(parse nil) ; Nb

on eval* code?

  (let [[r & _] (->> (apply evalsha* script numkeys key args)
                     (with-replies :as-pipeline)
                     (parse nil) ; Nb
                     )]

Is that why it doesn't work when the function is cached in redis? Does that override any specified parser?

Thanks for the workaround

from carmine.

Is that why it doesn't work the function is cached? Does that override any specified parser?

Yes, good eye.

That's there to keep external parsers from interfering with the ability to interpret a "NOSCRIPT" error reply internally. In most cases the enclosing parsers will get their chance to apply on the (return r) call below.

The parse-raw case is a little unusual since it cannot be applied after-the-fact. Actually it's not a real "parser", it's just the identity fn with some metadata that gets picked up during reply parsing.

One soln would be to replace the (parse nil) with something that disables general parsers but keeps the necessary metadata when present. There's a couple other ways we could solve the same problem though, want to weigh options a little.

from carmine.

Great. Looking forward.

from carmine.

Eh? Seems GitHub automatically closes issues with a pushed commit now - not sure if I like that.

Anyway, 2.4.4 is up with the simplest possible change for now. Let me know if you still run into any issues.

Cheers! :-)

from carmine.