Hi ! I am currently having some issues with some of my pipelines regarding the boolean

This is an issue with nippy / java serialization <div class="highlight highlight-s

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Boolean issue about datasplash HOT 4 CLOSED

ngrunwald commented on August 16, 2024

Boolean issue

from datasplash.

Comments (4)

RolT commented on August 16, 2024 1

This is an issue with nippy / java serialization

(-> (Boolean. false) nippy/freeze nippy/thaw)

;; => true

I tried overriding the serialization and it seems to work:

(nippy/extend-freeze Boolean :my/bool [x data-output] (.writeUTF data-output (if (boolean x) "true" "false")))
(nippy/extend-thaw :my/bool [data-input] (if (= "true" (.readUTF data-input)) true false))

Or directly in nippy's code:

-(freezer Boolean (if x (write-id out id-true) (write-id out id-false)))
+(freezer Boolean (if (boolean x) (write-id out id-true) (write-id out id-false)))

This is because:

user=> (if (Boolean. false) true false)
true

from datasplash.

edporras commented on August 16, 2024 1

@RolT thanks so much. I saw your nippy PR. I'll wait a few days to see if Peter pushes a new nippy release.

from datasplash.

xfthhxk commented on August 16, 2024

I can reproduce the same thing. If I change the values to integers or strings it works as expected.

(->> (ds/make-pipeline {})
         (ds/generate-input [{:deleted "false"} {:deleted "true"}])
         (ds/map #(log/info %) {:name "do-log"})
         (ds/run-pipeline)
         (ds/wait-pipeline-result))

;; => {:deleted "true"}  
;; => {:deleted "false"}