clj-commons / ordered Goto Github PK
View Code? Open in Web Editor NEWOrdered sets and maps, implemented in pure clojure
License: Eclipse Public License 1.0
Ordered sets and maps, implemented in pure clojure
License: Eclipse Public License 1.0
As the title says, is it possible to add that feature?
user=> (merge (ordered.map/ordered-map {:a 1}) {:a 2})
{:a 1, [:a 2] nil}
It looks like merge is acting like assoc.
Same as issue #12, but for ordered-maps. Similar test case should exhibit the bug, and similar fix of changing compact to build a new ordered-map from the old one should work.
The implementation of compact for ordered-sets has a bug. It compacts
the vector, but leaves the indices in the k->i map unchanged. This
can lead to incorrect set contents if more operations are performed
after compacting.
Possible correction: Change compact so that k->i map is updated to the
new indices. Constructing a brand new ordered-set is probably the
simplest way to do this, and should be just as fast as a more complex
method that involved traversing the k->i and i->k structures and
creating new ones.
Compactable
(compact [this](into-ordered-set this))
Steps to reproducing the bug below. Everything is OK for sets s1 and
s2. In s3's internals you can see the state after compacting s2. The
problem is exhibited on creating s4 by attempting to remove an element
from s3 that has a stale index in k->i.
user=> (require '[flatland.ordered.set :as s])
nil
user=> (def s1 (s/ordered-set :a :b :c))
user=> s1
user=> (.k->i s1)
{:a 0, :b 1, :c 2}
user=> (.i->k s1)
[:a :b :c]
user=> (def s2 (disj s1 :b))
user=> s2
user=> (.k->i s2)
{:a 0, :c 2}
user=> (.i->k s2)
[:a :flatland.ordered.set/empty :c]
user=> (def s3 (. s2 compact))
user=> (.k->i s3)
{:a 0, :c 2}
user=> (.i->k s3)
[:a :c]
user=> (def s4 (disj s3 :c))
user=> s4
user=> (.k->i s4)
{:a 0}
user=> (.i->k s4)
[:a :c :flatland.ordered.set/empty]
Make it so that ordered
is deployed to clojars whenever a Release-
tag is pushed
Should be (use 'flatland.ordered.map) rather than (use 'ordered.map).
OrderedMap doesn't implement ITransientAssociative2
(which the clojure.core
transient maps do), so using contains?
or find
on a transient OrderedMap throws a contains? not supported on type: flatland.ordered.map.TransientOrderedMap
error.
Would expect it to act like a hash-map.
(apply = (map #(merge (%) ()) [ordered-map hash-map])) ; -> false
Not sure if this is the desired behavior or not. See code snippet below.
(def m1 (ordered-map :a 1 :b 2))
(def m2 (ordered-map :b 2 :a 1))
(= m1 m2)
;; true
The easy work around is to wrap both maps in something like vec
or seq
.
(def m1 (ordered-map :a 1 :b 2))
(def m2 (ordered-map :b 2 :a 1))
(= (vec m1) (vec m2))
;; false
This produces itself when using AOT compilation :
Caused by: java.lang.ClassCastException: ordered.map.OrderedMap cannot be cast to ordered.map.OrderedMap
at ordered.map$transient_ordered_map.invoke(map.clj:183)
at ordered.map.OrderedMap.asTransient(map.clj:121)
at clojure.core$transient.invoke(core.clj:2929)
at clojure.core$into.invoke(core.clj:6004)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3337)
It's a known problem that runtime resolve isn't good for native image size. When upgrading from 1.5.9 to 1.15(!).10 I noticed that babashka's image size grew from 75mb to 120mb. This is due to a resolve
call in a reader function. I will follow up with a PR.
user> #ordered/map []
Syntax error (IllegalArgumentException) compiling fn* at (*cider-repl 127.0.0.1*:0:0).
Unable to resolve classname: IPersistentMap
user> (import '(clojure.lang IPersistentMap))
clojure.lang.IPersistentMap
user> #ordered/map []
Syntax error (IllegalArgumentException) compiling fn* at (*cider-repl 127.0.0.1*:0:0).
Unable to resolve classname: IPersistentVector
user> (import '(clojure.lang IPersistentVector))
clojure.lang.IPersistentVector
user> #ordered/map []
#ordered/map nil
Hi, first let me thank you very much for providing order on clojure maps. Very helpful!
Now I would like to implement a customized read to support ordered maps.
Until now I am using the following to read list of clojure maps from a text file which doesn't keep order in maps:
(defn read-one
[r]
(try
(read r)
(catch java.lang.RuntimeException e
(if (= "EOF while reading" (.getMessage e))
::EOF
(throw e)))))
(defn read-seq-from-file
"Reads a sequence of top-level objects in file at path."
[path]
(with-open [r (java.io.PushbackReader. (io/reader path))]
(binding [*read-eval* false]
(doall (take-while #(not= ::EOF %) (repeatedly #(read-one r)))))))
An example for a text file from which I want to read ordered maps from:
{:c 1 :a "value 3" :b 2}
{:c 2 :a "value 1" :b 3}
{:c 3 :a "value 2" :b 1}
How could I go about that?
https://clojuredocs.org/clojure.core/read
https://clojuredocs.org/clojure.edn/read
As far as I understand can clojure.core/read be customized.
I've noticed that the library has a changelog, but it hasn't really been updated, which makes it pretty useless. It'd be nice if the recent changes were reflected in it.
I honestly do not know if this is a bug in timbre or in flatland, so I'm filing an issue with each.
Timbre issue: taoensso/timbre#187
Here's a failing testcase for map_test.clj
:
(deftest read-direct
(let [s (ordered-map 1 2, 3 4, 5 6, 1 9, 7 8)
t #ordered/map ([1 9] [3 4] [5 6] [7 8])]
(is (= s t))))
This blows up with the error message in the issue title.
If I change code in map.clj
to fully-qualify all instances of IPersistentMap
, then the error message changes to "java.lang.IllegalArgumentException: No matching field found: backing-map for class flatland.ordered.map.OrderedMap".
My clojure-fu is not skillful enough to unwind your delegating-deftype
macro, so I thought I'd ask here and see if you know what's going on.
Thanks!
Consider the following REPL sequence:
user=> (require '[flatland.ordered.map])
nil
user=> #ordered/map[[:a 1] [:b 2]]
Syntax error (IllegalArgumentException) compiling fn* at (/private/var/folders/2m/h3cvrr1x4296p315vbk7m32c0000gp/T/form-init53327008725414542.clj:1:8317).
Unable to resolve classname: IPersistentMap
user=> (import 'clojure.lang.IPersistentMap)
clojure.lang.IPersistentMap
user=> #ordered/map[[:a 1] [:b 2]]
Syntax error (IllegalArgumentException) compiling fn* at (/private/var/folders/2m/h3cvrr1x4296p315vbk7m32c0000gp/T/form-init53327008725414542.clj:1:8317).
Unable to resolve classname: IPersistentVector
user=> (import 'clojure.lang.IPersistentVector)
clojure.lang.IPersistentVector
user=> #ordered/map[[:a 1] [:b 2]]
#ordered/map ([:a 1] [:b 2])
It seems the data reader only works when certain imports are in place?
which in turn pulls in a bunch of other deps.
@amalloy I know I've bugged you on various changes to this code in the past -- hope I haven't been too much of a thorn in your side there.
Someone recently created a Github group clj-commons, with the intent of being a shared umbrella for repositories that seem generally useful and there may be a group of people willing to make fixes and light maintenance to the repositories there. https://github.com/clj-commons
Would you have any interest if amalloy/useful and/or amalloy/ordered were forked there and became the "official" place from which new Clojars releases were made, by others? If so, you would still get full credit for all of your work in commit history, and the license would remain the same. About the only think I would ask from you if you did this would be adding a link to the top level README of the projects pointing at their new homes.
Since, as noted in the ordered-set
docstring, clojure.set/union
and other operations in clojure.set
may reorder elements for efficiency, perhaps it would be useful for ordered
to include (less efficient?) set operations that preserve order. For example, I needed to preserve order with union, so I am using this definition (which might also be called "union"):
(defn multi-conj
"Successively conj each element of ys onto xs."
[xs ys]
(reduce (fn [newxs y] (conj newxs y))
xs ys))
(I need it for two-element sets on the right, so efficiency shouldn't be an issue in my case.)
It looks like the conj function for sets is not usable?
(ns typedclj.rhizome
(:require [flatland.ordered.set :as fos]))
(let [s (fos/ordered-set 1 2 3)]
(fos/conj s 4 5))
CompilerException java.lang.RuntimeException: No such var: fos/conj, compiling:(/Users/kaiyin/personal_config_bin_files/workspace/typedclj/src/typedclj/rhizome.clj:5:3)
Thanks!
When rewriting from using delegating-deftype
to just deftype
the compilation starts giving reflection-warnings on cli-yaml, see
https://circleci.com/gh/clj-commons/clj-yaml/6?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link:
$ #!/bin/bash -eo pipefail
! lein check 2>&1 | grep 'Reflection warning'
Reflection warning, flatland/ordered/map.clj:113:5 - reference to field size on clojure.lang.IPersistentMap can't be resolved.
Reflection warning, flatland/ordered/map.clj:117:5 - reference to field isEmpty on clojure.lang.IPersistentMap can't be resolved.
Reflection warning, flatland/ordered/map.clj:119:5 - reference to field keySet on clojure.lang.IPersistentMap can't be resolved.
Exited with code exit status 1
The regression can be easily verified by running:
0:59 $ lein check
Compiling namespace flatland.ordered.common
Compiling namespace flatland.ordered.map
Reflection warning, flatland/ordered/map.clj:113:5 - reference to field size on clojure.lang.IPersistentMap can't be resolved.
Reflection warning, flatland/ordered/map.clj:117:5 - reference to field isEmpty on clojure.lang.IPersistentMap can't be resolved.
Reflection warning, flatland/ordered/map.clj:119:5 - reference to field keySet on clojure.lang.IPersistentMap can't be resolved.
Compiling namespace flatland.ordered.set
Hi, thanks for that free project :-)
I'm actually using it because it's a dependence of https://github.com/owainlewis/yaml but I don't understand when to purposefully use sorted sets. Would you mind to tell me when I would benefit to use such a datastructure?
Thanks in advance.
Ordered sets are equal to normal sets which is expected:
(= (flatland.ordered.set/ordered-set 1 2 3) #{1 2 3})
;=> true
However, sets containing ordered sets aren't equal to sets containing sets:
(= #{(flatland.ordered.set/ordered-set 1 2 3)} #{#{1 2 3}})
;=> false
I'm using [org.flatland/ordered "1.5.2"]
.
In order to keep things scoped, I'll close #43 and keep the rest of the work here
I'm seeing this at a fresh repl, using clojure 1.10.0 and ordered 1.5.9. Any idea what's causing it?
(ordered-map :a 1 :b 2)
#ordered/map ([:a 1] [:b 2])
(.toString (ordered-map :a 1 :b 2))
"{:a 1, :b 2}"
user> (select-keys (ordered-map :c 3 :a 1 :b 2) [:c :b])
{:b 2, :c 3}
Regular hash-map is hardcoded in select-keys, so unless there's a nice protocol to implement I guess this one's hard to fix.
Repo:
(.hashCode (os/ordered-set nil :a :b :c))
Stack trace:
#error {
:cause nil
:via
[{:type java.lang.NullPointerException
:message nil
:at [flatland.ordered.set.OrderedSet$fn__4371 invoke "set.clj" 59]}]
:trace
[[flatland.ordered.set.OrderedSet$fn__4371 invoke "set.clj" 59]
[clojure.core$map$fn__5935 invoke "core.clj" 2770]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 24]
[clojure.core.protocols$fn__8236 invokeStatic "protocols.clj" 75]
[clojure.core.protocols$fn__8236 invoke "protocols.clj" 75]
[clojure.core.protocols$fn__8178$G__8173__8191 invoke "protocols.clj" 13]
[clojure.core$reduce invokeStatic "core.clj" 6882]
[clojure.core$reduce invoke "core.clj" 6868]
[flatland.ordered.set.OrderedSet hashCode "set.clj" 59]
...
Comes from unconditionally calling .hashCode
on all of the contained items:
ordered/src/flatland/ordered/set.clj
Line 59 in 8cc133d
I believe this could be solved by either using clojure.core/hash
or by changing to a keep
(which is fine because the hash of nil
is 0):
(reduce + (keep #(when % (.hashCode ^Object %)) (.seq this))))
At least ordered-set and ordered-map, but there may be others useful to have doc strings on, too.
Please, add clojure script support.
Hi - This works with Clojure 10 and Java 10, but fails for Java 11. Stack trace:
Exception in thread "main" Syntax error compiling deftype* at (flatland/ordered/set.clj:12:1).
at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: Must hint overloaded method: toArray
at clojure.lang.Compiler$NewInstanceMethod.parse(Compiler.java:8496)
Happening in Version 1.5.6
Exception in thread "main" Syntax error compiling deftype* at (flatland/ordered/set.clj:19:1).
Must hint overloaded method: toArray
Sounds like a problem with the transient variant:
(into {:a 1 :b 2} [[:a 3]]) ; -> {:a 3, :b 2}
(into (ordered/ordered-map {:a 1 :b 2}) [[:a 3]]) -> {:a 1, :b 2}
Hi there,
It appears that there is a problem with ordered 1.5.5.
test1
is the ordered test data, and test2
is standard Clojure.
nREPL server started on port 56705 on host 127.0.0.1 - nrepl://127.0.0.1:56705
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.8.0
Java HotSpot(TM) 64-Bit Server VM 1.8.0_112-b16
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
frs.core=> (use 'flatland.ordered.map)
nil
frs.core=> (def test1 (ordered-map :a (ordered-map :deck '("spark")) :b (ordered-map :deck '("flare" "bolt"))))
#'frs.core/test1
frs.core=> (def test2 {:a {:deck '("spark")} :b {:deck '("flare" "bolt")}})
#'frs.core/test2
frs.core=> (update-in test1 [:a :deck] vec)
{:a {:deck ("spark")} :b {:deck ("flare" "bolt")}}
frs.core=> (update-in test2 [:a :deck] vec)
{:a {:deck ["spark"]} :b {:deck ("flare" "bolt")}}
frs.core=> (prn (update-in test1 [:a :deck] vec))
#ordered/map ([:a #ordered/map ([:deck ("spark")])] [:b #ordered/map ([:deck ("flare" "bolt")])])
nil
frs.core=> (update test1 :a update ,,, :deck vec)
{:a {:deck ("spark")} :b {:deck ("flare" "bolt")}}
frs.core=> (update test2 :a update ,,, :deck vec)
{:a {:deck ["spark"]} :b {:deck ("flare" "bolt")}}
I would expect that the update-in
or update
of test1
would be the same as test2
.
Please advise?
Thanks,
Doug
Hi there,
I just wanted to make you aware of a bug report I put in with specter of a problem I'm having with ordered maps and transforming them. It seems also to occur with instar. That might indicate a problem with your ordered library rather than those other two. I just don't know for sure, so I thought I'd inform you of such.
Thank you for the ordered library!
See: redplanetlabs/specter#225
Thanks,
Doug
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.