edicl / cl-unicode Goto Github PK
View Code? Open in Web Editor NEWPortable Unicode library for Common Lisp
Home Page: https://edicl.github.io/cl-unicode/
Portable Unicode library for Common Lisp
Home Page: https://edicl.github.io/cl-unicode/
Does anyknow know what's going on? This is how I run the tests:
sbcl --non-interactive \
--eval '(require "asdf")' \
--eval "(push #p\"${wrksrc}/\" asdf:*central-registry*)" \
--eval '(asdf:load-system "cl-unicode/test")' \
--eval '(uiop:quit (if (cl-unicode-test:run-all-tests)
0 1))'
Using mlisp 10.0 (patched up to Dec 2. 2016) and loading via quicklisp (patched up to 2016-10-31), I get this error:
cl-user(1): (ql:quickload :cl-unicode)
To load "cl-unicode":
Load 1 ASDF system:
cl-unicode
; Loading "cl-unicode"
; While file-compiling #'"lists.lisp" in
#P"/home/fcbr/quicklisp/dists/quicklisp/software/cl-unicode-0.1.5/lists.lisp"
; starting at file character position 141:
Error: Package "CL-UNICODE-NAMES" not found. [file position = 219]
[condition type: reader-error]
Restart actions (select using :continue):
0: retry the compilation of /home/fcbr/quicklisp/dists/quicklisp/software/cl-unicode-0.1.5/lists.lisp
After I installed a fresh cl-unicode (from the recent quicklisp), the first Lisp I used it with was CLISP.
After this on CCL and SBCL I have the following error:
Read error between positions 188 and 212 in C:/Users/anton/projects/cl-test-grid/quicklisp/dists/quicklisp/software/cl-unicode-0.1.2/methods.lisp.
Unhandled SERIOUS-CONDITION is signaled: There is no package named "CLOS" .
The file methods.lisp contains code like
(|COMMON-LISP|::|IN-PACKAGE| :|CL-UNICODE|)
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|SCRIPT| ((|CL-UN...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|CODE-BLOCK| ((|C...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|AGE| ((|CL-UNICO...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|GENERAL-CATEGORY...
As you see DEFMETHOD is from a CLOS package.
Trying to compile cl-unicode with:
Armed Bear Common Lisp 1.9.0
Java 11.0.19 Ubuntu
OpenJDK 64-Bit Server VM
It triggers:
Error loading /home/fermin/.cache/common-lisp/abcl-1.9.0-fasl43-linux-x64/home/fermin/quicklisp/dists/quicklisp/software/cl-unicode-20210228-git/hash-tables.abcl at line 11 (offset 1202352)
#<THREAD "interpreter" {481AD9B9}>: Debugger invoked on condition of type UNBOUND-VARIABLE
The variable #:LOOP-LIST-1090112 is unbound.
Restarts:
0: TRY-RECOMPILING Recompile hash-tables and try loading it again
1: RETRY Retry loading FASL for #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-unicode" "hash-tables">.
2: ACCEPT Continue, treating loading FASL for #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-unicode" "hash-tables"> as having been successful.
3: RETRY Retry ASDF operation.
4: CLEAR-CONFIGURATION-AND-RETRY Retry ASDF operation after resetting the configuration.
5: ABORT Give up on "lem-tests"
6: REGISTER-LOCAL-PROJECTS Register local projects and try again.
Triggering accept does continue working, but I don't think that's the idea.
Thanks!
I am linking this issue here because I'm not sure if it is an asdf-jar issue or if it is a problem with that way that cl-unicode is packaged.
This code should print the entire (code point -> canonical name)
mapping, right ? :
(with-hash-table-iterator ( it CL-UNICODE::*CODE-POINTS-TO-UNICODE1-NAMES* )
(loop
(multiple-value-bind (more i) (it)
(progn
(when (not more) (return))
(format t "~A ~A~%" i (gethash i CL-UNICODE::*CODE-POINTS-TO-UNICODE1-NAMES*))
)
)
)
)
But that print-out misses some characters, such as \u2248 and \u2249 - where are they ? :
an excerpt from the print-out obtained by running the above code:
...
8682 WHITE UP ARROW FROM BAR
8788 COLON EQUAL
8789 EQUAL COLON
8804 LESS THAN OR EQUAL TO
...
Why isn't it printing \u2248 (8776) : 'โ' or \u2449 (8777) : ' โ' ?
Yet unicode-name resolves them OK :
CL-USER> (CL-UNICODE:unicode-name #\u2248)
-> "ALMOST EQUAL TO"
CL-USER> (CL-UNICODE:unicode-name #\u2249)
-> "NOT ALMOST EQUAL TO"
And they are in Unicode v1 :
CL-USER> (CL-UNICODE:age #\u2248)
-> (1 1)
So that symbol is in unicode v1, so it should be a unicode1 name, and hence in
the hash table ? What am I missing ? Why doesn't the print-out produced
by above code include #\ALMOST_EQUAL_TO ?
Just wondering what the rules for inclusion in that table were,
and if there is a more complete way of printing ALL recognized
code points and names ?
Is cl-unicode somehow checking my locale and deciding which version
of unicode names to include in the table, and omitting some because of version issues ?
It is very easy to print out a unicode table with eg. bash, not so
easy to browse it by symbol name / meaning :-)
Thanks for cl-unicode!
Best Regards,
Jason
The system definition mode line specifies a package of CL-USER
, however the correct package name is ASDF
. On many platforms, asdf
takes care of this, but on Genera the mode line is honored and therefore defsystem
is not a known symbol. Either the mode line can be patched, or the fully qualified symbol used, e.g. asdf:defsystem
.
The error below occurs in LispWorks Personal Edition (version 6.1.1) running on macOs Mojave (version 10.14.1) while loading via quickload.
(ql:quickload "cl-unicode")
Error: Error while trying to load definition for system cl-unicode from pathname .../quicklisp/dists/quicklisp/software/cl-unicode-20180328-git/cl-unicode.asd:
(SYMBOL-CALL :CL-UNICODE (QUOTE #:CREATE-SOURCE-FILES)) does not match (ASDF::O ASDF::C).
For the data processing I do time to time it is quite often really useful to have a 'normalize' function. Such a function converts unicode character to their ASCII 'equivalent' character or string. This is useful as a lot of older systems do not support unicode and to match strings I have to convert the unicode ones to the older ASCII equivalent. The function I use to do this is a simple lookup in a large lookup table, like this:
(defun normalize (string)
(check-type string string)
(format nil "~{~A~}"
(loop :for el :across string
:collect (aref +unicode-lookup-table+
(char-code el)))))
I think such a function could be really useful in a unicode library. Is this something that fits this library and how would you feel about a pull request to add this functionality?
I'm currently working on improving SBCL's Unicode support as part of the Google Summer of Code. Part of my work, which you can find here, involved expanding the SBCL Unicode database and exposing its contents through external APIs, which are in the sb-unicode
package and defined in target-unicode.lisp
.
Many of the functions that my work provides overlap with those provided by cl-unicode
. I am writing to you to discuss whether cl-unicode
could become a (partial) wrapper around the SBCL-provided Unicode database when compiled on SBCL to avoid data duplication. I'm willing to add additional data to the internal database, such as block information, to accomplish this goal if you think it's a good idea. One possible issue is that my work stores full case mappings, while yours stores simple mappings.
If you have any other suggestions about how my work could integrate with your library, please let me know.
when trying to (ql:quickload "cl-unicode")
or (asdf:load-system :cl-unicode)
, clozure cl fails with
The value NIL, derived from the initform NIL, can not be used to set the value of
the slot CL-UNICODE::CASE-FOLDING-MAPPING in #<CL-UNICODE::CHAR-INFO #x30200268A09D>,
because it is not of type NIL.
[Condition of type CCL::BAD-SLOT-TYPE-FROM-INITFORM]
the error is coming from char-info:case-folding-mapping
which has the type nil
.
the documentation reads
The case folding mapping of the character (as a list of (status, code point)) if explicitly specified
so perhaps (or list null)
is a better option than nil
? what do you think?
When I attempt to install, quicklisp complains that there is no cl-unicode/build/data/UnicodeData.TXT.NEWEST file. The file is ..../Unicode.txt. I tried renaming the file, changing the case of the extension, clearing cache, updating quicklisp and asdf with no change.
Using the cl-repl app on Android.
Thanks!
(ql:quickload :cl-unicode)
on SBCL 2.1.0 and 2.0.1 on Linux amd64:
debugger invoked on a SIMPLE-TYPE-ERROR in thread
#<THREAD "main thread" RUNNING {1004A684B3}>:
Value of 'NIL in (THE NIL (PROGN 'NIL)) is NIL, not a NIL.
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [RETRY ] Retry
completing load for #<SYSTEM "cl-unicode/build">.
1: [ACCEPT ] Continue, treating
completing load for #<SYSTEM "cl-unicode/build">
as having been successful.
2: Retry ASDF operation.
3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
configuration.
4: Retry ASDF operation.
5: Retry ASDF operation after resetting the
configuration.
6: [ABORT ] Give up on "cl-unicode"
7: Exit debugger, returning to top level.
(SB-C::%COMPILE-TIME-TYPE-ERROR (NIL) NIL #<unused argument> ((QUOTE NIL)) "(THE NIL (PROGN 'NIL))" NIL)
Backtrace:
0: (SB-C::%COMPILE-TIME-TYPE-ERROR (NIL) NIL #<unused argument> ((QUOTE NIL)) "(THE NIL (PROGN 'NIL))" NIL)
1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.| SB-PCL::|.P3.| SB-PCL::|.P4.| SB-PCL::|.P5.| SB-PCL::|.P6.| SB-PCL::|.P7.| SB-PCL::|.P8.| SB-PCL::|.P9.| SB-PCL::|.P10.| SB-PCL::|.P11.| SB-PCL::|.P12.|)) #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)
2: ((FLET CL-UNICODE::THUNK :IN CL-UNICODE::READ-CHARACTER-DATA) 0)
3: (CL-UNICODE::READ-CHARACTER-DATA)
4: (CL-UNICODE::FILL-DATABASE)
5: (CL-UNICODE::BUILD-DATA-STRUCTURES)
6: (CL-UNICODE::CREATE-SOURCE-FILES)
7: (UIOP/PACKAGE:SYMBOL-CALL :CL-UNICODE #:CREATE-SOURCE-FILES)
8: ((:METHOD ASDF/ACTION:PERFORM (ASDF/LISP-ACTION:LOAD-OP (EQL #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">))) #<unused argument> #<unused argument>) [fast-method]
9: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">)
10: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
11: ((LAMBDA NIL :IN ASDF/ACTION:PERFORM))
12: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
13: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION) {1007D86DAB}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
14: (ASDF/ACTION:CALL-WHILE-VISITING-ACTION #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build"> #<FUNCTION (LAMBDA NIL :IN ASDF/ACTION:PERFORM) {1007D86D7B}>)
15: ((:METHOD ASDF/ACTION:PERFORM :AROUND (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
16: ((SB-PCL::SDFUN-METHOD ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">)
17: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
18: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
19: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
20: ((:METHOD ASDF/PLAN:PERFORM-PLAN (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10044D9A93}>) [fast-method]
21: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
22: ((FLET "LAMBDA0" :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
23: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
24: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10044D9A93}>) [fast-method]
25: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method]
26: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :VERBOSE NIL)
27: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
28: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
29: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044D618B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
30: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :VERBOSE NIL) [fast-method]
31: ((:METHOD ASDF/OPERATE:OPERATE (SYMBOL T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
32: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL)
33: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
34: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
35: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044CB82B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
36: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
37: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
38: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044CB01B}> :OVERRIDE T :KEY NIL :OVERRIDE-CACHE T :OVERRIDE-FORCING NIL)
39: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
40: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {100400826B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
41: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
42: (ASDF/OPERATE:LOAD-SYSTEM "cl-unicode" :VERBOSE NIL)
43: (QUICKLISP-CLIENT::CALL-WITH-MACROEXPAND-PROGRESS #<FUNCTION (LAMBDA NIL :IN QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY) {1004006BBB}>)
44: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "cl-unicode" :PROMPT NIL)
45: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (T T)) #<unused argument> #<FUNCTION (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B921B}>) [fast-method]
46: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL T)) #<QL-IMPL:SBCL {1004F34173}> #<FUNCTION (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B921B}>) [fast-method]
47: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (T)) :CL-UNICODE :PROMPT NIL :SILENT NIL :VERBOSE NIL) [fast-method]
48: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<FUNCTION (LAMBDA NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B302B}>)
49: (SB-INT:SIMPLE-EVAL-IN-LEXENV (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE) #<NULL-LEXENV>)
50: (EVAL (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE))
51: (INTERACTIVE-EVAL (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE) :EVAL NIL)
52: (SB-IMPL::REPL-FUN NIL)
53: ((LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL))
54: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL) {10033B2ECB}>)
55: (SB-IMPL::TOPLEVEL-REPL NIL)
56: (SB-IMPL::TOPLEVEL-INIT)
57: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
58: ((FLET "WITHOUT-INTERRUPTS-BODY-1" :IN SB-IMPL::START-LISP))
59: (SB-IMPL::START-LISP)
https://github.com/edicl/cl-unicode/blame/master/build/read.lisp#L238
Should this read (binary-props* char-info)?
In (binary-props* code-point) code-point is a fixnum and binary-props* is an accessor
Hi,
(ql:quickload :cl-unicode)
results in the error as in the title, the implementation I use is sbcl:
$ sbcl --version
SBCL 1.4.2-1.fc27
Would you please consider adding a :description option to your system definition of cl-unicode?
i'm running SBCL, with slime.
when i call (ql:quickload :cl-unicode)
i get an error as follows
no symbol named "*STANDARD-OPTIMIZE-SETTINGS*" in "CL-PPCRE"
if i intern it, i get:
compile-file-error while
compiling #<cl-source-file "cl-unicode/base" "util">
and:
; processing (DECLAIM (INLINE COMPUTE-HANGUL-NAME))
;
; caught ERROR:
; READ error during COMPILE-FILE:
;
; The variable *STANDARD-OPTIMIZE-SETTINGS* is unbound.
;
; (in form starting at line: 219, column: 0, position: 9870)
; compilation aborted after 0:00:00.156
the variable is from cl-ppcre, which is currently loaded.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.