Comments (11)
kostya
commented on June 11, 2024
from discord:
I experiencing that crystal with llvm13 compile things much faster:
crsytal from releases page:
$ rm -rf ~/.cache/crystal
$ crystal -v
Crystal 1.4.0 [ef05e26d6] (2022-04-06)
LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu
$ time crystal build bench.cr --release
real 0m15,775s
user 0m15,755s
sys 0m0,144s
my
$ rm -rf ~/.cache/crystal
$ ~/projects//crystal/bin/crystal -v
Using compiled compiler at /home/kostya/projects/crystal/.build/crystal
Crystal 1.4.0 [ef05e26d6] (2022-04-06)
LLVM: 13.0.0
Default target: x86_64-pc-linux-gnu
$ time ~/projects/crystal/bin/crystal build bench.cr --release
Using compiled compiler at /home/kostya/projects/crystal/.build/crystal
real 0m7,564s
user 0m7,541s
sys 0m0,109s
in stats Codegen (bc+obj) just 2 times faster, everything else similar.
Also compile compiler in release mode: llvm10 - 12min, llvm13 - 5.4min
from distribution-scripts.
straight-shoota
commented on June 11, 2024
LLVM 13 has just been added to Alpine edge. So there's some progress.
I'm quite surprised about this performance improvement. Not sure what's causing this. I would not expect such a huge effect from upgrading LLVM from 10 to 13.
And frankly, I can't reproduce it on my system. I see roughly the same build time regardless of LLVM version used in the compiler.
Can you share some system information?
from distribution-scripts.
kostya
commented on June 11, 2024
ryzen 3800x, Ubuntu 21.10 in VirtualBox on Windows (performance is the same as on pure ubuntu, because of amd-v, checked many times before). Did you use version downloaded from github releases page? May be there is some not release build things?
from distribution-scripts.
kostya
commented on June 11, 2024
Just tested again, downloaded this https://github.com/crystal-lang/crystal/releases/download/1.4.0/crystal-1.4.0-1-linux-x86_64.tar.gz, and put into PATH.
cd crystal-source at tag 1.4.0
$ rm -rf ~/.cache/crystal/
$ ./bin/crystal -v
Crystal 1.4.0 [ef05e26d6] (2022-04-06)
LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu
$ time make clean crystal release=1 stats=1
rm -rf .build
rm -rf ./docs
rm -rf src/llvm/ext/llvm_ext.o
Using /usr/bin/llvm-config-13 [version= 13.0.0]
g++ -c -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-13/include -std=c++14 -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="ef05e26d6" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1649265695" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign --release --stats -Dwithout_interpreter -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse: 00:00:00.000043031 ( 0.76MB)
Semantic (top level): 00:00:00.459027825 ( 122.93MB)
Semantic (new): 00:00:00.002437981 ( 122.93MB)
Semantic (type declarations): 00:00:00.043583599 ( 138.93MB)
Semantic (abstract def check): 00:00:00.016040574 ( 154.93MB)
Semantic (ivars initializers): 00:00:07.218215007 (1106.86MB)
Semantic (cvars initializers): 00:00:00.012205772 (1106.86MB)
Semantic (main): 00:00:12.133006871 (1282.86MB)
Semantic (cleanup): 00:00:00.000696819 (1282.86MB)
Semantic (recursive struct check): 00:00:00.001659750 (1282.86MB)
Codegen (crystal): 00:00:07.200266114 (1546.86MB)
Codegen (bc+obj): 00:10:04.382928159 (1546.86MB)
Codegen (linking): 00:00:00.974312303 (1546.86MB)
Macro runs:
- /home/kostya/projects/crystal/src/ecr/process.cr: 00:00:10.691528949
Codegen (bc+obj):
- no previous .o files were reused
real 10m35,210s
user 10m33,735s
sys 0m4,737s
after
$ rm -rf ~/.cache/crystal/
$ ./bin/crystal -v
Using compiled compiler at .build/crystal
Crystal 1.4.0 [ef05e26d6] (2022-04-06)
LLVM: 13.0.0
Default target: x86_64-pc-linux-gnu
$ time CRYSTAL_CONFIG_BUILD_COMMIT="ef05e26d6" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1649265695" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign --release --stats -Dwithout_interpreter -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Using compiled compiler at .build/crystal
Parse: 00:00:00.000047289 ( 0.76MB)
Semantic (top level): 00:00:00.296839306 ( 123.99MB)
Semantic (new): 00:00:00.002100901 ( 123.99MB)
Semantic (type declarations): 00:00:00.042825347 ( 139.99MB)
Semantic (abstract def check): 00:00:00.013856054 ( 155.99MB)
Semantic (ivars initializers): 00:00:05.212381241 (1107.92MB)
Semantic (cvars initializers): 00:00:00.010233332 (1107.92MB)
Semantic (main): 00:00:06.569793776 (1283.92MB)
Semantic (cleanup): 00:00:00.000698519 (1283.92MB)
Semantic (recursive struct check): 00:00:00.001459019 (1283.92MB)
Codegen (crystal): 00:00:04.946671101 (1547.92MB)
Codegen (bc+obj): 00:05:23.528932942 (1547.92MB)
Codegen (linking): 00:00:01.003124368 (1547.92MB)
Macro runs:
- /home/kostya/projects/crystal/src/ecr/process.cr: 00:00:05.252160888
Codegen (bc+obj):
- no previous .o files were reused
real 5m41,763s
user 5m41,447s
sys 0m2,798s
from distribution-scripts.
ysbaddaden
commented on June 11, 2024
Reproduced: Intel Haswell laptop, ElementaryOS 5 (based on Ubuntu 18.04, Linux 5.4, glibc 2.27), LLVM 13.0.1 installed from LLVM's APT repository. I'm compiling Crystal 1.4.1.
With Crystal 1.4.1 official DEB package:
$ bin/crystal --version
Crystal 1.4.1 [b7377c041] (2022-04-22)
LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu
$ make clean crystal release=1 stats=1
rm -rf .build
rm -rf ./docs
rm -rf src/llvm/ext/llvm_ext.o
Using /usr/bin/llvm-config-13 [version= 13.0.1]
g++ -c -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-13/include -std=c++14 -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="b7377c041" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1650653870" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign --release --stats -Dwithout_interpreter -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse: 00:00:00.000059032 ( 0.76MB)
Semantic (top level): 00:00:00.779523308 ( 123.14MB)
Semantic (new): 00:00:00.004029681 ( 123.14MB)
Semantic (type declarations): 00:00:00.085297779 ( 139.14MB)
Semantic (abstract def check): 00:00:00.026972275 ( 155.14MB)
Semantic (ivars initializers): 00:00:12.114145923 (1107.07MB)
Semantic (cvars initializers): 00:00:00.019897106 (1107.07MB)
Semantic (main): 00:00:15.740714243 (1283.07MB)
Semantic (cleanup): 00:00:00.001150210 (1283.07MB)
Semantic (recursive struct check): 00:00:00.002728421 (1283.07MB)
Codegen (crystal): 00:00:10.825581444 (1547.07MB)
Codegen (bc+obj): 00:16:34.960303616 (1547.07MB)
Codegen (linking): 00:00:02.053159918 (1547.07MB)
Macro runs:
- /home/github/crystal/src/ecr/process.cr: 00:00:13.788826770
Codegen (bc+obj):
- no previous .o files were reusedWith the freshly generated Crystal with LLVM 13:
$ CRYSTAL_CONFIG_BUILD_COMMIT="b7377c041" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1650653870" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign --release --stats -Dwithout_interpreter -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Using compiled compiler at .build/crystal
Parse: 00:00:00.000126235 ( 0.76MB)
Semantic (top level): 00:00:00.459376204 ( 123.14MB)
Semantic (new): 00:00:00.003528789 ( 123.14MB)
Semantic (type declarations): 00:00:00.084022735 ( 139.14MB)
Semantic (abstract def check): 00:00:00.030140651 ( 155.14MB)
Semantic (ivars initializers): 00:00:07.994777631 (1107.07MB)
Semantic (cvars initializers): 00:00:00.023465840 (1107.07MB)
Semantic (main): 00:00:09.633619318 (1283.07MB)
Semantic (cleanup): 00:00:00.001162493 (1283.07MB)
Semantic (recursive struct check): 00:00:00.002510932 (1283.07MB)
Codegen (crystal): 00:00:07.721239395 (1547.07MB)
Codegen (bc+obj): 00:08:20.788693715 (1547.07MB)
Codegen (linking): 00:00:01.815475772 (1547.07MB)
Macro runs:
- /home/github/crystal/src/ecr/process.cr: 00:00:07.561613304
Codegen (bc+obj):
- no previous .o files were reusedThis is a dramatic improvement. From 17+ minutes down to 9 minutes on my Haswell laptop.
Given that it impacts all stages, it's not directly related to LLVM. I heard that musl-libc was meant to be correct rather than fast, but still, that's too huge of a difference. Maybe something to do with memory allocations? I think it linked a locally installed BDWGC 8.1.0 library.
from distribution-scripts.
kostya
commented on June 11, 2024
on MacBook M1, crystal installed from homebrew:
Crystal 1.4.0 (2022-04-06)
LLVM: 13.0.1
Default target: aarch64-apple-darwin20.6.0
both compile and recompile release compiler takes 3.3min
from distribution-scripts.
straight-shoota
commented on June 11, 2024
@kostya Did you recompile on M1 with LLVM 13 (which the compiler from homebrew already uses), or with LLVM 10?
from distribution-scripts.
kostya
commented on June 11, 2024
Both used llvm13. also I not see llvm10 in homebrew, only 7,8,9,11,12,13
from distribution-scripts.
ysbaddaden
commented on June 11, 2024
Reproduced on compiling other Crystal programs. For example a testsuite I tried goes from 30s to 18s with an empty cache (6.5s to 4.2s with a full cache).
Having a boost with a Crystal compiler generated using LLVM 10 seem to imply that this has nothing to do with LLVM 13. We can probably reproduce the same by recompiling Crystal with LLVM 10.
It sounds more like a difference between Alpine/musl and Ubuntu/glibc, a library, or maybe the way the LLVM library was compiled, and what features are enabled by default? Or maybe with libgc or underlying musl vs glibc, or (why not) spectre mitigations?
from distribution-scripts.
kostya
commented on June 11, 2024
i compile this on linux, with different crystal versions:
require "json"
struct A
include JSON::Serializable
getter a : Int32
end
p A.from_json(%Q<{"a":1}>)$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-0.25.1-1/bin/crystal build 1.cr --release
real 0m3,870s
user 0m3,842s
sys 0m0,076s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-0.30.0-1/bin/crystal build 1.cr --release
real 0m3,654s
user 0m3,683s
sys 0m0,043s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-0.31.0-1/bin/crystal build 1.cr --release
real 0m7,401s
user 0m7,413s
sys 0m0,058s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-0.32.0-1/bin/crystal build 1.cr --release
real 0m7,535s
user 0m7,519s
sys 0m0,091s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-0.34.0-1/bin/crystal build 1.cr --release
real 0m7,569s
user 0m7,550s
sys 0m0,095s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-1.0.0-1/bin/crystal build 1.cr --release
real 0m10,458s
user 0m10,431s
sys 0m0,106s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-1.2.2-1/bin/crystal build 1.cr --release
real 0m10,823s
user 0m10,643s
sys 0m0,110s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-1.3.2-1/bin/crystal build 1.cr --release
real 0m12,721s
user 0m12,608s
sys 0m0,103s
$ rm -rf ~/.cache/crystal/ && time ~/Downloads/crystal-1.4.0-1/bin/crystal build 1.cr --release
real 0m13,276s
user 0m13,277s
sys 0m0,127s
$ rm -rf ~/.cache/crystal/ && time ~/projects/crystal/bin/crystal build 1.cr --release
Using compiled compiler at .build/crystal
real 0m6,367s
user 0m6,350s
sys 0m0,101s
last version is 1.4.0 recompiled. Funny that current compiler 4 times slower than 0.30.0 was
from distribution-scripts.
straight-shoota
commented on June 11, 2024
Let's continue the discussion about performance potentials in crystal-lang/crystal#12060
from distribution-scripts.
Related Issues (20)
- Release Crystal 1.9.1
- Release Crystal 1.9.2
- Debian 12 (Bookworm) not present in OBS builds HOT 2
- Release Crystal 1.10.0
- Release Crystal 1.10.1
- deb: crystal package depends on crystal1.10 itself conflicting with crystal HOT 6
- [WIP] make darwin work for MacOS M1 HOT 2
- [PROPOSAL] Allow to run crystal circleci tests for contributors HOT 5
- Require either pkg-config or pkgconf HOT 8
- Release Crystal 1.11.0
- Release Shards 0.17.4
- Upgrade Darwin package to build crystal against LLVM 17 HOT 1
- Stop building our own libgc HOT 1
- Release Crystal 1.11.1
- Debian crystal1.11 package's Conflicts: field is broken HOT 9
- Release Crystal 1.11.2
- CircleCI deprecation image
- Release Shards 0.18.0
- Release Crystal 1.12.0
- Release Crystal 1.12.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from distribution-scripts.