Comments (7)
@mooman219 : Yes, we do not need to require msys2 for windows-msvc. I updated README.md with 53408c3.
from shaderc-rs.
+1.
It feels pretty bad to require msys in the readme if you're using msvc on windows even with ninja and cmake on your path already.
from shaderc-rs.
@Lokathor: So shaderc-sys is already quite complicated regarding build options right now. I'm a little bit reluctant to make it even more so. Is there a compelling reason that downloading ninja and putting it in path is not acceptable for you? I think using ninja is generally better as for compilation speed? But I am definitely not aware of your settings. I guess I'd like to know some context to understand it better. :)
Besides, the temp directory prefix is not necessary shorter. It's C:\Users\xxx\Local\Temp
already...
from shaderc-rs.
Needing to install ninja is not hard but is one step further away from being able to do cargo build
to build a program. Given that shaderc-rs
is part of gfx-hal
, which is part of Amethyst and will eventually be part of ggez, making the build process as simple as possible will impact lots of users. ggez at least is targeted at hobbyist game devs, which means if life is difficult/complicated the ggez issue tracker will fill up with people asking for help and I'll have to deal with it.
If there's alternatives that make it simpler to be able to make a window that says "hello world", I'll happily help implement and maintain them.
from shaderc-rs.
(I say all of the following as a person who is myself the maintainer of a bindings crate, so I know somewhat the troubles you face)
So, the temp directory seems to default to %USERPROFILE%\AppData\Local\Temp
, which for me for example is C:\Users\Daniel\AppData\Local\Temp
, 34 code units. I'm going to measure this all in code units not code points because that's what the path length limit is actually measured in, but since all my paths here are ASCII those two numbers will be equal for WTF16 file paths.
So say we're developing learn-gfx-hal
. It's in a development folder that I keep at the root of my data drive, D:\dev\
, and so the base path is D:\dev\learn-gfx-hal
(20 code units). Many people don't necessarily do that, so their base path might be much longer. Perhaps C:\Users\Daniel\Documents\learn-gfx-hal
(39 units) or maybe something longer if there's a "code" folder that all the projects go in (which would cost you 5 units).
So naturally the files go into OUT_DIR
which is something like D:\dev\learn-gfx-hal\target\debug\build\shaderc-0b3a14f59f26d605\out\
(69 units, hold the jokes), but then that's where we just start the building, and in here we have several layers of project folder before we finally get to the folder that holds the output DLL: D:\dev\learn-gfx-hal\target\debug\build\shaderc-0b3a14f59f26d605\out\build\shaderc\libshaderc\Release
(101 units). Of course this isn't even the longest path'd thing I can find in that mess of files (and one of which could exceed the max path and blow up the build), a quick glance found one item with a 193 unit path when you include the filename. The .dll
/ .lib
is just the thing we care about the most, since that's the goal.
So since the limit is 240, then going from OUT_DIR
of ~70 units to TEMP
of ~35 units is a notable chunk of improvement. The big win is that we don't have to require developers to download and install any special tool.
However, here's a better idea, I don't know why it didn't come to me before: Just take all the library files that we need to link against: shaderc.lib
, shaderc_combined.lib
, shaderc_shared.dll
, shaderc_shared.exp
, and shaderc_shared.lib
, then decide what ones are actually necessary for building the crate on windows (probably just that .lib
if I had to guess) and then ship those in the repository files. And just immediately link to them unless there's some cargo flag that people flip on that says "please, i do want to kill my compile times, please build the whole project from absolute scratch".
And everyone else can just link to a pre-built lib file. and the whole crate will build in like 1 second instead of like 500 seconds.
from shaderc-rs.
I still feel it's amusing that in the year of 2019, we are still fighting some historical limitation from DOS time... But anyway, seems things will be better with MSBuild 16 (or at least a good step forward I guess).
@icefoxen: Yeah, it makes sense for me. Agreed that requiring an additional download step before building is not a pleasant thing for developers/users. People generally does not really look at README.md that much. Given shaderc-sys is likely a dependency of many other projects (which is nice ;-P), it helps to avoid the fraction.
@Lokathor: thanks a lot for the detailed analysis! I see your point that moving to use TEMP
is mitigating the problem. However, I'm worried that it's not a once-for-all solution and we can be bitten again because of this. Note that the username part can be arbitrarily long; and we control neither the OUT_DIR
path hierarchy of cargo nor the target names in SPIRV-Tools
... It is not a very scalable solution of moving directories every time something goes out of limit...
Instead, what about this:
I know that Visual Studio has already shipped CMake and Ninja: https://devblogs.microsoft.com/cppblog/cmake-support-in-visual-studio-whats-new-in-2017-15-3-update/.
So I first check the path for Ninja. If not found, I then check the Visual Studio directory for it. The fallback can be using MSBuild and hope for the best. (SPIRV-Tools recently landed a commit to reduce the length, which hopefully will make MSBuild work again. I need to refresh the SPIRV-Tools hash in this repo to adopt it. But that means Python3 is forced then...)
I'm also considering download Ninja. (Well downloading is a bit unconventional so we can renegotiate this, but we are already accessing the Internet using Git anyway.)
WDYT?
@Lokathor: Your idea of shipping pre-built binaries sounds interesting; I think we can explore that.
from shaderc-rs.
I think that just having pre-built binary files on Win32 is the best option. It makes the build time for new builds drop down to 1-2 seconds.
Unfortunately, as i recently learned when trying to add pre-built binary files to my own bindings crate, Linux and Mac aren't nearly as friendly to the idea, and those platforms might need to keep building shaderc from source. They don't have the path limit to worry about at least, though it still takes ages.
from shaderc-rs.
Related Issues (20)
- Updating shaderc version HOT 3
- Linking error when building with target-feature=+crt-static HOT 2
- Unable to build on Ubuntu 20.04 LTS - build wants "python" by name, but which one? HOT 3
- Can't find `/usr/lib/libshaderc_shared.so` HOT 1
- Build fails on Windows with Windows store Python HOT 7
- Can Compiler be Send, even Sync? HOT 1
- cannot find native shaderc library on system but Vulkan-SDK is installed
- Cannot find native shaderc library on system HOT 5
- Requesting a patch release 0.8.2 HOT 1
- [Possible Regression] Link failure on `1.67-x86_64-pc-windows-msvc` HOT 1
- shader_stage pragma for task and mesh not available
- Can't build a lib for Windows with MT_StaticRelease
- Failing to build on mac m1
- dylib not found when running without "cargo run" on macOS Sonoma
- Provide a feature to prefer static linking
- Crates that use shaderc at build time fail to build on docs.rs HOT 5
- Run in CI/GitHub Actions HOT 1
- mingw-w64-x86_64-ninja compiler errors for MSVC rust cargo build
- Extensions? HOT 1
- error: linking with `link.exe` failed: exit code: 1120 HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shaderc-rs.