Installing emsdk overwrites node on PATH about emsdk HOT 48 CLOSED

@juj:

The command prompt emsdk_env intends to enter a prompt environment that contains all the tools that are activated at the time, so it is intentional that this prompt has the Node and Python etc. tools explicitly and up front in PATH (if user installed and activated those tools via emsdk). The intent is to provide developers with a method of entering the exact environment where the same tools are available to the prompt that Emscripten uses.

If the issue is that emsdk_env adds those tools to PATH that clobbers the use of some other commands, then maybe the root realization is that those other commands would not be appropriate to be run under the emsdk_env provided environment? Nothing should force developers to run their own tool command lines under emsdk_env if they so choose.

I think the main issue here is that the documentation suggests that you should always be in emsdk_env. Specifically, running emsdk activate latest will prompt you to do

Next steps:
- To conveniently access emsdk tools from the command line,
  consider adding the following directories to your PATH:
    /emsdk
    /emsdk/node/14.18.2_64bit/bin
    /emsdk/upstream/emscripten
- This can be done for the current shell by running:
    source "/emsdk/emsdk_env.sh"
- Configure emsdk in your shell startup scripts by running:
    echo 'source "/emsdk/emsdk_env.sh"' >> $HOME/.zprofile`

and it is not at all clear that doing the "Configure emsdk in your shell startup scripts by running [...]" step is also going to clobber the system version of node for any future shell you open. That was very surprising to me.

It sounds like emsdk_env is kind of a specialist thing. Most of the time, I just want to invoke emcc in my shell, not mess around with all the other tools exactly as emcc sees them. So if, as mentioned above, it's possible for emcc to work correctly outside of an emsdk_env, I think that should be the thing suggested by the documentation, and the tools should users in reaching that state, with emsdk_env being reserved for more specialized applications instead of being the default.

If you want to clobber the system version of node when specifically entering an emsdk_env, that's fine, as long as the documentation does not suggest always entering an emsdk_env and it is possible to run emcc without being in emsdk_env.

To put it another way, here's the process I went through:

• google "install emscripten"
• docs suggest emsdk, I'll clone that
• ./emsdk install latest
• ./emsdk activate latest
• Output says "Next steps: [...] Configure emsdk in your shell startup scripts by running [...]", ok, I'll do that
• open a new shell
• emcc works, nice!
• node has been downgraded, WTF?

I would like to go through exactly the same process, including the "emcc works in a new shell part", except without the last step.

from emsdk.

Originally Emsdk was designed to allow people to control what tools they want to have in their PATH. If a user wanted to use all of Emscripten's provided tools (the most common scenario), they would do e.g.

emsdk activate sdk-upstream-main-64bit

(or some other sdk target, depending on the version they choose)

The SDKs are defined as convenience shorthands to collections of SDK Tools to use at the same time. For example, sdk-upstream-main-64bit is a collection of the following Tools:

  {
    "version": "upstream-main",
    "bitness": 64,
    "uses": ["python-3.9.2-nuget-64bit", "llvm-git-main-64bit", "node-14.18.2-64bit", "emscripten-main-64bit", "binaryen-main-64bit"],
    "os": "win"
  },

This means that typing

emsdk install/activate sdk-upstream-main-64bit

is identical to typing

emsdk install/activate python-3.9.2-nuget-64bit llvm-git-main-64bit node-14.18.2-64bit emscripten-main-64bit binaryen-main-64bit

If I wanted to omit one of the Tools, I can issue an Emsdk install/activate step which skips that tool altogether. E.g.

emsdk install/activate python-3.9.2-nuget-64bit llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

does an installation that does not bring Emsdk node into the mix. So if you wanted to drive Emscripten with your own node and not an emsdk provided node, you could do an installation/activation like that.

One should also recognize the behavior of the Emsdk --global option? (this is a Windows specific option) That will definitely globally install node and python to Windows registry PATH that would replace user's own. In such a case, the solution is the same as above. If user has been calling

emsdk activate --global sdk-upstream-main-64bit

it is the same as typing

emsdk activate --global  python-3.9.2-nuget-64bit llvm-git-main-64bit node-14.18.2-64bit emscripten-main-64bit binaryen-main-64bit

i.e. the user is explicitly requesting a global registry PATH installation of python and node. In this case, if it is undesirable, the recommended solution would be not to run that command, but instead do a activate --global with only the tools that one wants to add to Windows PATH registry, e.g.

emsdk activate --global  llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

The command prompt emsdk_env intends to enter a prompt environment that contains all the tools that are activated at the time, so it is intentional that this prompt has the Node and Python etc. tools explicitly and up front in PATH (if user installed and activated those tools via emsdk). The intent is to provide developers with a method of entering the exact environment where the same tools are available to the prompt that Emscripten uses.

If the issue is that emsdk_env adds those tools to PATH that clobbers the use of some other commands, then maybe the root realization is that those other commands would not be appropriate to be run under the emsdk_env provided environment? Nothing should force developers to run their own tool command lines under emsdk_env if they so choose.

Emscripten command invocations do not need or care about any of the emsdk provided node and python tools being in the PATH. They only care about the EM_CONFIG file. This ensures that it is possible to invoke Emscripten tools outside emsdk_env environment (using emsdk_env is optional), as long as you provide the appropriate config file, either by using the EM_CONFIG environment variable, or by specifying the configuration using --em-config command line flag. You can see how emsdk_env sets that env. var by checking the print output when entering emsdk_env. So one solution might be to manually set this environment variable.

For example, to run emcc outside the emsdk prompt while pointing it to the configuration that emsdk has provided, one can run

EM_CONFIG=/Users/clb/emsdk/.emscripten emcc tests/hello_world.c -o hello_world.js

or

emcc --em-config /Users/clb/emsdk/.emscripten tests/hello_world.c -o hello_world.js

This kind of use can ensure that people don't get any adjustments to their PATH if not desired.

Overall, if Emsdk did not add the needed tools to PATH, my impression is that there would be more users who would be confused of not getting the tools readily available. E.g. the getting started instructions

./emcc tests/hello_world.c
node a.out.js

would not work out of the box if emsdk did not provide all the tools for the users as necessary.

Also I think the idea of pushing the tools to the PATH last is not great, as it would weaken the reproducibility of emsdk_env itself, and could cause subtleties in reproducing the same builds across systems. Given the above methods of avoiding this PATH pollution, e.g. via manually choosing the Tools to activate, or explicitly specifying --em-config / EM_CONFIG options, I think the current behavior is the most sensible one to have that can cater to all use cases.

from emsdk.

This was fixed in #1189

from emsdk.

Note that you could also do:

# Prefer the default system-installed version of Node.js
echo "NODE_JS = '$(which node)'" >> .emscripten

Just before:

# Activate PATH and other environment variables in the current terminal
source ./emsdk_env.sh

To ensure that the Node.js from emsdk doesn't clobber the PATH environment variable. FWIW, in previous versions of emsdk it was also possible to do this:

# Prefer the default system-installed version of Node.js
./emsdk uninstall $(./emsdk list | grep INSTALLED | tr -d \(\*\) | grep node | awk '{print $1}')

(from: #96 (comment))

Just before:

./emsdk activate latest

But that doesn't seems to work anymore (error: tool is not installed and therefore cannot be activated: 'node-14.15.5-64bit').

from emsdk.

@david-fong that solution should work fine, yes. All you should really need is emcc in your PATH make emsdk work. You don't even need that if you call it directly.

emcc will find emsdk/.emscripten relative to itself and node based on the configured value in emsdk/.emscripten.. so node doesn't need to be in your PATH at all to use emsdk.

from emsdk.

Dont overwrite shell's node. use an environment variable or something else.

from emsdk.

The above seems more complicated than it should be. While a few people may want to expose emsdk's node, most won't. IMO exposing node should be opt-in not opt-out.

from emsdk.

It would be interesting to know how many developers using emscripten do not install their own node on their system, it's not a small unknown dependency of emscripten, it's a common package for every dev I know.
It would seem much more natural to me that emscripten requires having your own node on your system, but at least if it has to be included it should not replace the "native" node, that's very counter-intuitive.

from emsdk.

We no longer inject clang or llvm into the PATH, but we do still inject our node in to the PATH.

I tried to make a change where node would only be added to the PATH if it wasn't already there:
#714

But we decided against landed it.

Do you really need to have emsdk and your own version of node active at the same time in the same shell? Can you explain a little about your use case? If its really probelem maybe we can re-open #714

In #714 it was mentioned that as a workaround you should be able to install and activate just the sub-components of the SDK without also installing node. e.g.:

./emsdk activate releases-upstream-c69458f1bbf3ef5b8da4e934de210659cc9bca04-64bit

from emsdk.

I can see @juj pointer that changing the defaults might cause more confusion that it solves.

I do want to make a change to allow the behaviour that @arsnyder16 want though. I kind of want the same thing since I have version of node that I like to use in my PATH already.

I kind of like the idea of something like emsdk_env --exclude=node or maybe emsdk_env --minimal. The minimal mode would just be emcc on the PATH and nothing more. No need for EM_CONFIG since emcc will find the config relative to itself. This minimal mode would do minimal necessary to put a working emcc in your PATH.

(BTW in your example above emcc --em-config /Users/clb/emsdk/.emscripten tests/hello_world.c -o hello_world.js, emcc is still required to be in the PATH).

from emsdk.

from emsdk.

Currently we do recommend using emsdk_env whenever you want to run emcc. As it happens you can get away with running emcc without emsdk_env if you manually put emcc in your PATH.. but the recommented way to put emcc in your path is to run emsdk_env. I doubt that recommendation will change.

I think "1) I want emcc to work in a fresh shell and also 2) for node to be the system-installed version in a fresh shell" is almost certainly what almost every consumer of this project wants to accomplish.

Is that goal fundamentally not compatible with the goals of this project? If so, can you say more about why?

If that goal is compatible with the goals of the project, and it's just a matter of setting up the tooling so that this works and the documentation so that this is the default, what needs to be done? I don't have a problem with emsdk_env being the recommended way of putting emcc on the PATH (goal 1) as long as that does not also shadow the system-installed node (goal 2). But @juj says that emsdk_env is specifically intended to shadow the system-installed version of node, so using it seems like the wrong approach.

from emsdk.

I wrote my initial comment after reading #714 (comment). In that comment, it sounds like @juj is thinking of emsdk_env as being something the user does explicitly when they want to enter the emsdk environment, which means they shouldn't be surprised by having the emsdk-provided versions of tools like node, rather than being something which is done by default in every shell, which is how the documentation currently suggests using it.

So I think it would be best to separate those two notions: have one tool (or option) which is designed to drop you into an environment where all the tools are the emsdk-installed versions, plus another tool (or option) which is designed to be run in every shell and which only enables emcc (and other tools which are part of emscripten specifically).

The default behavior should be to not set node in the path at all, in exactly the same way emsdk does not set python or clang.

from emsdk.

I don't understand why Emscripten's internal Node is being exposed on PATH. Can the various Emscripten JS tools not find or be told where Emscripten's node is? IMO emsdk_env.sh should only be exposing the user-facing executables (emcc, emmake, etc), not internal executables.

But I only run Emscripten within Docker now, which does sidestep this whole issue.

from emsdk.

Yes it's opt-in.

But I don't think the script should be looking to see if node already exists in path. The docs shouldn't encourage exposing emsdk's node, but instead should recommend people install node for their OS as normal.

I don't see the wisdom in a * option either. What if a future version adds some new component? Exporting unspecified internal dependencies to the user's path is unsafe and should not be encouraged. It shouldn't be in the docs even as a first step. Even continuing to have a way to expose node is a concession I wish didn't need to happen.

But then I'm not part of the project team, so you decide what you want to do :)

from emsdk.

The simplest solution might just be to avoid using emsdk_env.sh completely. Instead you can simply add emsdk/upstream/emscripten to your PATH.. that alone should be enough to use emcc (in that past emsdk_env.sh did a lot more but these days its not really doing more than just that).

from emsdk.

(I don't think there is any way to do that, but I will see if I can edit the description of the add a note).

from emsdk.

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

from emsdk.

Any updates?

from emsdk.

Reopening, as this does sound like it would be nice to fix. I think just not adding those things to the path would be fine, as emscripten does have direct paths to them in the config file for its own use. Perhaps someone has time to test that and see if it works? If so let me know if you have any questions about how to modify the emsdk for this!

from emsdk.

Any updates again? It's been 2 years and I can't use emscripten without this being fixed as I need node.js 16 and emscripten overwrites my nvm version with 14.

Update: I found https://github.com/Sxxov/emscripten-sdk-npm and it seems to be working, node -v still reports v16.10.0.
Second update: Nevermind, once I ran a source command it started using the wrong node.js version again...

from emsdk.

Does the emsdk_env.sh script do anything other than modify the PATH? And will the emscripten tools be okay if its node executable isn't found on the path?

If so, I'm just going to settle for manually adding its stuff to the end of my path and excluding its node executable. I found it surprising (in an annoying way) that my snap installation of node wasn't the one found on my PATH.

from emsdk.

@sbc100 I am running into a similar problem. With the release of firefox 100 wasm exceptions seems to now be supported by all major browsers by default. So i was switching my build process to use wasm exceptions, what i found was when using node 14 i was hitting the following error Aborted(CompileError: WebAssembly.instantiate(): Compiling function emscripten-core/emscripten#37 failed: Invalid opcode @+5216). Switching to node 16 works fine.

It seems that emsdk_env.sh is still overwriting node even if it exists in the path contrary to what #714

~# which node
/usr/bin/node
~# nvm use 16
Now using node v16.13.0 (npm v8.1.0)
~# which node
/root/.nvm/versions/node/v16.13.0/bin/node
~# source ~/emsdk/emsdk_env.sh
Adding directories to PATH:
PATH += /root/emsdk
PATH += /root/emsdk/upstream/emscripten
PATH += /root/emsdk/node/14.18.2_64bit/bin

Setting environment variables:
PATH = /root/emsdk:/root/emsdk/upstream/emscripten:/root/emsdk/node/14.18.2_64bit/bin:/root/.nvm/versions/node/v16.13.0/bin:/root/.wasmtime/bin:/root/.wasmer/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
EMSDK = /root/emsdk
EM_CONFIG = /root/emsdk/.emscripten
EMSDK_NODE = /root/emsdk/node/14.18.2_64bit/bin/node
~# which node
/root/emsdk/node/14.18.2_64bit/bin/node

from emsdk.

We go push back on #714 so it didn't land. I'll see if we can revist.

from emsdk.

@juj Fair enough i will work around it.

I would rather not have to try and keep up to date with all the dependencies of emscripten because emsdk install/activate sdk-upstream-main-64bit could change

Trying to keep this aligned seems painful as well
If something new is added, that will need tracked.
Some of these include specific version numbers, so any upgrade on the emscripten side would cause churn downstream

emsdk install/activate python-3.9.1-nuget-64bit llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

Could there be an omit/exclude flag added instead?

emsdk activate "3.1.10" --exclude node

Let me ask a another question, and more specific to why i need to exclude node. What is the cadence that emscripten will update nodejs? For example emscripten is producing wasm that the node version it includes does not support. That's really the only reason i need to use a different node. I would happily use the bundled nodejs, unfortunately i can't.

For dependencies in the toolchain like clang emscripten point to trunk, are there any plan to keep nodejs more inline with active versions?

from emsdk.

Dont overwrite shell's node. use an environment variable or something else.

@juj makes the opposite argument in #1142. Its not clear which approach is better/easier.

from emsdk.

Currently we do recommend using emsdk_env whenever you want to run emcc. As it happens you can get away with running emcc without emsdk_env if you manually put emcc in your PATH.. but the recommented way to put emcc in your path is to run emsdk_env. I doubt that recommendation will change.

I think the expression "clobber the system version node" is a little strong. What we are doing is putting the emsdk version of node first in the PATH when emsdk is active, the system version is still there, its just no longer first in the PATH.

from emsdk.

Another user run into this issue and is confused by the current behaviour: #1074

from emsdk.

Currently we don't have a good way to use emsdk without shadowing the system node. I tried to make it do that we would not shadow by default in #714 but @juj though it was too magical.

Perhaps we can/should revisit #714. Failing that I can think of two other options (714 being option 1 here):

1. Only add node to PATH if there isn't one there already.
2. Add our version of node to the end of the user's existing PATH (unlike the other PATHs we add, i.e. the path emcc)
3. Add an explicit option to opt out of node JS being added to PATH.

from emsdk.

from emsdk.

It not being exposing in the PATH for the any internal usage. Its being added to the PATH do that users can run the their generated output.

See #1142 for the argument.

I tend to agree that this is not very useful compared the downside (overriding any node in the user's PATH).

from emsdk.

Its being added to the PATH do that users can run the their generated output.

Okay, I've gone back and read juj's explanation. (It was only a few comments above, so sorry I didn't read it before.)

This use of emsdk_env.sh is quite unexpected (to me at least). I wonder how many people have actually been making use of it?

While there is perhaps a valid use for emsdk_env.sh as it currently operates, I don't think it should be the default. My recommendation would be for the docs to instruct the user to install their OS's Node package, and to only run a source ... script that will add emcc etc to PATH.

But maybe if it is just added to the end of PATH then the problem will be solved? I assume it's currently added to the front of PATH so that if you've installed, for example, Debian's version of Emscripten then Emsdk will take precedence over Debian's 2.0.12? That's reasonable. But it would also make sense that if Emscripten's node is only being exposed as a convenience that it not take priority over other instances of Node.

from emsdk.

@sbc100 It seems like there's pretty broad agreement that the current behavior is bad. People keep running into it and being surprised.

There's various ways to fix it, but the simplest would be to just not set node in the path at all. I think this would be much less confusing than the current behavior, and could be done now without without needing to design something fancier. Would you be willing to merge a PR which implements that? I'm happy to submit it such a PR if that would help (looks like just a matter of removing activated_path from this config?).

Otherwise, what needs to be done to make this happen?

from emsdk.

@bakkot, yes, my option is that it would be best to remove node from the path (just like we don't add binayen, llvm, or python). However @juj made it clear that he though it was useful to have node added to the PATH.

I tried in #714 (comment) to land a compromise and only add node to that PATH is it wasn't already found, which is kind of the best of both approaches but adds complexity and is kind of magic.

I wonder if @juj can be persuaded to bless one of these approaches?

I just though of another thing that might help: What is we had an exploit ./emsdk install node for those that want it in the PATH and ./emsdk install sdk would not depend on that version but instead it would depend on e.g. node-internal. @juj WDYT?

from emsdk.

I sent a message to the list to try to see who out there is depending on this: https://groups.google.com/g/emscripten-discuss/c/KnG7io3zu2Q

from emsdk.

@juj Fair enough i will work around it.

I would rather not have to try and keep up to date with all the dependencies of emscripten because emsdk install/activate sdk-upstream-main-64bit could change

Agree, this problem makes sense - in order to be able to get "emsdk activate everything - node", one will currently need to first learn and track what "everything" is in order to arrive to that everything minus node. That is a shortcoming.

Could there be an omit/exclude flag added instead?

emsdk activate "3.1.10" --exclude node

This would be a clean feature to add. Name bikeshedding, I'd prefer --skip=node, e.g.

emsdk activate foo bar xyz --skip=node

would match to skip node-*-*bit (i.e. all tools with name node, any version, any bitness)

However, reading further about the confusion, I think this would probably not be a best of all worlds solution, because it would be solving the problem by layering more concepts on people (rather than simplifying). That being said, as a general feature, I think such a --skip= feature would be good to have. (note I think the equals sign there would be preferable, since a command line emsdk install emscripten --skip node python is a bit troublesome for the reader, would prefer that to raise an ambiguity error)

Let me ask a another question, and more specific to why i need to exclude node. What is the cadence that emscripten will update nodejs? For example emscripten is producing wasm that the node version it includes does not support. That's really the only reason i need to use a different node. I would happily use the bundled nodejs, unfortunately i can't.

If/when Emscripten is producing wasm content that Emsdk-bundled node is unable to run, that is by itself an issue with Emsdk, and I think we should address that issue as well. If Emsdk has fallen badly behind, then updating is certainly in order.

Though using a relatively old version in Emsdk to ensure that our CI tests against the "minimum supported version" is also important - but I think here we can explicitly instruct the CI to download the min spec version, and default users to download a new version (in case we get into these types of testing trouble). Maybe straddle different testing targets to download different nodes.

I kind of like the idea of something like emsdk_env --exclude=node or maybe emsdk_env --minimal. The minimal mode would just be emcc on the PATH and nothing more.

I'd recommend against emsdk --minimal, because such a name is introducing a new concept, and requires one to learn what meaning it carries.

Also, if we do a minimal mode that only has Emscripten in PATH and nothing else, then we can still get other users complaining why they don't have a way to say they want Emscripten-provided Python or Clang in PATH, but not Node or Java. The permutations are large, so a general option like --exclude=node seems like it could reach more use cases. (although now we'd be mixing emsdk activate --skip vs emsdk_env --skip uses.. but maybe in general those might apply equally for easier cognitive load)

I think the main issue here is that the documentation suggests that you should always be in emsdk_env.

It sounds like emsdk_env is kind of a specialist thing.

I agree with this characterisation. emsdk_env has intended to be the "full solution in isolation", that would enable the simplest getting started scenarios (just get me everything) and the CI/distribution scenarios (make sure I have the 100% exact same bits across systems). We can not lose these features.

But like people have mentioned in the above comments, they are running into a problem that the node that Emsdk provides is not enough to run their Emscripten produced wasm features. If that is the case, then that issue should specifically be addressed - sounds like Emsdk bundled node has fallen too far behind(?). Please raise specific issues about those (sorry if you did already, my attention bandwidth may be struggling)

Most of the time, I just want to invoke emcc in my shell, not mess around with all the other tools exactly as emcc sees them.

In this scenario, do you want Emscripten to invoke your supplied node, or Emsdk-bundled node? (I presume this?) The issue Emsdk intends to solve is to avoid the possibility that Emsdk gives the users a partial installation of the toolchain where they are unable to compile code, since python or node or java or clang or some other dependency the user has installed would not be up to what Emscripten would expect. I.e. to be able to source the 100% same tools that the Emscripten CIs also are testing.

If #714 (comment), that's fine, as long as the documentation does not suggest always entering an emsdk_env and it is possible to run emcc without being in emsdk_env.

Output says "Next steps: [...] Configure emsdk in your shell startup scripts by running [...]", ok, I'll do that

node has been downgraded, WTF?

I would like to go through exactly the same process, including the "emcc works in a new shell part", except without the last step.

Thanks, now I understand. I agree this is to a great extent a documentation problem. emsdk_env indeed is intended to be entered specifically when one wants to do Emscripten compilation activities - having the getting started docs tell people to put it in their profile does go beyond that usage, and that instruction is what is guiding people astray when they want to do non-Emscripten things in their terminals.

it sounds like @juj is thinking of emsdk_env as being something the user does explicitly when they want to enter the emsdk environment, which means they shouldn't be surprised by having the emsdk-provided versions of tools like node, rather than being something which is done by default in every shell, which is how the documentation currently suggests using it.

I agree with this characterization.

To solve these issues, I recommend that we modify the behavior of emsdk_env by making it also accept a list of tool (/command) names as a parameter, which will tell emsdk_env which tools to add to PATH. For example, if one writes

source ./emsdk_env.sh emcc clang

then emsdk_env would enter an environment where these two tools only will be brought into the user's PATH. The earlier emsdk activate step that wrote Emscripten .config file will still also persist, and will dictate which tools Emscripten compiler itself uses. (so the above command will not redirect Emscripten compiler to utilize user's custom Node.js when compiling)

Then, let's create an alias *, i.e. if one writes

source ./emsdk_env.sh *

then all the tools that were previously activated with emsdk activate will be provided in PATH.

If no parameters are specified, i.e.

source ./emsdk_env.sh

then it will be the same as if one had written

source ./emsdk_env.sh emcc

(this will be a breaking change. For a transition period, we can issue a hint if this produces no node at all in user's PATH)

This way, we can have the getting started docs recommend people to run source ./emsdk_env.sh * in their initial command line on terminal to be able to right after run node hello_world.js, but when we recommend people to add a thing in their ~/.profile, we can recommend them to write source ./emsdk_env.sh in there so that Emsdk won't override node for them (and as the other option, recommend them to write *, or explicitly list out the Emscripten toolchain tools they want to have available 24/7 from emsdk).

Hopefully that would make it easy for people to target all scenarios they wish for, with minimal pain?

from emsdk.

To solve these issues, I recommend that we modify the behavior of emsdk_env by making it also accept a list of tool (/command) names as a parameter, which will tell emsdk_env which tools to add to PATH.

That sounds good to me!

This way, we can have the getting started docs recommend people to run source ./emsdk_env.sh * in their initial command line on terminal to be able to right after run node hello_world.js, but when we recommend people to add a thing in their ~/.profile, we can recommend them to write source ./emsdk_env.sh in there so that Emsdk won't override node for them

I would suggest instead testing if node is already available, and then prompting the user to run either source ./emsdk_env.sh emcc or source ./emsdk_env.sh emcc node depending on if it is or is not respectively (i.e., only suggest they use emsdk to set up node in the PATH if they do not already have it). And making the same suggestion for modifying the shell profile. That way the experience is the same during first setup as on subsequent use.

And then source ./emsdk_env.sh * can be reserved only for CI environments.

from emsdk.

@curiousdannii with the above proposal it would be opt-in, as I read it?

from emsdk.

Modifying emsdk_env like this SGTM. (And yes my reading is that node would then be opt in.. either via writing node or * on the command line.. since he default would be just emcc).

from emsdk.

Sure, I think whether and how we recommend using * is quite a minor detail

The main point is that there exists some way to preserve the old behaviour for those what do want emsdk to be a completely solution (some folks I think do want emsdk to install all dependencies without having to go use apt-get for some of the stuff).

from emsdk.

Is there no workaround for this issue as of today? I am working both on a project that uses emscripten and an unrelated project that needs Node v18, and I don't want to be forced to manually call source emsdk_env.sh every time I work on the emscripten project. I also would rather not have to modify emsdk.py because (asides from it being really bad practice) I'll totally forget I did so the next time I install emscripten, and I'll again be puzzled by my system Node version being silently trumped by someone else.

Is my best shot sed-ing the emscripten version of Node out of my PATH in my .zshrc? Is there a more elegant solution?

from emsdk.

Is there no workaround for this issue as of today?

See #1142 (comment).

from emsdk.

Is there no workaround for this issue as of today?

See #1142 (comment).

IIUC the ideal solution doesn't avoid installing the emsdk version of node, or have emscripten use an unexpected/system version. The goal is to avoid having emsdk's override system node in the PATH, right? Avoiding emsdk node, even within emsdk itself, seems like overkill and could cause other issues.

from emsdk.

IIUC the ideal solution doesn't avoid installing the emsdk version of node

As mentioned in that comment, you cannot activate Emscripten without installing the bundled Node.js provided by emsdk.

or have emscripten use an unexpected/system version.

This assumes that the globally-installed Node.js version is newer than what emsdk provides. emsdk currently provides Node.js 14.x, which active support ended one year ago. Security updates for this version ends in 5 months. So, I would argue to at least update the bundled Node.js version to the latest LTS version, which would also fix issues like emscripten-core/emscripten#18084.

FWIW, this workaround is currently used in at least libjxl and wasm-vips to avoid compatibility issues with Wasm SIMD, which requires Node.js 16.4.0 or later to match the final SIMD opcodes.
https://github.com/libjxl/libjxl/blob/a58dd9de85f512d6b8870d2ab40caa8423ae0e51/.github/workflows/build_test.yml#L401
https://github.com/kleisauke/wasm-vips/blob/e89ea44d7376c0bbd35cc3ff28d79d97a6340104/Dockerfile#L23-L24

The goal is to avoid having emsdk's override system node in the PATH, right?

I agree that would be the ideal solution. I only pointed out that comment since he was looking for a workaround.

Avoiding emsdk node, even within emsdk itself, seems like overkill and could cause other issues.

Just wondering, what do you mean with other issues? With the same assumption as above, at least for Emscripten, using a globally-installed Node.js should be safe after PR emscripten-core/emscripten#18070.

Another solution would be that emsdk only installs the bundled Node.js, if a globally-installed Node.js is not available, or if the globally-installed Node.js version is lower than the minimum required Node.js version (or the one provided by emsdk). This would also fix this PATH clobbering issue in most cases.

from emsdk.

Just wondering, what do you mean with other issues? With the same assumption as above, at least for Emscripten, using a globally-installed Node.js should be safe after PR emscripten-core/emscripten#18070.

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

Another solution would be that emsdk only installs the bundled Node.js, if a globally-installed Node.js is not available, or if the globally-installed Node.js version is lower than the minimum required Node.js version (or the one provided by emsdk). This would also fix this PATH clobbering issue in most cases.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

from emsdk.

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

I understand. There's a test-latest-node CI job in Emscripten, but that currently lacks some coverage as it only runs two tests.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

Great, I had forgotten about that PR. IIUC, that only affects the emsdk activate phase and still downloads and extracts the Node.js tarball (or zipball on Windows)?

If so, perhaps a better solution would be to do this check in the emsdk install phase? We should probably only do this if Emscripten is installed (i.e. only for the tags/aliases listed in emscripten-releases-tags.json). This way, users can still install and activate the bundled Node.js provided by emsdk via:

$ ./emsdk install node-14.18.2-64bit
$ ./emsdk active node-14.18.2-64bit

from emsdk.

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

I understand. There's a test-latest-node CI job in Emscripten, but that currently lacks some coverage as it only runs two tests.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

Great, I had forgotten about that PR. IIUC, that only affects the emsdk activate phase and still downloads and extracts the Node.js tarball (or zipball on Windows)?

Yes, in fact it still uses the bundled version of node within emscripten (i.e. it sets NODE_JS in the config file). All it does it avoid adding this version of node to the PATH during emsdk_env. In this sense it is a very minimal solution to this bug (which is about PATH conflicts, and not so much about want emscripten to use the host node).

If so, perhaps a better solution would be to do this check in the emsdk install phase? We should probably only do this if Emscripten is installed (i.e. only for the tags/aliases listed in emscripten-releases-tags.json). This way, users can still install and activate the bundled Node.js provided by emsdk via:

$ ./emsdk install node-14.18.2-64bit
$ ./emsdk active node-14.18.2-64bit

That would be an alternative, and slightly more intrusive/risky change yes. It would solve a slightly different issue which is "I want emcc to use my system version of node".

from emsdk.

The simplest solution might just be to avoid using emsdk_env.sh completely. Instead you can simply add emsdk/upstream/emscripten to your PATH.

@sbc100 Is there anyway to pin the current solution to the top of this issue? It took me a while to find this amongst the full discussion.

As a JavaScript developer learning Emscripten, this was a very confusing. I am using Emscripten along with JavaScript/TypeScript build tools to bundle an application and I am using a very recent version of Node.js.

from emsdk.