flowfuse / dev-env Goto Github PK

Current Behavior

Running this project for the first time I've run into an issue while the init task ran npm install.

npm ERR! code 1
npm ERR! path /Users/zegerjan/src/github.com/flowforge/flowforge-dev-env/node_modules/sqlite3
npm ERR! command failed
npm ERR! command sh -c node-pre-gyp install --fallback-to-build
npm ERR! CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o
npm ERR!   LIBTOOL-STATIC Release/nothing.a
npm ERR!   ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3340000/sqlite3.c
npm ERR! Failed to execute '/opt/homebrew/Cellar/node/18.2.0/bin/node /opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/Users/zegerjan/src/github.com/flowforge/flowforge-dev-env/node_modules/sqlite3/lib/binding/napi-v3-darwin-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/Users/zegerjan/src/github.com/flowforge/flowforge-dev-env/node_modules/sqlite3/lib/binding/napi-v3-darwin-arm64 --napi_version=8 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)

Given I don't know which package required it (I think only flowforge requires it?) I filled the bug here.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

• FlowForge version: "main" at 29th of June, 2022
• Node.js version: v18.2.0
• npm version: 8.9
• Platform/OS: Darwin (Apple silicon)
• Browser: -

Current Behavior

If packages/ does not exist then init fails

Expected Behavior

Init should work

Steps To Reproduce

Delete packages directory, run npm init

Environment

• FlowForge version:
• Node.js version:
• npm version:
• Platform/OS:
• Browser:

Description

Right now, all FlowForge applications share the same package lock if they are installed via https://github.com/flowforge/flowforge-dev-env.

However, on staging, production, and on some developers locals, flowforge and the sister packages, are installed without a lock file. This has the potential to introduce small hard to reproduce bugs at best, and break production or introduce security issues at worst.

Without a lock file, there's no guarantee what version of a package will be installed as part of npm install. The installer will resolve to the "best" version at the time of install. This means that local/staging/production can all be running different versions. Production could even run on a newer version of a package than is running on staging.

To fix this, FlowForge repos should introduce their own package-lock files. This gives an absolute guarantee of what version of a package will be installed, and ensures consistency.

In order to do that, we either have to stop using npm workspace or find a way to have our servers all use the flowforge-dev-env to do our installs. I propose the former as a much easier route.

The main downsides, is that npm installs will take slightly longer, as the same version of a package might not be shared across all repos, and loss of npm workspace conveniences.

Epic/Story

No response

When running the init.js script the repositories are cloned using

git clone [email protected]:flowforge/${options.packages[i]}.git

which give this error

Failed to clone flowforge/flowforge: Error: Command failed: git clone [email protected]:flowforge/flowforge.git
Cloning into 'flowforge'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

use the https clone
git clone https://github.com/flowforge/${options.packages[i]}.git

works correctly

Story

No response

Description

Add new persistent context repos to project

Current Behavior

The readme states

Changing a repos dependencies
Do not run npm install in one of the repository directories under packages.

If you do, you'll need to delete the node_modules directory that gets created.

If you need to modify a repository's dependencies:

Edit its package.json to add the dependencies in the normal way
Run npm install in the root of this repository.

Since NPM workspaces are no longer used, this is incorrect.

Running npm install does not install sub package dependencies. It is now necessary to run npm install inside (for example) packages/flowforge

This however replaces the sym links with actual NPM package installs - which can catch you off guard if you are making changes across multiple repositories

Before running npm i in a sub package dir

After running npm i in a sub package dir

NOTE:

This might be a windows only thing? Will need a MAC and LINUX try-out before committing to a solution.

Expected Behavior

Either the docs should be update to reflect new practices or npm install in dev-env should have a post script to do npm install across all sub packages & then run npm run link

Steps To Reproduce

No response

Environment

• FlowForge version: NA
• Node.js version: v16.18.1
• npm version: 9.8.1
• Platform/OS: Windows 11
• Browser: NA