electron / build-tools Goto Github PK

The GN scripts to use for Electron dev-flows

License: MIT License

JavaScript 98.37% Batchfile 1.14% Shell 0.49%

build-tools's Introduction

📝 Available Translations: 🇨🇳 🇧🇷 🇪🇸 🇯🇵 🇷🇺 🇫🇷 🇺🇸 🇩🇪. View these docs in other languages on our Crowdin project.

The Electron framework lets you write cross-platform desktop applications using JavaScript, HTML and CSS. It is based on Node.js and Chromium and is used by the Visual Studio Code and many other apps.

Follow @electronjs on Twitter for important announcements.

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

Installation

To install prebuilt Electron binaries, use npm. The preferred method is to install Electron as a development dependency in your app:

npm install electron --save-dev

For more installation options and troubleshooting tips, see installation. For info on how to manage Electron versions in your apps, see Electron versioning.

Platform support

Each Electron release provides binaries for macOS, Windows, and Linux.

macOS (Catalina and up): Electron provides 64-bit Intel and ARM binaries for macOS. Apple Silicon support was added in Electron 11.
Windows (Windows 10 and up): Electron provides ia32 (x86), x64 (amd64), and arm64 binaries for Windows. Windows on ARM support was added in Electron 5.0.8. Support for Windows 7, 8 and 8.1 was removed in Electron 23, in line with Chromium's Windows deprecation policy.
Linux: The prebuilt binaries of Electron are built on Ubuntu 20.04. They have also been verified to work on:
- Ubuntu 18.04 and newer
- Fedora 32 and newer
- Debian 10 and newer

Quick start & Electron Fiddle

Use Electron Fiddle to build, run, and package small Electron experiments, to see code examples for all of Electron's APIs, and to try out different versions of Electron. It's designed to make the start of your journey with Electron easier.

Alternatively, clone and run the electron/electron-quick-start repository to see a minimal Electron app in action:

git clone https://github.com/electron/electron-quick-start
cd electron-quick-start
npm install
npm start

Resources for learning Electron

electronjs.org/docs - All of Electron's documentation
electron/fiddle - A tool to build, run, and package small Electron experiments
electron/electron-quick-start - A very basic starter Electron app
electronjs.org/community#boilerplates - Sample starter apps created by the community

Programmatic usage

Most people use Electron from the command line, but if you require electron inside your Node app (not your Electron app) it will return the file path to the binary. Use this to spawn Electron from Node scripts:

const electron = require('electron')
const proc = require('node:child_process')

// will print something similar to /Users/maf/.../Electron
console.log(electron)

// spawn Electron
const child = proc.spawn(electron)

Mirrors

China

See the Advanced Installation Instructions to learn how to use a custom mirror.

Documentation translations

We crowdsource translations for our documentation via Crowdin. We currently accept translations for Chinese (Simplified), French, German, Japanese, Portuguese, Russian, and Spanish.

Contributing

If you are interested in reporting/fixing issues and contributing directly to the code base, please see CONTRIBUTING.md for more information on what we're looking for and how to get started.

Community

Info on reporting bugs, getting help, finding third-party tools and sample apps, and more can be found on the Community page.

License

MIT

When using Electron logos, make sure to follow OpenJS Foundation Trademark Policy.

build-tools's People

Contributors

Stargazers

Watchers

build-tools's Issues

Cannot find module after installing @electron/build-tools on Arch

internal/modules/cjs/loader.js:895
  throw err;
  ^

Error: Cannot find module '/home/ross/.electron_build_tools/src/e'
Require stack:
- /usr/lib/node_modules/@electron/build-tools/run.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:892:15)
    at Function.Module._load (internal/modules/cjs/loader.js:742:27)
    at Module.require (internal/modules/cjs/loader.js:964:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/usr/lib/node_modules/@electron/build-tools/run.js:15:1)
    at Module._compile (internal/modules/cjs/loader.js:1075:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1096:10)
    at Module.load (internal/modules/cjs/loader.js:940:32)
    at Function.Module._load (internal/modules/cjs/loader.js:781:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/usr/lib/node_modules/@electron/build-tools/run.js' ]
}

I just ran npm install -g @electron/build-tools on Arch LInux and got this error after rehashing in zsh and running e init --root=~/electron --bootstrap testing.

`e build` fails when Goma isn't running

OS: Windows 10 64-bit (Build 18362)

After restarting my machine, Goma will no longer be running. If I attempt to run e build, it will fail when compiling the first module.

gomacc_common.cc:166] failed to connect to: C:\Users\Sam\AppData\Local\Temp\goma\goma.ipc: 10061
compiler_proxy isn't running. Run 'goma_ctl.bat ensure_start'.
ninja: build stopped: subcommand failed.
ERROR Error: Command failed: ninja electron
    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.depotExecFileSync [as execFileSync] (D:\work\build-tools\src\utils\depot-tools.js:91:16)
    at runNinja (D:\work\build-tools\src\e-build.js:61:9)
    at Object.<anonymous> (D:\work\build-tools\src\e-build.js:105:3)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)

To fix this, I need to run the following

build-tools/third_party/goma/goma_ctl.py ensure_start

Starting a build should check whether Goma is running, if not disabled, before continuing.

Successful build on first-run on clean Ubuntu machine

Running e init on a clean Linux (Ubuntu) machine should work first time and self-correct missing dependencies along the way (by prompting the user or by auto running commands) so that it will build Electron with no prompts and with maximum goma cache hits.

Chose Ubuntu here as it's probably the easiest support and the most commonly used in a VM for development.

build electron on MacOS, getting the code steps, "gclient sync" command fail

I am build electron on MacOS.
While getting the code steps,
I found a problem using the command "gclient sync --with_branch_heads --with_tags",
Please help me out.
Issue is reproduced on 3 different macbooks.

System: MacOS 10.14.6

________ running 'vpython src/electron/script/apply_all_patches.py src/electron/patches/config.json' in '/Users/user/Documents/webrtc/electron_gn'
...
Applying: feat: enable passing cli flags
Hook 'vpython src/electron/script/apply_all_patches.py src/electron/patches/config.json' took 371.61 secs
Traceback (most recent call last):on_external_binaries
  File "/Users/user/Documents/webrtc/depot_tools/metrics.py", line 267, in print_notice_and_exit
    yield
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 3182, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 3168, in main
    return dispatcher.execute(OptionParser(), argv)
  File "/Users/user/Documents/webrtc/depot_tools/subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 2725, in CMDsync
    ret = client.RunOnDeps('update', args)
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 1797, in RunOnDeps
    self.RunHooksRecursively(self._options, pm)
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 1083, in RunHooksRecursively
    hook.run()
  File "/Users/user/Documents/webrtc/depot_tools/gclient.py", line 259, in run
    always_show_header=self._verbose)
  File "/Users/user/Documents/webrtc/depot_tools/gclient_utils.py", line 549, in CheckCallAndFilter
    **kwargs)
  File "/Users/user/Documents/webrtc/depot_tools/subprocess2.py", line 170, in __init__
    % (str(e), kwargs.get('cwd'), args[0]))
OSError: Execution failed with error: [Errno 2] No such file or directory.
Check that /Users/user/Documents/webrtc/electron_gn or python3 exist and have execution permission.

document the python 2 mac certificate issue

Leaving this as a breadcrumb so that I don't forget, because I don't have enough time to write it up properly right now. Reported this morning in #askanything:

Hook 'python3 src/electron/script/update-external-binaries.py' took 34.35 secs
________ running 'vpython -c import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["python", "script/lib/npx.py", "[email protected]", "install", "--frozen-lockfile"]);' in '/Users/chrisberry/Projects/electron/boot'
npx: installed 1 in 7.184s
yarn install v1.15.2
$ node -e 'process.exit(0)'
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > [email protected]" has unmet peer dependency "eslint-plugin-promise@>=4.0.0".
[4/4] Building fresh packages...
Done in 251.06s.
Hook 'vpython -c 'import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["python", "script/lib/npx.py", "[email protected]", "install", "--frozen-lockfile"]);'' took 260.46 secs
________ running 'vpython src/build/landmines.py' in '/Users/chrisberry/Projects/electron/boot'
________ running 'vpython src/third_party/depot_tools/update_depot_tools_toggle.py --disable' in '/Users/chrisberry/Projects/electron/boot'
________ running 'vpython src/tools/remove_stale_pyc_files.py src/android_webview/tools src/build/android src/gpu/gles2_conform_support src/infra src/ppapi src/printing src/third_party/blink/renderer/build/scripts src/third_party/blink/tools src/third_party/catapult src/tools' in '/Users/chrisberry/Projects/electron/boot'
________ running 'vpython src/buildtools/ensure_gn_version.py git_revision:0c5557d173ce217cea095086a9c9610068123503' in '/Users/chrisberry/Projects/electron/boot'
________ running 'vpython src/build/mac_toolchain.py' in '/Users/chrisberry/Projects/electron/boot'
Skipping Mac toolchain installation for mac
________ running 'vpython src/tools/clang/scripts/update.py' in '/Users/chrisberry/Projects/electron/boot'
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac/clang-n332890-c2443155-1.tgz 
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>
Retrying in 5 s ...
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac/clang-n332890-c2443155-1.tgz 
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>
Retrying in 10 s ...
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac/clang-n332890-c2443155-1.tgz 
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>
Retrying in 20 s ...
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac/clang-n332890-c2443155-1.tgz 
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)>
Failed to download prebuilt clang package clang-n332890-c2443155-1.tgz

Pedro chimed in with this helpful info:

This variant of Python 2.7 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the Keychain Access application and the security command line utility are no longer used as defaults by the Python ssl module. A sample command script is included in /Applications/Python 2.7 to install a curated bundle of default root certificates from the third-party certifi package (https://pypi.python.org/pypi/certifi). Click on Install Certificates to run it. If you choose to use certifi, you should consider subscribing to the project's email update service to be notified when the certificate bundle is updated.
If you don't have /Applications/Python 2.7, do: pip install --upgrade certifi

If `e fetch` fails, it cannot be run again since `electron` already exiasts

I've tried running e fetch multiple times with failures for various things and each time it creates electron without cleaning up on failures.

Initializing in root directory fails to create .gclient config

When initializing the code on Windows, the process fails due to .gclient file not existing. I'm going to keep this issue open while I work through fixing it.

Sam@blackmesa MINGW64 /d/work/electron/master
$ e init --root=./ --bootstrap testing
New build config testing created in D:\work\build-tools\configs\evm.testing.json
Now using config testing
Creating D:\work\electron\master\src
Running "python D:\work\build-tools\third_party\depot_tools\gclient.py sync --with_branch_heads --with_tags -v --ignore_locks" in D:\work\electron\master\src
Looking for .gclient starting from D:\work\electron\master\src

Couldn't find configuration file.
Error: client not configured; see 'gclient config'
ERROR Error: Command failed: python D:\work\build-tools\third_party\depot_tools\gclient.py sync --with_branch_heads --with_tags -v --ignore_locks
    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.depotExecFileSync [as execFileSync] (D:\work\build-tools\src\utils\depot-tools.js:86:16)
    at runGClientSync (D:\work\build-tools\src\e-sync.js:35:9)
    at Object.<anonymous> (D:\work\build-tools\src\e-sync.js:48:3)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12) 
ERROR Error: Command failed: node D:\work\build-tools\src\e sync -v --ignore_locks      
    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.<anonymous> (D:\work\build-tools\src\e-init.js:183:18)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12) 
    at internal/main/run_main_module.js:18:47

One pain point I came across is that when the command fails, the user needs to know to delete build-tools/configs/evm.testing.json or else it can't be re-run.

`e init` fails on Windows when there are spaces in a directory name

The full text of the error is here: https://gist.github.com/strega-nil/32b1eee85f5c24e8f648e41ce7552ecb

The important part looks like it's here:

gclient_utils(833) enqueue:enqueued(src/electron)
gclient(565) requirements:Dependency(src/electron).requirements = ()
gclient(565) requirements:Dependency(src/electron).requirements = ()
gclient_utils(1011) __init__:_Worker(src/electron) reqs:()
gclient_utils(1020) run:_Worker.run(src/electron)
gclient(916) run:Dependency(src/electron).run()
subprocess2(154) __init__:git --version;  cwd=.
gclient_utils(1036) run:Caught exception in thread src/electron
gclient_utils(1037) run:(<class 'FileNotFoundError'>, FileNotFoundError(2, 'The system cannot find the file specified', None, 2, None), <traceback object at 0x000001D5F174A700>)
gclient_utils(1040) run:_Worker.run(src/electron) done

e show configs throw error if configs not exist

if i have not configs then command e show configs throw error:

The variable "goma_dir" was set as a build argument

C:\Users\x99999\src\electron>gn.bat gen "out/Testing0903" --args="import("//electron/build/args/testing.gn") import("D:\electron\0827\000\build-tools-master\third_party\goma.gn")
ERROR at /D:/electron/0827/000/build-tools-master/third_party/goma.gn:1:12: Build argument has no effect.
goma_dir = "D:\electron\0827\000\build-tools-master\third_party\goma"
^---------------------------------------------------------
The variable "goma_dir" was set as a build argument
but never appeared in a declare_args() block in any buildfile.

To view possible args, run "gn args --list "
ERROR No targets.
I could not find any targets to write, so I'm doing nothing.
ERROR

why AppData\Local\Temp\goma\goma.ipc is empty?

Successful build on first-run on clean macOS machine

Running e init on a clean macOS machine should work first time and self-correct missing dependencies along the way (by prompting the user or by auto running commands) so that it will build Electron with no prompts and with maximum goma cache hits

e build failed

D:\electron\0827\000\build-tools-master>e build
Running "goma_ctl.py ensure_start" in D:\electron\0827\000\build-tools-master\third_party\goma
Not logged in
E0831 14:44:30.165356 11780 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:44:30.166384 11780 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
E0831 14:45:50.312211 12256 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:45:50.313288 12256 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
E0831 14:47:10.508966 14292 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:47:10.508966 14292 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
Traceback (most recent call last):
File "goma_ctl.py", line 3194, in
sys.exit(main())
File "goma_ctl.py", line 3189, in main
goma.Dispatch(sys.argv[1:])
File "goma_ctl.py", line 1434, in Dispatch
self._action_mappings.get(args[0], self._DefaultAction)()
File "goma_ctl.py", line 808, in _EnsureStartCompilerProxy
self._GenericStartCompilerProxy(ensure=True, force_update=False)
File "goma_ctl.py", line 767, in _GenericStartCompilerProxy
self._env.ExecCompilerProxy()
File "goma_ctl.py", line 2117, in ExecCompilerProxy
return self._GetCompilerProxyPort(proc=proc) # set the new gomacc_port.
File "goma_ctl.py", line 1677, in _GetCompilerProxyPort
raise e
main.Error: compiler_proxy is not ready? pid=12416
ERROR Error: Command failed: python goma_ctl.py ensure_start
Not logged in
E0831 14:44:30.165356 11780 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:44:30.166384 11780 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
E0831 14:45:50.312211 12256 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:45:50.313288 12256 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
E0831 14:47:10.508966 14292 gomacc_common.cc:166] failed to connect to: C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc: 10061
E0831 14:47:10.508966 14292 goma_ipc.cc:71] call failed: Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc
GOMA: port request failed. connect_success: 0, err: -1 Failed to connect to C:\Users\ZWX9531\AppData\Local\Temp\goma\goma.ipc, http_return_code: 0
Traceback (most recent call last):
File "goma_ctl.py", line 3194, in
sys.exit(main())
File "goma_ctl.py", line 3189, in main
goma.Dispatch(sys.argv[1:])
File "goma_ctl.py", line 1434, in Dispatch
self._action_mappings.get(args[0], self._DefaultAction)()
File "goma_ctl.py", line 808, in _EnsureStartCompilerProxy
self._GenericStartCompilerProxy(ensure=True, force_update=False)
File "goma_ctl.py", line 767, in _GenericStartCompilerProxy
self._env.ExecCompilerProxy()
File "goma_ctl.py", line 2117, in ExecCompilerProxy
return self._GetCompilerProxyPort(proc=proc) # set the new gomacc_port.
File "goma_ctl.py", line 1677, in _GetCompilerProxyPort
raise e
main.Error: compiler_proxy is not ready? pid=12416

at checkExecSyncError (child_process.js:621:11)
at Object.execFileSync (child_process.js:639:15)
at Object.ensureGomaStart [as ensure] (D:\electron\0827\000\build-tools-master\src\utils\goma.js:129:16)
at runNinja (D:\electron\0827\000\build-tools-master\src\e-build.js:63:10)
at Object.<anonymous> (D:\electron\0827\000\build-tools-master\src\e-build.js:132:3)
at Module._compile (internal/modules/cjs/loader.js:868:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:879:10)
at Module.load (internal/modules/cjs/loader.js:731:32)
at Function.Module._load (internal/modules/cjs/loader.js:644:12)
at Function.Module.runMain (internal/modules/cjs/loader.js:931:10)

Ship a vs_toolchain so that windows users can get maximum cache hits

Looks like we just want to run this https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/refs/heads/master/win_toolchain/package_from_installed.py on our CI machine and take the resulting zip as our toolchain.

Goma hangs on first run

OS: Windows 10 64-bit

If Goma hasn't been started yet, e build seems to hang when ensuring it's started.

D:\work\electron>e build
Running "goma_ctl.py ensure_start" in D:\work\build-tools\third_party\goma
Terminate batch job (Y/N)? y

D:\work\electron>e build
Running "goma_ctl.py ensure_start" in D:\work\build-tools\third_party\goma
Not logged in
Running "D:\work\build-tools\third_party\depot_tools\gn.bat gen "out/Testing" --args="import(\"//electron/build/args/testing.gn\") import(\"D:\work\build-tools\third_party\goma.gn\")"" in D:\work\electron\master\src
Done. Made 12190 targets from 2156 files in 12242ms

To reproduce, kill compiler_proxy.exe in the task manager then run e build again.

switch branch or tags fail

On windows, when I use
e init master-testing --root=~/src/electron
e sync
It's ok.

Now I want to swith to branch 7-3-x. I can't figure out where to go.

I use " e sync --revision 7-3-x", it shows checkout 7-3-x success. But source code is not update. When I force sync, it error.
or should I modify .gclient to http://github.com/electron/electron@7-3-x on unmanaged url?
Does unmanaged affect switch branch?

I don't find any tip on internet.

goma_auth.py login failed

D:\electron\0827\000\build-tools-master\third_party\goma>goma_auth.py login
You can use Electron Goma (distributed compiler service) if you agree on the
way we use data and the way we share data.

How do we use data?
a. the data used for compiling

source code (including contents and file paths) to be compiled.
header files (including contents and file paths) used during the compile.
other files accessed by compilers. e.g. asan_blacklist.txt, crtbegin.o,
profiling data for pgo.
identifier of a compiler to use. (SHA256 hash value of a compiler, version,
target).
command line arguments and environment variables necessary for a compile,
system include paths, current working directory.

b. the data used for authentication

OAuth2 access token to use service, and email address gotten from access
token.

Electron may use data for logging and tracking (including abuse detection).
Electron keeps identifier of each compile (goma client start time, goma client
id that changes when compiler_proxy starts, sequential compile id)

What data will be shared?

Contents in source code and header files are stored in Electron servers and
shared among users who send SHA256 hash values of them. Compile results
are shared among users who have sent the requests that bring the same
compile result.

How long data will be kept?

Contents in source code and header files will be kept at most 153 days
since last used.

Do you agree to our data usage policy? (y/n) -->y
You have agreed.
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FVFRH877X to authenticate.

AADSTS165000: Invalid Request: The request did not include the required tokens for the user context. One or more of the user context values (cookies; form fields; headers) were not supplied, every request must include these values and maintain them across a complete single user flow. The request did not return all of the form fields. Failure Reasons:[Token was not provided;]
故障排除详细信息
如果你联系管理员，请向其发送此信息。
将信息复制到剪贴板
Request Id: 2a80cc5c-248c-446c-9ff4-fff686f93c00
Correlation Id: 4f2a0371-9360-4b3c-9af0-bdf32a46adb9
Timestamp: 2020-08-31T07:08:49Z
Message: AADSTS165000: Invalid Request: The request did not include the required tokens for the user context. One or more of the user context values (cookies; form fields; headers) were not supplied, every request must include these values and maintain them across a complete single user flow. The request did not return all of the form fields. Failure Reasons:[Token was not provided;]
高级诊断: 启用
如果打算通过获取支持来解决问题，请启用此功能，并尝试重现错误。这会收集有助于排查此问题的其他信息。

Detect missing python 3 on macOS

#94

This is something we can easily detect and give a helpful message for.

e build electron failed when running build-tools

Windows cmd.exe:
ERROR Can't find source root.
I could not find a ".gn" file in the current directory or any parent,
and the --root command-line argument was not specified.

download Dependency squirrel.mac all the time

[0:11:40] Still working on: [0:11:40] src gclient(555) requirements:Dependency(src/third_party/electron_node).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/nan).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/Mantle).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/ReactiveObjC).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') [0:11:50] Still working on: [0:11:50] src gclient(555) requirements:Dependency(src/third_party/electron_node).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/nan).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/Mantle).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/ReactiveObjC).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') [0:12:00] Still working on: [0:12:00] src gclient(555) requirements:Dependency(src/third_party/electron_node).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/nan).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/Mantle).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/ReactiveObjC).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') [0:12:10] Still working on: [0:12:10] src gclient(555) requirements:Dependency(src/third_party/electron_node).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/nan).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/Mantle).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/ReactiveObjC).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') [0:12:20] Still working on: [0:12:20] src gclient(555) requirements:Dependency(src/third_party/electron_node).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/nan).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac).requirements = ('src', 'src/electron') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/Mantle).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac') gclient(555) requirements:Dependency(src/third_party/squirrel.mac/vendor/ReactiveObjC).requirements = ('src', 'src/electron', 'src/third_party/squirrel.mac')

for 50 minutes

Does not create `.git_cache` by default

It would be nice if the script automatically created this directory on first-run rather than forcing the user to create it :)

Python error when running "e init"

Operating system: OSX 10.14.6

When following the README and I run the following command,

e init --root=~/electron --bootstrap testing

I get the following error (error truncated to most relevant parts),

[0:00:02] fetch +refs/tags/*:refs/tags/* took 0.0 minutes
[0:00:02] Fetching +refs/branch-heads/*:refs/branch-heads/*
[0:00:02] running "git fetch -v --progress --prune origin +refs/branch-heads/*:refs/branch-heads/*" in "/Users/macbook/.git_cache/github.com-electron-electron"
[0:00:02] fetch +refs/branch-heads/*:refs/branch-heads/* took 0.0 minutes
Traceback (most recent call last):
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1044, in _Clone
    print_stdout=print_stdout, filter_fn=filter_fn)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1400, in _Run
    if self._IsRunningUnderRosetta():
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1383, in _IsRunningUnderRosetta
    libSystem = ctypes.CDLL("libSystem.dylib")
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libSystem.dylib, 6): image not found
----------------------------------------
Traceback (most recent call last):
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/metrics.py", line 267, in print_notice_and_exit
    yield
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py", line 3195, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py", line 3181, in main
    return dispatcher.execute(OptionParser(), argv)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py", line 2737, in CMDsync
    ret = client.RunOnDeps('update', args)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py", line 1780, in RunOnDeps
    patch_refs=patch_refs, target_branches=target_branches)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_utils.py", line 947, in flush
    reraise(e[0], e[1], e[2])
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_utils.py", line 1024, in run
    self.item.run(*self.args, **self.kwargs)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py", line 926, in run
    file_list)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 132, in RunCommand
    return getattr(self, command)(options, args, file_list)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 546, in update
    self._Clone(revision, url, options)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1044, in _Clone
    print_stdout=print_stdout, filter_fn=filter_fn)
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1400, in _Run
    if self._IsRunningUnderRosetta():
  File "/Users/macbook/.electron_build_tools/third_party/depot_tools/gclient_scm.py", line 1383, in _IsRunningUnderRosetta
    libSystem = ctypes.CDLL("libSystem.dylib")
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libSystem.dylib, 6): image not found
ERROR Error: Command failed: python /Users/macbook/.electron_build_tools/third_party/depot_tools/gclient.py sync --with_branch_heads --with_tags -vv -v --ignore_locks
    at checkExecSyncError (child_process.js:621:11)
    at Object.execFileSync (child_process.js:639:15)
    at Object.depotExecFileSync [as execFileSync] (/Users/macbook/.electron_build_tools/src/utils/depot-tools.js:97:16)
    at runGClientSync (/Users/macbook/.electron_build_tools/src/e-sync.js:37:9)
    at Object.<anonymous> (/Users/macbook/.electron_build_tools/src/e-sync.js:50:3)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
    at Function.Module._load (internal/modules/cjs/loader.js:723:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
ERROR Error: Command failed: /Users/macbook/.nvm/versions/node/v12.14.1/bin/node /Users/macbook/.electron_build_tools/src/e sync -v --ignore_locks
    at checkExecSyncError (child_process.js:621:11)
    at Object.execFileSync (child_process.js:639:15)
    at Object.<anonymous> (/Users/macbook/.electron_build_tools/src/e-init.js:184:18)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
    at Function.Module._load (internal/modules/cjs/loader.js:723:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
    at internal/main/run_main_module.js:17:11

I was able to get around this error by disabling SIP protection - not an approach I plan to keep for the longterm.

Has anyone encountered this and how did you go about getting e init to work without having to disable SIP?

Goma build dependency fails to install

OS: Windows 10 Pro 64-bit (Build 18362)
Node: v12.16.0
NPM: 6.2.0
Yarn: 1.19.1

After running e init --root=~/electron --bootstrap testing, I eventually receive a failure with the following error log:

Downloading https://electron-build-tools.s3-us-west-2.amazonaws.com/build-dependencies/f97c88aa5d49395ae20387b6329ad406fd019f5fb4aac4ba639ca928b7151f6b/goma-win.zip to C:\Users\Sam\AppData\Local\Temp\electron-s3t4p345\goma-win.zip
Download goma-win.zip    52486903  [1049738060.0%] 

...

________ running 'vpython.bat -vpython-spec src/.vpython -vpython-tool install' in 'D:\work\electron\master'
Writing new goma.gn file D:\work\build-tools\third_party\goma.gn
Downloading "https://electron-build-tools.s3-us-west-2.amazonaws.com/build-dependencies/d13f5e07c3306fa0367a662feea6133870e30a03767c23123da72a4e3a385d2e/goma-win.zip" into D:\work\build-tools\third_party\goma-win.zip
[====================================================================================================================================================] 17581491/bps 100% 0.0s 
Running "goma_ctl.py ensure_start" in D:\work\build-tools\third_party\goma
Traceback (most recent call last):
  File "goma_ctl.py", line 3164, in <module>
    sys.exit(main())
  File "goma_ctl.py", line 3158, in main
    goma = GetGomaDriver()
  File "goma_ctl.py", line 3152, in GetGomaDriver
    env = _GOMA_ENVS[os.name]()
  File "goma_ctl.py", line 2485, in __init__
    raise e
ImportError: No module named win32process
Exception AttributeError: "'GomaEnvWin' object has no attribute '_delete_tmp_dir'" in <bound method GomaEnvWin.__del__ of <__main__.GomaEnvWin object at 0x0351FB90>> ignored 
ERROR Error: Command failed: python goma_ctl.py ensure_start
Traceback (most recent call last):
  File "goma_ctl.py", line 3164, in <module>
    sys.exit(main())
  File "goma_ctl.py", line 3158, in main
    goma = GetGomaDriver()
  File "goma_ctl.py", line 3152, in GetGomaDriver
    env = _GOMA_ENVS[os.name]()
  File "goma_ctl.py", line 2485, in __init__
    raise e
ImportError: No module named win32process
Exception AttributeError: "'GomaEnvWin' object has no attribute '_delete_tmp_dir'" in <bound method GomaEnvWin.__del__ of <__main__.GomaEnvWin object at 0x0351FB90>> ignored 

    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.ensureGomaStart [as ensure] (D:\work\build-tools\src\utils\goma.js:111:16)
    at runNinja (D:\work\build-tools\src\e-build.js:49:10)
    at Object.<anonymous> (D:\work\build-tools\src\e-build.js:105:3)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
ERROR Error: Command failed: node D:\work\build-tools\src\e build
    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.<anonymous> (D:\work\build-tools\src\e-init.js:202:18)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
    at internal/main/run_main_module.js:18:47

Running e sync then e build after receiving this error results in the same exception.

$ e build
Running "goma_ctl.py ensure_start" in D:\work\build-tools\third_party\goma
Traceback (most recent call last):
  File "goma_ctl.py", line 3164, in <module>
    sys.exit(main())
  File "goma_ctl.py", line 3158, in main
    goma = GetGomaDriver()
  File "goma_ctl.py", line 3152, in GetGomaDriver
    env = _GOMA_ENVS[os.name]()
  File "goma_ctl.py", line 2485, in __init__
    raise e
ImportError: No module named win32process
Exception AttributeError: "'GomaEnvWin' object has no attribute '_delete_tmp_dir'" in <bound method GomaEnvWin.__del__ of <__main__.GomaEnvWin object at 0x03824B90>> ignored 
ERROR Error: Command failed: python goma_ctl.py ensure_start
Traceback (most recent call last):
  File "goma_ctl.py", line 3164, in <module>
    sys.exit(main())
  File "goma_ctl.py", line 3158, in main
    goma = GetGomaDriver()
  File "goma_ctl.py", line 3152, in GetGomaDriver
    env = _GOMA_ENVS[os.name]()
  File "goma_ctl.py", line 2485, in __init__
    raise e
ImportError: No module named win32process
Exception AttributeError: "'GomaEnvWin' object has no attribute '_delete_tmp_dir'" in <bound method GomaEnvWin.__del__ of <__main__.GomaEnvWin object at 0x03824B90>> ignored 

    at checkExecSyncError (child_process.js:630:11)
    at Object.execFileSync (child_process.js:648:15)
    at Object.ensureGomaStart [as ensure] (D:\work\build-tools\src\utils\goma.js:111:16)
    at runNinja (D:\work\build-tools\src\e-build.js:49:10)
    at Object.<anonymous> (D:\work\build-tools\src\e-build.js:105:3)
    at Module._compile (internal/modules/cjs/loader.js:1157:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
    at Module.load (internal/modules/cjs/loader.js:1001:32)
    at Function.Module._load (internal/modules/cjs/loader.js:900:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)

Build tools `--interactive` parameter does not work

Problem Description

Running e start --interactive should start the interactive REPL but then it immediately exits preventing any stdin

Expectation

e start -i should behave the same as ~/electron/src/out/Testing/Electron.app/Contents/MacOS/Electron -i

Proposed Solution

TBD

Additional Information

Adjust ninja parallism based on Goma cache status

By default, builds run using ninja -j 200. I assume this works well when using the custom Electron Goma backend. However, when building locally this parallelism likely doesn't help speedup builds when it greatly exceeds the number of local CPU cores.

I'd propose looking into checking whether the local user will be running the full build locally or utilizing a remote Goma backend. If running locally, the number of parallel build commands shouldn't far exceed the number of CPU cores.

For now, I'm adjusting the number of parallel commands by running e build -j [number-of-cores + 2].

evm config migration doesn't persist

OS: Windows 10 Pro 64-bit (2004)

After running e build, I'm informed that my configuration is outdated and will be updated for me. However, each time I run e build, I get the same warning.

D:\work\electron>e build
WARN Your evm config main-testing is using an old remote configuration "{
  origin: {
    electron: '[email protected]:electron/electron.git',
    node: '[email protected]:electron/node.git'
  }
}" - we've updated it for you
Updating C:\Users\Sam\.electron_build_tools\third_party\depot_tools
Running "cmd.exe /c C:\Users\Sam\.electron_build_tools\third_party\depot_tools\update_depot_tools.bat"
Running "ninja -j 200 electron" in D:\work\electron\main\src\out\Testing

documentation should say "do not put spaces in your home directory"

Hook 'vpython.bat src/electron/script/apply_all_patches.py src/electron/patches/config.json' took 15669.14 secs
subprocess2(154) __init__:python3 src/electron/script/update-external-binaries.py;  cwd=C:\src\electron
________ running 'python3 src/electron/script/update-external-binaries.py' in 'C:\src\electron'
Downloading https://electron-build-tools.s3-us-west-2.amazonaws.com/build-dependencies/b6a20fd1c2026f3792e7286bc768a7ebc261847b76449b49f55455e1f841fecd/sccache-win32-x64.zip to C:\Users\NICOLE~1\AppData\Local\Temp\electron-xzab9lsb\sccache-win32-x64.zip
Download sccache-win32-x64.zip     4151947  [103798675.0%]
Hook 'python3 src/electron/script/update-external-binaries.py' took 11.25 secs
subprocess2(154) __init__:vpython.bat -c import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["python", "script/lib/npx.py", "[email protected]", "install", "--frozen-lockfile"]);;  cwd=C:\src\electron
________ running 'vpython.bat -c import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["python", "script/lib/npx.py", "[email protected]", "install", "--frozen-lockfile"]);' in 'C:\src\electron'
Error: EPERM: operation not permitted, mkdir 'C:\Users\Nicole'

Building electron_dist_zip on ubuntu

Hello,

I'm building electron v4.2.12 from source code on Ubuntu 16.04 64bit (I use instructions https://github.com/electron/electron/blob/v4.2.12/docs/development/build-instructions-gn.md) by commands:

gn gen out/Release-x64 --args='import("//electron/build/args/release.gn")'
ninja -C out/Release-x64 electron:electron_dist_zip

and after all i have large size dist.zip file (745mb). If I will use this file for building linux electron application, then application will have large size too.

How I can build lightweight dist.zip file from source code on Ubuntu?

How to prepare compiler_proxy?

Traceback (most recent call last):
File "goma_ctl.py", line 3194, in
sys.exit(main())
File "goma_ctl.py", line 3189, in main
goma.Dispatch(sys.argv[1:])
File "goma_ctl.py", line 1434, in Dispatch
self._action_mappings.get(args[0], self._DefaultAction)()
File "goma_ctl.py", line 808, in _EnsureStartCompilerProxy
self._GenericStartCompilerProxy(ensure=True, force_update=False)
File "goma_ctl.py", line 767, in _GenericStartCompilerProxy
self._env.ExecCompilerProxy()
File "goma_ctl.py", line 2117, in ExecCompilerProxy
return self._GetCompilerProxyPort(proc=proc) # set the new gomacc_port.
File "goma_ctl.py", line 1677, in _GetCompilerProxyPort
raise e
main.Error: compiler_proxy is not ready? pid=9888

Automatically fix global git config issues

We should detect if we're running in an interactive environment (stdin must be a valid pipe and process.env.CI should not be set) and if so prompt the user if they want us to automatically fix the issues with the git config that we detect.

Usage with existing electron checkout?

Using an existing Electron checkout that I already had, I seem to be unable to run any commands.

Configuration '/Users/erickzhao/Developer/electron-gn-scripts/nix/commands/../../generated.env.sh' not found. (Do you need to run 'evm' or 'e fetch'?)

As per the error message, I tried to run evm to no avail, as I don't already have a config.yml:

Config file: /Users/erickzhao/Developer/electron-gn-scripts/config..yml not found

And I don't want to run e fetch to install a new Electron build. Is there a way around this?

Support using two different configs in different terminals

I want to be able to evm use foo in one shell and evm use bar in the other shell and both to still work. nvm supports this so technically it must be possible to set this kind of thing up

Subpar configuration options for users without write access

During the setup process it's possible to run

$ mkdir electron && cd electron
$ gclient config --name "src/electron" --unmanaged https://github.com/electron/electron
$ gclient sync --with_branch_heads --with_tags
# This will take a while, go get a coffee.

And use one's own fork instead of https://github.com/electron/electron. This should also be configurable from e init to prevent users from needing to add a new remote later in the process.

electron build failing

As indicated in
electron/electron#21835

To me this does not look to be a "build-tools" problem but the source tree is having invalid entries. But my issue was closed and told to file it here.

The command used here was "e init ... --bootstrap testing" resulted in the failure below.

Failure cause is the path to npx.cmd isn't defined. If I override manually at the shell, then it errors out for npx-cli.js misplaced. The errors will compound. The patches or something is screwing up the paths and location where files need to be unzipped at.

Besides this, the build-tools wasn't working initially many times, it just gets stuck after init command displaying contents of .gclient and do nothing for more than 20 minutes. I had multi level folder like "my-electron/master". Switched to single level, set the nodist to v12.8.1 and not sure what else, this morning it started to pull code. But took a lot longer than standard gclient sync. Maybe network was busy during the day.

And there is no option to checkout a branch I created in my local repo of electron (such as " gclient sync -r release/v8.0.0-beta.7 --with_branch_heads --with_tags")

Applying: Revert "crypto: add oaepLabel option"
Applying: refactor: TransferrableModule is deprecated, use CompiledWasmModule instead
Applying: enable 31 bit smis on 64bit arch and ptr compression
Applying: fix: remove unused llhttp variables
Applying: fix: include libuv header in node_binding
Applying: remove deprecated task API override removed in latest v8
Hook 'vpython.bat src/electron/script/apply_all_patches.py src/electron/patches/config.json' took 234.87 secs
________ running 'python3 src/electron/script/update-external-binaries.py' in 'C:\Users\muralir-hf\bt-electron'
Downloading https://github.com/electron/electron-frameworks/releases/download/v1.4.1/directxsdk-x64.zip to C:\Users\MURALI~1\AppData\Local\Temp\electron-cb0z92ox\directxsdk-x64.zip
Download directxsdk-x64.zip     1806291  [180629100.0%]
Downloading https://github.com/electron/electron-frameworks/releases/download/v1.4.1/sccache-win32-x64.zip to C:\Users\MURALI~1\AppData\Local\Temp\electron-0u_cwsuz\sccache-win32-x64.zip
Download sccache-win32-x64.zip     4151947  [103798675.0%]
Downloading https://github.com/electron/electron-frameworks/releases/download/v1.4.1/goma-win.zip to C:\Users\MURALI~1\AppData\Local\Temp\electron-ue1bvdc3\goma-win.zip
Download goma-win.zip    52486903  [1049738060.0%]
Hook 'python3 src/electron/script/update-external-binaries.py' took 52.53 secs
________ running 'vpython.bat -c import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["pyt    npx(*sys.argv[1:])
  File "script/lib/npx.py", line 7, in npx
    subprocess.check_call(call_args)
  File "C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\bootstrap-3_8_0_chromium_8_bin\python\bin\Lib\subprocess.py", line 185, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\bootstrap-3_8_0_chromium_8_bin\python\bin\Lib\subprocess.py", line 172, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\bootstrap-3_8_0_chromium_8_bin\python\bin\Lib\subprocess.py", line 394, in __init__
    errread, errwrite)
  File "C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\bootstrap-3_8_0_chromium_8_bin\python\bin\Lib\subprocess.py", line 644, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\bootstrap-3_8_0_chromium_8_bin\python\bin\Lib\subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'script/lib/npx.py', '[email protected]', 'install', '--frozen-lockfile']' returned non-zero exit status 1
Error: Command 'vpython.bat -c import os, subprocess; os.chdir(os.path.join("src", "electron")); subprocess.check_call(["python", "script/lib/npx.py", "[email protected]", "install", "--frozen-lockfile"]);' returned non-zero exit status 1 in C:\Users\muralir-hf\bt-electron
ERROR Error: Command failed: python C:\Users\muralir-hf\sdev\build-tools\src\third_party\depot_tools\gclient.py sync --with_branch_heads --with_tags -v
    at checkExecSyncError (child_process.js:621:11)
    at Object.execFileSync (child_process.js:639:15)
    at Object.depotExecFileSync [as execFileSync] (C:\Users\muralir-hf\sdev\build-tools\src\utils\depot-tools.js:80:16)
    at runGClientSync (C:\Users\muralir-hf\sdev\build-tools\src\e-sync.js:35:9)
    at Object.<anonymous> (C:\Users\muralir-hf\sdev\build-tools\src\e-sync.js:48:3)
    at Module._compile (internal/modules/cjs/loader.js:868:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:879:10)
    at Module.load (internal/modules/cjs/loader.js:731:32)
    at Function.Module._load (internal/modules/cjs/loader.js:644:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:931:10)
ERROR Error: Command failed: node C:\Users\muralir-hf\sdev\build-tools\src\e sync -v
    at checkExecSyncError (child_process.js:621:11)
    at Object.execFileSync (child_process.js:639:15)
    at Object.<anonymous> (C:\Users\muralir-hf\sdev\build-tools\src\e-init.js:192:18)
    at Module._compile (internal/modules/cjs/loader.js:868:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:879:10)
    at Module.load (internal/modules/cjs/loader.js:731:32)
    at Function.Module._load (internal/modules/cjs/loader.js:644:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:931:10)
    at internal/main/run_main_module.js:17:11

How to use the cache-only Goma config?

when running `e init` on a bootstrap that already exists, the path of how to progress is unclear

I've picked up build_tools a few times, either from fresh or after a few months. Each time, my brain goes to the same bootstrap name which ends up failing since it already exists. I can't tell if it being in some destroyed/incomplete state matters here, but that is usually the state of things when I'm trying to re-run it.

An example of the command I run once:

$ e init --root=~/electron --bootstrap local

Then I get this:

ERROR Error: Build config local already exists! (/Users/cyren/GitHub/electron/build-tools/configs/evm.local.json)
    at Object.<anonymous> (/Users/cyren/GitHub/electron/build-tools/src/e-init.js:152:11)
    at Module._compile (internal/modules/cjs/loader.js:1200:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1220:10)
    at Module.load (internal/modules/cjs/loader.js:1049:32)
    at Function.Module._load (internal/modules/cjs/loader.js:937:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47

It would be nice if there was:

a way to either pick up where I left off with that e init, or
a way to wipe that config

e build always regenerates with gn gen on windows

I think there's a line-endings issue with the comparison we do

`e init` fails on Windows 1903

In cmd:

> set PATH=C:\Users\Nicole Mazzuca\src\langs\electron-build-tools\src;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Python36\Scripts\;C:\Program Files\Python36\;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\LLVM\bin;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Users\Nicole Mazzuca\AppData\Local\Programs\Python\Python38\Scripts\;C:\Users\Nicole Mazzuca\AppData\Local\Programs\Python\Python38\;C:\Users\Nicole Mazzuca\AppData\Roaming\local\bin;C:\Users\Nicole Mazzuca\.cargo\bin;C:\Users\Nicole Mazzuca\AppData\Local\Microsoft\WindowsApps;C:\Users\Nicole Mazzuca\AppData\Roaming\npm;C:\Users\Nicole Mazzuca\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Nicole Mazzuca\AppData\Local\Microsoft\WindowsApps;C:\Users\Nicole Mazzuca\.dotnet\tools;C:\Program Files (x86)\Vim\vim80

> del /S /Q ..\electron
Deleted file - C:\Users\Nicole Mazzuca\src\langs\electron\.gclient

> del /S /Q configs\*
Deleted file - C:\Users\Nicole Mazzuca\src\langs\electron-build-tools\configs\evm.testing.json

> e init --root=..\electron --bootstrap testing
New build config testing created in C:\Users\Nicole Mazzuca\src\langs\electron-build-tools\configs\evm.testing.json
ERROR Error: spawnSync node ENOENT
    at Object.spawnSync (internal/child_process.js:1045:20)
    at spawnSync (child_process.js:597:24)
    at Object.execFileSync (child_process.js:624:15)
    at Object.<anonymous> (C:\Users\Nicole Mazzuca\src\langs\electron-build-tools\src\e-init.js:184:16)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47

Create `e lint` to make linting available to users using exclusively build-tools

Refs electron/electron#25411 (comment).

Compilation error .o file

Hi, when I try to build the Electron 4.2.12 build at some point the error falls:
[2889/34291] ACTION // electron: build_electron_definitions (// build / toolchain / mac: clang_x64) FAILED: gen / electron / tsc / typings / electron.d.ts python ../../electron/build/npm-run.py --silent gn-typescript-definitions - / Users / evgen / Documents / webrtc / electron-gn / electron / src / out / Testing / gen / electron /tsc/typings/electron.d.ts NPM script 'gn-typescript-definitions' failed with code '1':

After that, an error occurs on a random .o file, for example:
[2938/34291] OBJCXX obj / skia / skia_metal / GrMtlRenderTarget.o ninja: build stopped: subcommand failed.

I'm building on macOs mojave 10.14.6, xcode 11.3.1
P.S. progress is moving very fast and reaches 3600 in about a minute

goma_auth.py login failed

D:\electron\0827\000\build-tools-master\third_party\goma>goma_auth.py login You can use Electron Goma (distributed compiler service) if you agree on the way we use data and the way we share data. 1. How do we use data? a. the data used for compiling - source code (including contents and file paths) to be compiled. - header files (including contents and file paths) used during the compile. - other files accessed by compilers. e.g. asan_blacklist.txt, crtbegin.o, profiling data for pgo. - identifier of a compiler to use. (SHA256 hash value of a compiler, version, target). - command line arguments and environment variables necessary for a compile, system include paths, current working directory. b. the data used for authentication - OAuth2 access token to use service, and email address gotten from access token. Electron may use data for logging and tracking (including abuse detection). Electron keeps identifier of each compile (goma client start time, goma client id that changes when compiler_proxy starts, sequential compile id) 2. What data will be shared? Contents in source code and header files are stored in Electron servers and shared among users who send SHA256 hash values of them. Compile results are shared among users who have sent the requests that bring the same compile result. 3. How long data will be kept? Contents in source code and header files will be kept at most 153 days since last used. Do you agree to our data usage policy? (y/n) -->y You have agreed. To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FVFRH877X to authenticate.

after sign in, error below:

Permission Denied When using `npm` for installation (Ubuntu)

My OS:

Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.4 LTS
Release:	18.04
Codename:	bionic

When I used sudo npm i -g @electron/build-tools It didn't install the build tools and this was the result :

islam@islam-Inspiron-5558:~$ sudo npm i -g @electron/build-tools

> @electron/[email protected] preinstall /usr/lib/node_modules/@electron/build-tools
> node preinstall.js

fatal: could not create work tree dir '/home/islam/.electron_build_tools': Permission denied
Failed to install build-tools:  Error: Command "git clone" failed with exit code 128
    at throwForBadSpawn (/usr/lib/node_modules/@electron/build-tools/preinstall.js:10:11)
    at install (/usr/lib/node_modules/@electron/build-tools/preinstall.js:37:7)
    at Object.<anonymous> (/usr/lib/node_modules/@electron/build-tools/preinstall.js:61:1)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47
/usr/bin/e -> /usr/lib/node_modules/@electron/build-tools/run.js
+ @electron/[email protected]
added 1 package from 1 contributor in 1.529s
islam@islam-Inspiron-5558:~$ e
bash: /usr/local/bin/e: No such file or directory

I tried changing the permissions for the folder sudo chown -R $(whoami):$(whoami) /home/islam, It didn't work. however when I used yarn it worked without any issues.

islam@islam-Inspiron-5558:~$ sudo yarn global add @electron/build-tools
yarn global v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "@electron/[email protected]" with binaries:
      - e
Done in 2.33s.
islam@islam-Inspiron-5558:~$ e
Usage: e <command> [commandArgs...]

Electron build tool

Options:
  -h, --help                 output usage information

Commands:
  init|new [options] <name>  Create a new build config
  sync [gclientArgs...]      Get or update source code
  build|make [options]       Build Electron and other things
  start|run                  Run the Electron executable
  node                       Run the Electron build as if it were a Node.js executable
  debug                      Run the Electron build with a debugger (gdb or lldb)
  use <name>                 Use build config <name> when running other `e` commands
  show <subcommand>          Show info about the current build config
  test [specRunnerArgs...]   Run Electron's spec runner
  pr [options]               Open a GitHub URL where you can PR your changes
  patches <basename>         Refresh the patches in $root/src/electron/patches/$basename
  open <sha1|PR#>            Open a GitHub URL for the given commit hash / pull # / issue #
  load-xcode                 Loads required versions of Xcode and the macOS SDK and symlinks them.  This may require sudo
  check-for-updates          Check for build-tools updates
  update-goma                Ensure a fresh copy of Goma is installed
  depot-tools|d              Run a command from the depot-tools directory with the correct configuration
  help [cmd]                 display help for [cmd]

See https://github.com/electron/build-tools/blob/master/README.md for usage.

small note: If I use yarn without sudo It won't work.
I think we can add notes about this for ubuntu or recommend to use yarn as a first choice.

MacOS 10.15.3, xcode 11.3 fails for build-tools, electron builds etc

If I could get the requirements for getting electron build on latest macos & xcode, it will be really helpful.

Using build-tools, in pre-requisites, fails here:
build-tools murali-hf$ e load-xcode
Creating /Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode
Downloading "https://electron-build-tools.s3-us-west-2.amazonaws.com/macos/Xcode-10.3.zip" into /Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode/Xcode.zip
[=========================================================================================================] 29257222/bps 100% 0.0s
internal/fs/utils.js:220
throw err;
^

Error: ENOENT: no such file or directory, rename '/Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode/tmp_unzip/Xcode.app' -> '/Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode/Xcode.app'
at Object.renameSync (fs.js:643:3)
at Object.ensureXcode (/Users/murali-hf/sdev/ftl-electron/build-tools/src/utils/xcode.js:48:8)
at Object. (/Users/murali-hf/sdev/ftl-electron/build-tools/src/e-load-xcode.js:16:7)
at Module._compile (internal/modules/cjs/loader.js:959:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
at Module.load (internal/modules/cjs/loader.js:815:32)
at Function.Module.load (internal/modules/cjs/loader.js:727:14)
at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
at internal/main/run_main_module.js:17:11 {
errno: -2,
syscall: 'rename',
code: 'ENOENT',
path: '/Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode/tmp_unzip/Xcode.app',
dest: '/Users/murali-hf/sdev/ftl-electron/build-tools/third_party/Xcode/Xcode.app'
}

If I ignore & continue further, e build fails:
MR-MBP:build-tools murali-hf$ e build electron
Running "goma_ctl.py ensure_start" in /Users/murali-hf/sdev/ftl-electron/build-tools/third_party/goma
ERROR Error: spawnSync python ENOENT
at Object.spawnSync (internal/child_process.js:1041:20)
at spawnSync (child_process.js:607:24)
at Object.execFileSync (child_process.js:634:15)
at Object.ensureGomaStart [as ensure] (/Users/murali-hf/sdev/ftl-electron/build-tools/src/utils/goma.js:113:16)
at runNinja (/Users/murali-hf/sdev/ftl-electron/build-tools/src/e-build.js:46:10)
at Object. (/Users/murali-hf/sdev/ftl-electron/build-tools/src/e-build.js:102:3)
at Module._compile (internal/modules/cjs/loader.js:959:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
at Module.load (internal/modules/cjs/loader.js:815:32)
at Function.Module._load (internal/modules/cjs/loader.js:727:14)

And using electron-gn builds, it fails at
....
../../electron/external_binaries -fobjc-weak -std=c++14 -stdlib=libc++ -fobjc-call-cxx-cdtors -Wobjc-missing-property-synthesis -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -c ../../electron/shell/browser/mac/atom_application_delegate.mm -o obj/electron/electron_lib/atom_application_delegate.o
../../electron/shell/browser/mac/atom_application_delegate.mm:36:60: error: too many arguments to function call, expected 0, have 5
g_swizzle_imk_input_session->GetOriginalImplementation()(self, _cmd, range,

1 error generated.
[58/883] CXX obj/electron/electron_lib/network_hints_handler_impl.o

"e init --use-goma" is broken on new checkouts

Currently the e-init.js goma utils test for electron/external_binaries/goma before sync is run.

Unless you already have electron already checked out from another project file, this isn't going to be true...

Successful build on first-run on clean Windows machine

Running e init on a clean Windows machine should work first time and self-correct missing dependencies along the way (by prompting the user or by auto running commands) so that it will build Electron with no prompts and with maximum goma cache hits

consider always running with ignore_locks in gclient sync

e sync has failed with lock errors for a few users, who've asked about it in #ask-anything

Maybe we should just always run with --ignore_locks?

e start doesn't properly parse directory arguments

When running e start my-electron-app locally, only the default app runs.

When i log program.rawArgs it looks like:

repros ❯ e start my-electron-app                                      7:40PM
[
  '/Users/codebytere/.nvm/versions/node/v12.7.0/bin/node',
  '/Users/codebytere/build-tools/src/e',
  'start',
  'my-electron-app'
]

so i'm fairly certain in this case we'll want to grab or parse these options differently 🤔

Goma fails to update on Windows when already running

When running build tools, it fails with the following error:

D:\electron\src>e build
ERROR Error: EPERM: operation not permitted, unlink 'C:\Users\Sam\.electron_build_tools\third_party\goma\compiler_proxy.exe'
    at Object.unlinkSync (fs.js:1052:3)
    at fixWinEPERMSync (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:213:13)
    at rimrafSync (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:314:28)
    at C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:344:39
    at Array.forEach (<anonymous>)
    at rmkidsSync (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:344:26)
    at rmdirSync (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:337:7)
    at fixWinEPERMSync (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:211:5)
    at Function.rimrafSync [as sync] (C:\Users\Sam\.electron_build_tools\node_modules\rimraf\rimraf.js:314:28)
    at Object.downloadAndPrepareGoma [as downloadAndPrepare] (C:\Users\Sam\.electron_build_tools\src\utils\goma.js:46:10)

I believe the Goma hash update likely caused this in 524810f

To fix it, I had to kill compiler_proxy.exe in task manager prior to running again.

build-tools should check for updates at launch

The simplest implementation here is probably just git pull && (if-changes npx yarn) in the root directory. Not sure if we can be smarted about it, but self-updating is important with the rate of fixes / features we're landing recently.

Ensure global git settings are correct on windows

$ git config --global core.autocrlf false
$ git config --global core.filemode false
$ git config --global branch.autosetuprebase always

Not sure if we should set these automatically 🤔 But we should definitely tell users they are wrong