janitortechnology / dockerfiles Goto Github PK

View Code? Open in Web Editor NEW

53.0 53.0 20.0 295 KB

popular development environments as containers

Home Page: https://hub.docker.com/u/janitortechnology/

JavaScript 9.73% Shell 3.95% Dockerfile 86.32%

dockerfiles's People

Contributors

Stargazers

Watchers

Forkers

dlg-yahoo kewisch freaktechnik jankeromnes jgmize nt1m notriddle beaufortfrancois ishitatsuyuki coder206 rcatolino phistuck privatebin tp-tc dengyanuoapp omirror tinti vandertyson aklbafiga

dockerfiles's Issues

VNC in-IDE window should work

VNC page opened from IDE is a blank page ;(

Ensure that projects really build with Clang and not GCC

It seems that CC=clang-3.9 and CXX=clang++-3.9 are only set as Dockerfile ENV variables, and are missing from user environments. This might cause Firefox and Thunderbird to build with GCC instead of Clang. If that's the case, we should fix it (e.g. by adding these exports to .bashrc).

[ubuntu-dev] install clang with static-analysis headers and stuff

cd
git clone https://github.com/llvm-mirror/llvm.git
cd llvm/tools
git clone https://github.com/llvm-mirror/clang.git
cd clang/tools
git clone https://github.com/llvm-mirror/clang-tools-extra.git extra
cd ~/llvm/projects
git clone https://github.com/llvm-mirror/libcxx.git

cd
mkdir llvm-build
cd llvm-build
cmake -G "Unix Makefiles" ../llvm
make
sudo make install

[firefox-git] switch from moz-git-tools to git-cinnabar

This is the preferred way of using Git for Firefox nowadays.

However, it doesn't seem to work quite well with the https://github.com/mozilla/gecko-dev repository (incompatible SHAs?), so we need to make sure our cinnabar setup makes every Firefox/Git developer happy.

Make Chromium xterm as nice as Fx

I don't have history commands, <up> and <down> keys don't work, and even completion doesn't work.

[ubuntu-dev] better support unicode

There is a Firefox developer called Dão Gottwald, but in git log they show up as D<C3><A3>o Gottwald and in hg log as D?o Gottwald.

This issue prevented @nt1m from easily applying a Firefox patch in Janitor. We should make our Terminal handle Unicode better.

[chromium] switch to gn builds

GN is now the default way to build Chromium, so chromium.dockerfile and chromium-update.dockerfile should switch to it. This means using gn gen out/Default and ninja -C out/Default chrome to build.

[thunderbird] Error: Please use FINAL_TARGET_PP_FILES instead of DIST_FILES

When I try to build thunderbird.docker the following error:

$ sudo docker build - < thunderbird.docker
[ ... a bunch of stuff happens ...]
0:34.36 Reticulating splines...
0:35.03 Traceback (most recent call last):
0:35.03   File "./config.status", line 1060, in <module>
0:35.03     config_status(**args)
0:35.03   File "/home/user/thunderbird/mozilla/python/mozbuild/mozbuild/config_status.py", line 175, in config_status
0:35.03     definitions = list(definitions)
0:35.03   File "/home/user/thunderbird/mozilla/python/mozbuild/mozbuild/frontend/emitter.py", line 165, in emit
0:35.03     for out in output:
0:35.03   File "/home/user/thunderbird/mozilla/python/mozbuild/mozbuild/frontend/reader.py", line 1062, in read_mozbuild
0:35.03     raise bre
0:35.03 mozbuild.frontend.reader.BuildReaderError:
0:35.03 ==============================
0:35.03 ERROR PROCESSING MOZBUILD FILE
0:35.03 ==============================
0:35.03 
0:35.03 The error occurred while processing the following file:
0:35.03 
0:35.04     /home/user/thunderbird/mail/components/devtools/extension/moz.build
0:35.04 
0:35.04 The error was triggered on line 11 of this file:
0:35.04 
0:35.04     DIST_FILES += [
0:35.04 
0:35.04 The underlying problem is an attempt to read a reserved UPPERCASE variable that does not exist.
0:35.04 
0:35.04 The variable read causing the error is:
0:35.04 
0:35.04     DIST_FILES
0:35.04 
0:35.04 Maybe you meant TESTING_FILES or BRANDING_FILES?
0:35.04 
0:35.04 Please use
0:35.04 
0:35.04     FINAL_TARGET_PP_FILES += [ 'foo' ]
0:35.04 
0:35.04 instead of
0:35.04 
0:35.04     DIST_FILES += [ 'foo' ]
0:35.04 
0:35.07 *** Fix above errors and then restart with\
0:35.07                "/usr/bin/make -f client.mk build"
0:35.07 make[1]: *** [configure] Error 1

This looks like a build issue in Thunderbird. @kewisch any ideas?

[ubuntu-dev] install additional plugins into cloud9 sdk

We should install additional plugins into our Cloud9 SDK folder, e.g.:

https://github.com/shadowcodex/c9.ide.desktop
https://github.com/nt1m/c9.ide.reviewcomments
and more plugins to implement the project-agnostic parts of JanitorTechnology/janitor#30

Project-specific plugins should be installed by each project-specific Dockerfile (e.g. servo.dockerfile could install some plugins that set up Servo-specific contribution workflows, for example).

This can be done by importing this file into our ubuntu-dev image, into the folder /home/user/.c9sdk/configs/: https://github.com/JanitorTechnology/c9-workspace-config/blob/master/client-workspace-janitor.js

EDIT: I started doing this in #34.

[chromium] more scripted workflows for janitor.json

From this chromium-dev thread:

git cl upload -s -r [email protected] --cc [email protected] --tbrs [email protected]
git cl try
git cl try-results
git cl description
git cl comments
git cl set-commit
git cl web
git cl status
git cl issue
git cl help

Consider adding alternatives to cloud9

It would be nice to have cloud9 alternatives like brackets/the koding editor/...

[ubuntu-dev] improve cloud9 c++ auto-completion

@fiji-flo recently emailed me some details about how he improved C++ autocompletion in Cloud9 IDE a few years ago. This could be useful for us:

I dug up my old project to look how I did the autocomplete: https://github.com/fiji-flo/mozdock/blob/master/c9cpp/Dockerfile

It's just using https://github.com/invokr/c9.ide.language.cpp

Worked pretty good back in the days. If I find some time I'll make PR, but please don't wait for it, if you wanna add it yourself (I have some busy holidays coming up).

Thanks Florian! 😄 And happy holidays!

[ubuntu-dev] maybe add a "restart supervisor" command to all projects?

It may be useful to allow Janitor developers to easily restart their Supervisor daemons (e.g. if they modified their /etc/supervisord.conf file).

It could look like this:

"scripts": {
  "Restart Supervisor": "sudo kill -HUP 1"
}

Thank you @notriddle for this useful trick!

[firefox] install mozregression

This would allow anyone to easily use mozregression with Janitor's amazing bandwidth.

[firefox] support both git and hg checkouts

It would be great to support both git and hg versions of the Firefox repository "natively" (as in, telling Mercurial users to rm -rf $REPO && hg clone mozilla-central $REPO is not a great solution).

One way we could achieve this is to split the existing firefox.dockerfile into two versions: firefox-git.dockerfile and firefox-hg.dockerfile.

@nt1m Since you expressed interest in trying this, I'll tentatively assign you :)

[chromium] support chrome_sandbox to be able to run Chromium out of the box

When you build Chromium using chromium.docker, and then try to run it, you get the following error:

~/chromium/src$ out/Release/chrome
[7636:7636:0209/223219:FATAL:browser_main_loop.cc(215)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
#0 0x7fc892621c5e base::debug::StackTrace::StackTrace()
#1 0x7fc89263430b logging::LogMessage::~LogMessage()
#2 0x7fc894b26825 content::BrowserMainLoop::EarlyInitialization()
#3 0x7fc894967314 content::BrowserMainRunnerImpl::Initialize()
#4 0x7fc894966ed6 content::BrowserMain()
#5 0x7fc8925e6901 content::ContentMainRunnerImpl::Run()
#6 0x7fc8925e56b0 content::ContentMain()
#7 0x7fc8921ee1b8 ChromeMain
#8 0x7fc88b130ec5 __libc_start_main
#9 0x7fc8921ee099 <unknown>

Aborted (core dumped)

It's probably worth it to properly set-up Chrome's sandbox, e.g. by sourcing build/update-linux-sandbox.sh in the Dockerfile.

[firefox] add performance one-click actions

E.g. "make a profile" or "run benchmark".

[chromium] build fail

Built the docker image, but then couldn't build chromium. Looks like there's a version mismatch.

[ubuntu-dev] allow using `supervisorctl`

All Janitor containers use supervisor to run multiple programs in the background (e.g. an SSH daemon, the Cloud9 SDK, noVNC server, etc) and automatically restart them if they encounter a problem.

Our current configuration file is quite basic, and is missing a few fields that would enable using supervisorctl from inside the containers directly. See improvement suggestions here and here.

Additionally we could also move the configuration file from /etc/supervisord.conf to /etc/supervisor/supervisord.conf and include any files from /etc/supervisor/conf.d/*.conf (this would allow dockerfiles like the one for Discourse to simply import a new file like /etc/supervisor/conf.d/discourse.conf instead of appending rules to /etc/supervisord.conf.

[ubuntu-dev] support Remote Desktop Protocol

This could improve Remote Desktop performance for all containers.

[ubuntu-dev] web-terminal doesn't have TTY dimensions, making `less` go on forever

I heard that Mozilla's TaskCluster team uses the Chrome OS web terminal, which is apparently open.

@nt1m Would you be interested in trying it out, to see if it works better with less or git log and the likes?

[firefox] rr doesn't work

rr:

[FATAL /home/roc/rr/rr/src/RecordSession.cc:1693:create() errno: SUCCESS] rr needs /proc/sys/kernel/perf_event_paranoid <= 1, but it is 2.

gdb:

~~warning: Error disabling address space randomization: Operation not permitted~~ (just a warning, gdb will still work)

[fennec] build fails due to unavailable proguard-jar

 0:01.19 Could not find a toolchain build named `proguard-jar`

Since https://bugzilla.mozilla.org/show_bug.cgi?id=1440428 is now fixed, we can remove proguard from our Fennec image.

[chromium] install goma

From this thread on chromium-dev:

Chrome's build time is getting longer and longer. According to https://groups.google.com/a/chromium.org/forum/#!msg/chromium-dev/ThDAjO7fTro/bPQzvbVLAgAJ, build time was 30 minutes in July 2014, but it is now more than 150 minutes (without jumbo) in 2017 on particular Mac.

Google is internally using a distributed compiler service, Goma. With Goma, chrome can be built in a few minutes. However, it's available only for googlers, so non googlers need to wait for long time to build chrome, and this must be painful.

To cope with this, Goma team is working so that all Chromium developers can utilize improved compile performance enabled by Goma.

As a first step, Goma team has made the client code open source in https://chromium.googlesource.com/infra/goma/client/

Unfortunately, Goma server is not publicly available yet, but we're working on this. We hope we can make the server publicly available this year so that all Chromium developers can be more productive. Stay tuned.

I think we should install Goma in the Chromium Docker image, allowing users to configure and use Goma once it becomes more accessible (e.g. with a Chromium committer access token of some sort).

[chromium] Dockerfile parse error line 10: unknown instruction: EXPORT

It seems like the latest commit ba22b53 introduces a Dockerfile syntax error at line 10:

Error response from daemon: Dockerfile parse error line 10: unknown instruction: EXPORT
Exited with code 1

@beaufortfrancois Maybe the multi-line cat doesn't work after all in a Dockerfile?

Build logs: https://circleci.com/gh/JanitorTechnology/dockerfiles/432

[firefox-git] 'update source code' button doesn't work

When clicking on the Update source code button in Janitor in a Firefox (git) container, it tries to run:

git fetch origin && git rebase origin/master

Which fails with:

fatal: Couldn't find remote ref &&

Probably janitor.json commands / Cloud9 runner commands don't support chaining commands with &&.

[chromium] git repo in new containers is in "detached HEAD" by default

In a brand new container:

user@065ba82c187c:~/chromium/src ((03ef2fd...)) $ git branch
* (HEAD detached at origin/master)
  master

[chromium] run-webkit-tests fails at the beginning

$USER or $USERNAME (I forget which one is needed, sorry, let me know if you need the exact one) is not defined out of the box, so run-webkit-tests fails at the beginning.

We should fix this. Thanks @phistuck for reporting!

[firefox] support rr

Record & Replay debugging can be very useful to developers, so we should install and set up rr in the Firefox base image.

[servo] can't run graphically in noVNC

When trying to run Servo from within noVNC, a new window opens but then quickly disappears, and we get the following error:

~/servo$ ./mach run
Xlib:  extension "XFree86-VidModeExtension" missing on display ":99.0".
thread '<main>' panicked at 'glx::ChooseFBConfig returned no configurations.', /home/user/servo/.cargo/git/checkouts/rust-layers-[...]

A different error appears when trying to run with webrender:

~/servo$ ./mach run -w
Xlib:  extension "XFree86-VidModeExtension" missing on display ":99.0".
compile "/home/user/servo/resources/shaders/quad.vs.glsl"
Failed to compile shader: 0:5(10): error: GLSL 1.50 is not supported. Supported versions are: 1.10, 1.20, 1.30, and 1.00 ES
thread '<main>' panicked at '-- Shader compile failed - exiting --', /home/user/servo/.cargo/git/checkouts/webrender-[...]

[ubuntu-dev] switch git profile build to profile-fast

Currently, we optimize Git builds by running the entire test suite to generate an optimization profile. This makes our CircleCI builds run for 8h+.

Maybe we could switch to using Git's "profile-fast" that only runs benchmarks for the profile instead of the entire test suite. I believe this would still make the resulting binary really fast, but not take as long as 8h+ to achieve.

[firefox-git] Permission denied: '/usr/local/lib/python2.7/dist-packages/urllib3-1.22.dist-info'

Failing command: cd firefox docker login -u $DOCKER_USER -p $DOCKER_PASS docker pull janx/ubuntu-dev docker build -t janx/firefox -f firefox-git.dockerfile . docker push janx/firefox
Exit code: 2
Output:

/libpulse0_1%3a8.0-0ubuntu3.4_amd64.deb ...

Unpacking libpulse0:amd64 (1:8.0-0ubuntu3.4) ...

Selecting previously unselected package libpulse-mainloop-glib0:amd64.

Preparing to unpack .../libpulse-mainloop-glib0_1%3a8.0-0ubuntu3.4_amd64.deb ...

Unpacking libpulse-mainloop-glib0:amd64 (1:8.0-0ubuntu3.4) ...

Selecting previously unselected package libpulse-dev:amd64.

Preparing to unpack .../libpulse-dev_1%3a8.0-0ubuntu3.4_amd64.deb ...

Unpacking libpulse-dev:amd64 (1:8.0-0ubuntu3.4) ...

Selecting previously unselected package libxt-dev:amd64.

Preparing to unpack .../libxt-dev_1%3a1.1.5-0ubuntu1_amd64.deb ...

Unpacking libxt-dev:amd64 (1:1.1.5-0ubuntu1) ...

Selecting previously unselected package python-dbus.

Preparing to unpack .../python-dbus_1.2.0-3_amd64.deb ...

Unpacking python-dbus (1.2.0-3) ...

Selecting previously unselected package python-gi.

Preparing to unpack .../python-gi_3.20.0-0ubuntu1_amd64.deb ...

Unpacking python-gi (3.20.0-0ubuntu1) ...

Selecting previously unselected package yasm.

Preparing to unpack .../yasm_1.3.0-2_amd64.deb ...

Unpacking yasm (1.3.0-2) ...

Selecting previously unselected package libiw-dev:amd64.

Preparing to unpack .../libiw-dev_30~pre9-8ubuntu1_amd64.deb ...

Unpacking libiw-dev:amd64 (30~pre9-8ubuntu1) ...

Processing triggers for libc-bin (2.23-0ubuntu9) ...

Processing triggers for man-db (2.7.5-1) ...

Processing triggers for libglib2.0-0:amd64 (2.48.2-0ubuntu1) ...

Setting up libjson-c2:amd64 (0.11-4ubuntu2) ...

Setting up libasyncns0:amd64 (0.8-5build1) ...

Setting up libogg0:amd64 (1.3.2-1) ...

Setting up libpcrecpp0v5:amd64 (2:8.38-3.1) ...

Setting up libiw30:amd64 (30~pre9-8ubuntu1) ...

Setting up libgirepository-1.0-1:amd64 (1.46.0-3ubuntu1) ...

Setting up gir1.2-glib-2.0:amd64 (1.46.0-3ubuntu1) ...

Setting up autotools-dev (20150820.1) ...

Setting up dbus-x11 (1.10.6-1ubuntu3.3) ...

Setting up intltool-debian (0.35.0+20060710.4) ...

Setting up po-debconf (1.0.19) ...

Setting up libarchive-zip-perl (1.56-2) ...

Setting up libfile-stripnondeterminism-perl (0.015-1) ...

Setting up gconf2 (3.2.6-3ubuntu6) ...

Setting up gir1.2-atk-1.0 (2.18.0-1) ...

Setting up gir1.2-atspi-2.0 (2.18.3-4ubuntu1) ...

Setting up gir1.2-freedesktop:amd64 (1.46.0-3ubuntu1) ...

Setting up gir1.2-gconf-2.0 (3.2.6-3ubuntu6) ...

Setting up gir1.2-gdkpixbuf-2.0:amd64 (2.32.2-1ubuntu1.3) ...

Setting up libgtk2.0-common (2.24.30-1ubuntu1.16.04.2) ...

Setting up libpangoxft-1.0-0:amd64 (1.38.1-1) ...

Setting up gir1.2-pango-1.0:amd64 (1.38.1-1) ...

Setting up libgtk2.0-0:amd64 (2.24.30-1ubuntu1.16.04.2) ...

Setting up gir1.2-gtk-2.0 (2.24.30-1ubuntu1.16.04.2) ...

Setting up gir1.2-gtk-3.0:amd64 (3.18.9-1ubuntu3.3) ...

Setting up gir1.2-notify-0.7 (0.7.6-2svn1) ...

Setting up libasound2-dev:amd64 (1.1.0-0ubuntu1) ...

Setting up libglib2.0-bin (2.48.2-0ubuntu1) ...

Setting up libpcre16-3:amd64 (2:8.38-3.1) ...

Setting up libpcre32-3:amd64 (2:8.38-3.1) ...

Setting up libpcre3-dev:amd64 (2:8.38-3.1) ...

Setting up pkg-config (0.29.1-0ubuntu1) ...

Setting up libglib2.0-dev (2.48.2-0ubuntu1) ...

Setting up libdbus-1-dev:amd64 (1.10.6-1ubuntu3.3) ...

Setting up libxi-dev (2:1.7.6-1) ...

Setting up x11proto-record-dev (1.14.2-1) ...

Setting up libxtst-dev:amd64 (2:1.2.2-1) ...

Setting up libatspi2.0-dev (2.18.3-4ubuntu1) ...

Setting up libatk-bridge2.0-dev:amd64 (2.18.1-2ubuntu1) ...

Setting up libatk1.0-dev (2.18.0-1) ...

Setting up libcairo-script-interpreter2:amd64 (1.14.6-1) ...

Setting up libpng12-dev:amd64 (1.2.54-1ubuntu1) ...

Setting up libfreetype6-dev:amd64 (2.6.1-0.1ubuntu2.3) ...

Setting up libfontconfig1-dev:amd64 (2.11.94-0ubuntu1.1) ...

Setting up x11proto-render-dev (2:0.11.1-2) ...

Setting up libxrender-dev:amd64 (1:0.9.9-0ubuntu1) ...

Setting up libice-dev:amd64 (2:1.0.9-1) ...

Setting up libsm-dev:amd64 (2:1.2.2-1) ...

Setting up libpixman-1-dev (0.33.6-1) ...

Setting up libxcb-shm0-dev:amd64 (1.11.1-1ubuntu1) ...

Setting up libcairo2-dev (1.14.6-1) ...

Setting up libdbus-glib-1-dev (0.106-1) ...

Setting up libwayland-bin (1.12.0-1~ubuntu16.04.2) ...

Setting up libwayland-dev:amd64 (1.12.0-1~ubuntu16.04.2) ...

Setting up libmircore-dev:amd64 (0.26.3+16.04.20170605-0ubuntu1.1) ...

Setting up libprotobuf9v5:amd64 (2.6.1-1.3) ...

Setting up libprotobuf-dev:amd64 (2.6.1-1.3) ...

Setting up libxkbcommon-dev (0.5.0-1ubuntu2) ...

Setting up libmircommon-dev:amd64 (0.26.3+16.04.20170605-0ubuntu1.1) ...

Setting up libmircookie2:amd64 (0.26.3+16.04.20170605-0ubuntu1.1) ...

Setting up libmircookie-dev:amd64 (0.26.3+16.04.20170605-0ubuntu1.1) ...

Setting up libmirclient-dev:amd64 (0.26.3+16.04.20170605-0ubuntu1.1) ...

Setting up libegl1-mesa-dev:amd64 (17.0.7-0ubuntu0.16.04.2) ...

Setting up libepoxy-dev:amd64 (1.3.1-1ubuntu0.16.04.2) ...

Setting up libflac8:amd64 (1.3.1-4) ...

Setting up libgconf2-dev (3.2.6-3ubuntu6) ...

Setting up libgconf2-doc (3.2.6-3ubuntu6) ...

Setting up libgdk-pixbuf2.0-dev (2.32.2-1ubuntu1.3) ...

Setting up libgmpxx4ldbl:amd64 (2:6.1.0+dfsg-2) ...

Setting up libgmp-dev:amd64 (2:6.1.0+dfsg-2) ...

Setting up libharfbuzz-gobject0:amd64 (1.0.1-1ubuntu0.1) ...

Setting up libharfbuzz-dev (1.0.1-1ubuntu0.1) ...

Setting up libxft-dev (2.3.2-1) ...

Setting up libpango1.0-dev (1.38.1-1) ...

Setting up x11proto-xinerama-dev (1.2.1-2) ...

Setting up libxinerama-dev:amd64 (2:1.1.3-1) ...

Setting up x11proto-randr-dev (1.5.0-1) ...

Setting up libxrandr-dev:amd64 (2:1.5.0-1) ...

Setting up libxcursor-dev:amd64 (1:1.1.14-1) ...

Setting up x11proto-composite-dev (1:0.4.2-2) ...

Setting up libxcomposite-dev (1:0.4.4-1) ...

Setting up nettle-dev (3.2-1ubuntu0.16.04.1) ...

Setting up libgtk-3-dev:amd64 (3.18.9-1ubuntu3.3) ...

Setting up libgtk2.0-bin (2.24.30-1ubuntu1.16.04.2) ...

Setting up libgtk2.0-dev (2.24.30-1ubuntu1.16.04.2) ...

Setting up libsys-hostname-long-perl (1.5-1) ...

Setting up libmail-sendmail-perl (0.79.16-1) ...

Setting up libnotify-dev (0.7.6-2svn1) ...

Setting up libvorbis0a:amd64 (1.3.5-3) ...

Setting up libvorbisenc2:amd64 (1.3.5-3) ...

Setting up libsndfile1:amd64 (1.0.25-10ubuntu0.16.04.1) ...

Setting up libpulse0:amd64 (1:8.0-0ubuntu3.4) ...

Setting up libpulse-mainloop-glib0:amd64 (1:8.0-0ubuntu3.4) ...

Setting up libpulse-dev:amd64 (1:8.0-0ubuntu3.4) ...

Setting up libxt-dev:amd64 (1:1.1.5-0ubuntu1) ...

Setting up python-dbus (1.2.0-3) ...

Remove stale byte-compiled files...

Setting up python-gi (3.20.0-0ubuntu1) ...

Setting up yasm (1.3.0-2) ...

Setting up libiw-dev:amd64 (30~pre9-8ubuntu1) ...

Setting up dh-strip-nondeterminism (0.015-1) ...

Setting up debhelper (9.20160115ubuntu3) ...

Processing triggers for libc-bin (2.23-0ubuntu9) ...

Executing as root: sudo apt-get install -y autoconf2.13 build-essential ccache python-dev python-pip python-setuptools unzip uuid zip
Executing as root: sudo apt-get install -y libasound2-dev libcurl4-openssl-dev libdbus-1-dev libdbus-glib-1-dev libgconf2-dev libgtk-3-dev libgtk2.0-dev libiw-dev libnotify-dev libpulse-dev libx11-xcb-dev libxt-dev mesa-common-dev python-dbus xvfb yasm
Your version of Mercurial (4.3.2) is sufficiently modern.
Your version of Python (2.7.12) is new enough.
Your version of Rust (1.21.0) is new enough.
Rust supports x86_64-unknown-linux-gnu targets.

Source code can be obtained by running

    hg clone https://hg.mozilla.org/mozilla-unified

Or, if you prefer Git, you should install git-cinnabar, and follow the
instruction here to clone from the Mercurial repository:

    https://github.com/glandium/git-cinnabar/wiki/Mozilla:-A-git-workflow-for-Gecko-development

Or, if you really prefer vanilla flavor Git:

    git clone https://github.com/mozilla/gecko-dev.git


Your system should be ready to build Firefox for Desktop!

 ---> bd8e85b04961
Removing intermediate container affff74fc4c9
Step 4/15 : RUN pip install requests
 ---> Running in 3f4cfbf4eb19
Collecting requests
  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
Collecting urllib3<1.23,>=1.21.1 (from requests)
  Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)
Collecting idna<2.7,>=2.5 (from requests)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting certifi>=2017.4.17 (from requests)
  Downloading certifi-2017.7.27.1-py2.py3-none-any.whl (349kB)
Installing collected packages: urllib3, idna, chardet, certifi, requests
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/urllib3-1.22.dist-info'
The command '/bin/sh -c pip install requests' returned a non-zero code: 2
Exited with code 2

[all] custom wallpapers in noVNC

From the #janitor IRC channel:

11:19:51 fbeaufort> Nit: Can we have a custom wallpaper in novnc for each container?
11:20:11 fbeaufort> If I build chromium, I'll have the chromium logo in the wallpaper
11:20:19 fbeaufort> If I build firefox, I'll have the firefox logo in the wallpaper

[ubuntu-dev] consider switching to nightly rustfmt

Stable rustfmt can be a bit too old for developers of projects like Servo and Firefox.

We could install its nightly version instead, e.g. with cargo install --force rustfmt-nightly (it could then be run with something like rustup run nightly rustfmt).

[servo] circleci build fails

error: failed to run custom build command for `gecko-media v0.1.0 (https://github.com/servo/gecko-media.git#4417372d)`
process didn't exit successfully: `/home/user/servo/target/debug/build/gecko-media-ced88898ad69a8fd/build-script-build` (exit code: 101)

[chromium] `chromium-update.docker` attempts to rebuild with the wrong clang

When trying to build janx/chromium with chromium-update.docker, everything works right until the rebuild (ninja -v -C out/Release chrome -j18), which fails on a lot of error: unknown warning option. These errors are symptomatic of trying to use the host's old clang instead of the more recent in-tree version.

Log sample:

FAILED: clang++ -MMD -MF obj/v8/src/libplatform/v8_libplatform.worker-thread.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D_FILE_OFFSET_BITS=64 -DCHROMIUM_BUILD -DCR_CLANG_REVISION=259396-1 -DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_PANGO=1 -DUSE_CAIRO=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJPEG_TURBO=1 -DUSE_X11=1 -DUSE_CLIPBOARD_AURAX11=1 -DENABLE_ONE_CLICK_SIGNIN -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_PEPPER_CDMS -DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DENABLE_TOPCHROME_MD=1 -DUSE_UDEV -DFIELDTRIAL_TESTING_ENABLED -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PDF=1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_PRINTING=1 -DENABLE_BASIC_PRINTING=1 -DENABLE_PRINT_PREVIEW=1 -DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_MDNS=1 -DENABLE_SERVICE_DISCOVERY=1 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DV8_TARGET_ARCH_X64 -DV8_I18N_SUPPORT -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DENABLE_HANDLE_ZAPPING -I../../v8 -Igen -fstack-protector --param=ssp-buffer-size=4 -Werror -pthread -fno-strict-aliasing -Wall -Wno-unused-parameter -Wno-missing-field-initializers -fvisibility=hidden -pipe -fPIC -Xclang -load -Xclang /home/user/chromium/src/third_party/llvm-build/Release+Asserts/lib/libFindBadConstructs.so -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang check-templates -Xclang -plugin-arg-find-bad-constructs -Xclang follow-macro-expansion -fcolor-diagnostics -B/home/user/chromium/src/third_party/binutils/Linux_x64/Release/bin -Wheader-hygiene -Wno-char-subscripts -Wno-unneeded-internal-declaration -Wno-covered-switch-default -Wstring-conversion -Wno-c++11-narrowing -Wno-deprecated-register -Wno-inconsistent-missing-override -Wno-shift-negative-value -Wno-unused-variable -B/home/user/chromium/src/third_party/binutils/Linux_x64/Release/bin -m64 -march=x86-64 --sysroot=/home/user/chromium/src/build/linux/debian_wheezy_amd64-sysroot -m64 -fno-ident -fdata-sections -ffunction-sections -funwind-tables -fdata-sections -ffunction-sections  -O3 -fno-exceptions -fno-rtti -fno-threadsafe-statics -fvisibility-inlines-hidden -std=gnu++11 -Wno-deprecated  -c ../../v8/src/libplatform/worker-thread.cc -o obj/v8/src/libplatform/v8_libplatform.worker-thread.o
error: unknown warning option '-Wno-inconsistent-missing-override' [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-shift-negative-value'; did you mean '-Wno-shift-sign-overflow'? [-Werror,-Wunknown-warning-option]

[firefox-hg] enable watchman

Because otherwise hg diff is very slow.

[ubuntu-dev] install sccache to accelerate rust builds

https://github.com/mozilla/sccache has experimental rust support, and can significantly reduce Firefox build times. It may be helpful to other software projects as well.

[firefox][thunderbird] firefox-git, firefox-hg and thunderbird builds are broken

I just tried building firefox-git.dockerfile, firefox-hg.dockerfile and thunderbird.dockerfile, and they all fail with similar apt-get errors:

Err:96 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpulse0 amd64 1:8.0-0ubuntu3.2
  404  Not Found [IP: 91.189.88.149 80]
Err:97 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpulse-mainloop-glib0 amd64 1:8.0-0ubuntu3.2
  404  Not Found [IP: 91.189.88.149 80]
Err:98 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpulse-dev amd64 1:8.0-0ubuntu3.2
  404  Not Found [IP: 91.189.88.149 80]
Get:99 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxt-dev amd64 1:1.1.5-0ubuntu1 [394 kB]
Get:100 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-dbus amd64 1.2.0-3 [83.5 kB]
Get:101 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-gi amd64 3.20.0-0ubuntu1 [194 kB]
Get:102 http://archive.ubuntu.com/ubuntu xenial/universe amd64 yasm amd64 1.3.0-2 [351 kB]
Get:103 http://archive.ubuntu.com/ubuntu xenial/main amd64 libiw-dev amd64 30~pre9-8ubuntu1 [38.2 kB]
Fetched 20.4 MB in 5s (3927 kB/s)
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libe/libepoxy/libepoxy-dev_1.3.1-1ubuntu0.16.04.1_amd64.deb  404  Not Found [IP: 91.189.88.149 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/pulseaudio/libpulse0_8.0-0ubuntu3.2_amd64.deb  404  Not Found [IP: 91.189.88.149 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/pulseaudio/libpulse-mainloop-glib0_8.0-0ubuntu3.2_amd64.deb  404  Not Found [IP: 91.189.88.149 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/pulseaudio/libpulse-dev_8.0-0ubuntu3.2_amd64.deb  404  Not Found [IP: 91.189.88.149 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Executing as root: sudo apt-get install -y autoconf2.13 build-essential ccache python-dev python-pip python-setuptools unzip uuid zip
Executing as root: sudo apt-get install -y libasound2-dev libcurl4-openssl-dev libdbus-1-dev libdbus-glib-1-dev libgconf2-dev libgtk-3-dev libgtk2.0-dev libiw-dev libnotify-dev libpulse-dev libx11-xcb-dev libxt-dev mesa-common-dev python-dbus xvfb yasm
Traceback (most recent call last):
  File "/tmp/bootstrap.py", line 170, in <module>
    sys.exit(main(sys.argv))
  File "/tmp/bootstrap.py", line 161, in main
    dasboot.bootstrap()
  File "/tmp/tmpAVLQ4t/mozboot/bootstrap.py", line 243, in bootstrap
    
  File "/tmp/tmpAVLQ4t/mozboot/debian.py", line 103, in install_browser_packages
  File "/tmp/tmpAVLQ4t/mozboot/debian.py", line 116, in ensure_browser_packages
  File "/tmp/tmpAVLQ4t/mozboot/base.py", line 346, in apt_install
  File "/tmp/tmpAVLQ4t/mozboot/base.py", line 302, in run_as_root
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '[u'sudo', u'apt-get', u'install', u'-y', 'libasound2-dev', 'libcurl4-openssl-dev', 'libdbus-1-dev', 'libdbus-glib-1-dev', 'libgconf2-dev', 'libgtk-3-dev', 'libgtk2.0-dev', 'libiw-dev', 'libnotify-dev', 'libpulse-dev', 'libx11-xcb-dev', 'libxt-dev', 'mesa-common-dev', 'python-dbus', 'xvfb', 'yasm']' returned non-zero exit status 100
The command '/bin/sh -c wget -O /tmp/bootstrap.py https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py  && python /tmp/bootstrap.py --no-interactive --application-choice=browser  && rm -f /tmp/bootstrap.py' returned a non-zero code: 1

This looks like an apt-get cache problem that happens when you have a apt-get install or apt-get upgrade in a RUN command that has no apt-get update first.

[chromium] running chromium fails because of missing SETUID sandbox

When building and running Chromium with the commands provided in janitor.json, the browser crashes because of a missing sandbox:

$ ninja -C out/Default chrome
[...]
$ out/Default/chrome
[1377:1377:0920/115903.612452:FATAL:zygote_host_impl_linux.cc(123)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.
[...]
Calling _exit(1). Core file will not be generated.
Process exited with code: 1

@beaufortfrancois it looks like the sandbox is now required for non-Debug builds. Should we switch the janitor.json commands to Debug builds, or somehow install a sandbox instead?

EDIT: A third option would be to simply add the --no-sandbox argument to the out/Default/chrome command.

[thunderbird] `make -C objdir/calendar/lightning` is the wrong command

objdir is the wrong dir name, it's objdir-linux-... normally.

Either this or the mozconfig should be changed to set the correct dir.

[all] circleci builds fail due to "hash sum mismatch" on apt.llvm.org

All Dockerfiles contain a variant of the command sudo apt-get update -q && sudo apt-get upgrade -qy.

Currently, all image builds are broken on Circle CI, with the following error:

E: Failed to fetch https://apt.llvm.org/xenial/dists/llvm-toolchain-xenial-6.0/main/binary-amd64/Packages.gz  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

@sylvestre is this a known issue?

[fennec] build fails due to /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi

 0:06.13 checking for linker... gold
 [...]
 0:11.59 checking for gcc... /usr/bin/clang-6.0 -std=gnu99 --target=arm-linux-androideabi
 0:11.63 checking whether the C compiler (/usr/bin/clang-6.0 -std=gnu99 --target=arm-linux-androideabi -fno-short-enums -fno-exceptions  -L/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib -Wl,-rpath-link=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib --sysroot=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm -Wl,--allow-shlib-undefined -gcc-toolchain /home/user/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 ) works... no
 0:11.63 configure: error: installation or configuration problem: C compiler cannot create executables.
 0:11.63 DEBUG: <truncated - see config.log for full output>
 0:11.63 DEBUG: configure:883: checking build system type
 0:11.63 DEBUG: configure:1304: checking for ranlib
 0:11.64 DEBUG: configure:1339: checking for ar
 0:11.64 DEBUG: configure:1379: checking for gcc
 0:11.64 DEBUG: configure:1492: checking whether the C compiler (/usr/bin/clang-6.0 -std=gnu99 --target=arm-linux-androideabi -fno-short-enums -fno-exceptions  -L/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib -Wl,-rpath-link=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib --sysroot=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm -Wl,--allow-shlib-undefined -gcc-toolchain /home/user/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 ) works
 0:11.64 DEBUG: configure:1508: /usr/bin/clang-6.0 -std=gnu99 --target=arm-linux-androideabi -o conftest -fno-short-enums -fno-exceptions  -isystem /home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/include -isystem /home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/include -gcc-toolchain /home/user/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -D__ANDROID_API__=9  -L/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib -Wl,-rpath-link=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/lib --sysroot=/home/user/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm -Wl,--allow-shlib-undefined -gcc-toolchain /home/user/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64  conftest.c  1>&5
 0:11.64 DEBUG: configure:1505:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
 0:11.64 DEBUG: main(){return(0);}
 0:11.64 DEBUG: ^
 0:11.64 DEBUG: 1 warning generated.
 0:11.64 DEBUG: /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
 0:11.64 DEBUG: Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
 0:11.64 DEBUG: clang: error: linker command failed with exit code 1 (use -v to see invocation)
 0:11.64 DEBUG: configure: failed program was:
 0:11.64 DEBUG:
 0:11.64 DEBUG: #line 1503 "configure"
 0:11.64 DEBUG: #include "confdefs.h"
 0:11.64 DEBUG:
 0:11.64 DEBUG: main(){return(0);}
 0:11.64 DEBUG: configure: error: installation or configuration problem: C compiler cannot create executables.
 0:11.64 ERROR: old-configure failed
 0:11.67 *** Fix above errors and then restart with\
 0:11.67                "/usr/bin/make -f client.mk build"
 0:11.67 client.mk:145: recipe for target 'configure' failed
 0:11.67 make: *** [configure] Error 1
The command '/bin/sh -c ./mach build  && ./mach package' returned a non-zero code: 2
Exited with code 2

19:49:54 ahal> janx: pushed a patch up to review in no time with the janitor!
19:50:49 janx> Whoa thanks for letting me know! :D any annoying problems we could fix?
19:57:17 ahal> janx: I'd mostly like to be able to push to try with |mach try|
19:57:30 ahal> though I guess that's hard to solve securely
19:58:33 janx> ahal: we're planning to add a "sign in with Bugzilla" to automatically deploy your API key to your containers (and also work with Bugzilla so that Janitor API keys are scope-limited, e.g. only try access and not L3, and maybe time-limited too)
20:00:09 ahal> that would be cool
20:00:21 ahal> I don't care about pushing directly, try access is all I would care about
20:00:43 janx> ahal: thanks a lot for the feedback! I hope we can do this within a few months

[thunderbird] cannot find project browser

Currently, it's not possible to build the Thunderbird image, because ./mozilla/mach build fails.

Relevant logs:

0:09.74 Cannot find project browser
0:09.78 *** Fix above errors and then restart with\
0:09.78                "/usr/bin/make -f client.mk build"
0:09.78 make[1]: *** [configure] Error 1
0:09.78 make: *** [/home/user/thunderbird/obj-x86_64-unknown-linux-gnu/Makefile] Error 2

[firefox][thunderbird] build fails with llvm/clang 5.0

 0:09.39 checking for llvm-config... not found
 0:09.39 ERROR: Could not find LLVM/Clang installation for compiling stylo build-time
 0:09.39 bindgen.  Please specify the 'LLVM_CONFIG' environment variable
 0:09.39 (recommended), pass the '--with-libclang-path' and '--with-clang-path'
 0:09.39 options to configure, or put 'llvm-config' in your PATH.  Altering your
 0:09.39 PATH may expose 'clang' as well, potentially altering your compiler,
 0:09.39 which may not be what you intended.
 0:09.42 *** Fix above errors and then restart with\
 0:09.42                "/usr/bin/make -f client.mk build"
 0:09.42 client.mk:384: recipe for target 'configure' failed
 0:09.42 make: *** [configure] Error 1
The command '/bin/sh -c ./mach build' returned a non-zero code: 2
Exited with code 2

[fennec] missing proguard dependency

 0:11.20 checking for proguard.jar version...
 0:11.20 ERROR: proguard.jar 5.3.3 or higher is required (looked for /home/user/.mozbuild/proguard/lib/proguard.jar). Run |mach artifact toolchain --from-build proguard-jar && mv proguard ~/.mozbuild/| or add `export PROGUARD_JAR=/path/to/proguard.jar` to your mozconfig.
 0:11.24 *** Fix above errors and then restart with\
 0:11.24 "/usr/bin/make -f client.mk build"
 0:11.24 client.mk:145: recipe for target 'configure' failed