master chromium-local is breaking unless`--no-sandbox` is applied about rules_webtesting HOT 19 CLOSED

Oh, and I checked ./bazel-out/darwin-fastbuild/bin/external/io_bazel_rules_webtesting/third_party/chromium/chromium.out/chrome-mac/Chromium.app/Contents/MacOS/Chromium --version and it was Chromium 70.0.3538.0

That's a modern version and the one that rules_webtesting sets up.

from rules_webtesting.

I betting this is a bazel 0.18 thing? Maybe because user info gets scrubbed or something in some newer containerization as alluded to in the original blog post on headless Chrome? (search for "no-sandbox" in there)

from rules_webtesting.

It could also be a macOS Mojave thing? Not sure.

from rules_webtesting.

Ping

from rules_webtesting.

We actually used to have to do this internally on Linux because of a lack of support for nested sandboxing (basically Blaze/Forge was sandboxing the action, and it was not configured to allow a nested sandbox to be created within that top-level sandbox). I suspect that something similar is happening on MacOS. However, I know nothing about how sandboxing works on MacOS, so I would suggest finding and talking to whoever has done the Bazel sandboxing for Bazel.

from rules_webtesting.

Ah, as I'm not a Googler, do you know who that would be?

from rules_webtesting.

No. If I was trying to figure it out (since, although I am Googler, I don't work on Bazel), I would try to identify where the MacOS sandbox is configured in Bazel, and look at the commit history of that file to try to identify a user who might know more about it.

from rules_webtesting.

Is there any reason to believe this is fixed? I'm trying to run a very simple web browser test, and it's failing with org.openqa.selenium.SessionNotCreatedException: [Go WebDriver Client] (session not created) unable to create session.

I'm on Mac OS 10.15.1 with bazel 1.1.0-homebrew.

from rules_webtesting.

The actual issue is not fixed (AFAIK), but will not be fixed on this side. You can use "--no-sandbox" to work around it, and the containerization used for tests is a Bazel thing, not something that can be handled from within this project.

from rules_webtesting.

I got it to work locally adding tags = ["no-sandbox", "native"], (the test errored out without the native tag… ¯_(ツ)_/¯) to my java_web_test_suite rule.

Would you mind explaining what the incantation would be to pass --no-sandbox down to the Chromium process? I've tried reading through some of the code, but it's getting deep :-)

from rules_webtesting.

the containerization used for tests is a Bazel thing, not something that can be handled from within this project

I'm not sure I understand this. As I understood it, this project exists exactly to run WebDriver tests in bazel, so if something needs to be added to chromium commandlines to get that to work, this project would be exactly the right place to fix it. Or am I misunderstanding something? Thanks again for your help!

from rules_webtesting.

no-sandbox in tags doesn't do anything AFAIK.

You can send:
{ "goog:chromeOptions": { "args": ["--no-sandbox"] } }
in capabilities when starting the browser (I don't remember the exact incantation to build that up in DesiredCapabilities for Java, but it should be easy enough to figure it out by looking at Selenium Docs).

from rules_webtesting.

It is also possible to include the --no-sandbox tag as part of the browser definition (it would be added to browsers/chromium-native.json file), but we don't want to disable the sandbox for Windows and Linux users. A separate browsers/chromium-native-macos.json file could be created and selected in browsers/BUILD using select and the configs defined in common/conditions. Feel free to send a PR if you want to do this.

from rules_webtesting.

I tried changing my test thusly:

custom_browser(
    name = "no-sandbox-chromium-native",
    browser = "@io_bazel_rules_webtesting//browsers:chromium-native",
    metadata = "chromium-no-sandbox.json",
)

java_web_test_suite(
    name = "com/squareup/service/container/exemplar/BrowserTest",
    srcs = ["//service/exemplar/src/test/java:com/squareup/service/container/exemplar/BrowserTest.java"],
    browsers = [
        # "@io_bazel_rules_webtesting//browsers:chromium-local",
        ":no-sandbox-chromium-native",
    ],
   ⋮

(the json file contains the snippet at the top of this bug). But now I'm getting:

ERROR: /private/var/tmp/_bazel_zellyn/c56e2a60ec67d2dd598d789409bed348/external/io_bazel_rules_webtesting/web/web.bzl:111:12: Traceback (most recent call last):
	File "/Users/zellyn/Development/java/service/exemplar/src/test/java/BUILD.bazel", line 86
		java_web_test_suite(<11 more arguments>)
	File "/private/var/tmp/_bazel_zellyn/c56e2a60ec67d2dd598d789409bed348/external/io_bazel_rules_webtesting/web/java.bzl", line 33, in java_web_test_suite
		wrap_web_test_suite(name = name, <4 more arguments>)
	File "/private/var/tmp/_bazel_zellyn/c56e2a60ec67d2dd598d789409bed348/external/io_bazel_rules_webtesting/web/internal/wrap_web_test_suite.bzl", line 80, in wrap_web_test_suite
		web_test_suite(name = name, <14 more arguments>)
	File "/private/var/tmp/_bazel_zellyn/c56e2a60ec67d2dd598d789409bed348/external/io_bazel_rules_webtesting/web/web.bzl", line 81, in web_test_suite
		_get_kwargs(unqualified_browser, <1 more arguments>)
	File "/private/var/tmp/_bazel_zellyn/c56e2a60ec67d2dd598d789409bed348/external/io_bazel_rules_webtesting/web/web.bzl", line 111, in _get_kwargs
		out_kwargs["tags"]
key "tags" not found in dictionary

I'm not exactly sure what's going on under the covers, but this sure seems like the wrong way to check whether a key is in a dict:

from rules_webtesting.

Looks like a bug there. Should be if "tags" not in out_kwargs: Will try to get a fix submitted today.

from rules_webtesting.

Hmmm. That makes me slightly doubtful that custom_browser actually works :-)

from rules_webtesting.

fwiw, chromium is failing on our EC2 workers too… looks like that --no-sandbox flag is needed in more than one place:

$ /data/app/kochiku-worker/.cache/bazel/_bazel_kochiku-worker/0b36dc59c757f69574925eb2ab4de8ae/execroot/__main__/bazel-out/k8-fastbuild/bin/service/exemplar/src/test/java/com/squareup/service/container/exemplar/BrowserTest_chromium-local.sh.runfiles/io_bazel_rules_webtesting/third_party/chromium/chromium.out/chrome-linux/chrome
[31996:31996:1125/213655.060264:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.
#0 0x558b61db5a79 base::debug::CollectStackTrace()
#1 0x558b61d19553 base::debug::StackTrace::StackTrace()
#2 0x558b61d2dac2 logging::LogMessage::~LogMessage()
#3 0x558b633da68e service_manager::ZygoteHostImpl::Init()
#4 0x558b6193d177 content::ContentMainRunnerImpl::Initialize()
#5 0x558b619881ba service_manager::Main()
#6 0x558b6193b731 content::ContentMain()
#7 0x558b5f82b1b3 ChromeMain
#8 0x7f01570bc3d5 __libc_start_main
#9 0x558b5f82b02a _start

Received signal 6
#0 0x558b61db5a79 base::debug::CollectStackTrace()
#1 0x558b61d19553 base::debug::StackTrace::StackTrace()
#2 0x558b61db5601 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7f015d2445d0 <unknown>
#4 0x7f01570d0207 __GI_raise
#5 0x7f01570d18f8 __GI_abort
#6 0x558b61db4435 base::debug::BreakDebugger()
#7 0x558b61d2dfe6 logging::LogMessage::~LogMessage()
#8 0x558b633da68e service_manager::ZygoteHostImpl::Init()
#9 0x558b6193d177 content::ContentMainRunnerImpl::Initialize()
#10 0x558b619881ba service_manager::Main()
#11 0x558b6193b731 content::ContentMain()
#12 0x558b5f82b1b3 ChromeMain
#13 0x7f01570bc3d5 __libc_start_main
#14 0x558b5f82b02a _start
  r8: 00007f015d83eb00  r9: 0000000000000000 r10: 0000000000000008 r11: 0000000000000206
 r12: 00007ffc342749f8 r13: 00007ffc342749f8 r14: 00007ffc34275a20 r15: 00007ffc34275a18
  di: 0000000000007cfc  si: 0000000000007cfc  bp: 00007ffc342749a0  bx: 00007f0157461868
  dx: 0000000000000006  ax: 0000000000000000  cx: ffffffffffffffff  sp: 00007ffc34274868
  ip: 00007f01570d0207 efl: 0000000000000206 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

from rules_webtesting.

The tags bug can be worked around by including a tags attribute on your test (doesn't matter what it has in it, can even be [] I think).

I am not surprised that no-sandbox is needed in other contexts. When I created this project, I intended users to create their own browser definitions, and to use SauceLabs (or BrowserStack or whatever other provider in this space they want), not to use the ones in this project (they are mostly there as examples and for internal testing of the project). People have pushed back and said they want them defined in the project, so I acquiesced and have done some work to make them more reliable for others to use. That said, I only test these native browsers locally on Linux (and occasionally on MacOS in the past), and exclusively use SauceLabs for tests that run in my CI environment.

from rules_webtesting.

The tags bug can be worked around by including a tags attribute on your test (doesn't matter what it has in it, can even be [] I think).

That is of course correct. I could have sworn I tried it before posting :-) btw, the chromedriver-local will not be happy unless "native" is in the list of tags you specify. The multiple flavors of default-vs-provided tag logic as you walk down the nested starlark functions are a bit confusing.

from rules_webtesting.