genodelabs / goa Goto Github PK
View Code? Open in Web Editor NEWTool for streamlining the development of Genode applications
License: GNU Affero General Public License v3.0
Tool for streamlining the development of Genode applications
License: GNU Affero General Public License v3.0
=========================================== Goa - goal, but reached a little bit sooner =========================================== Goa is a command-line-based workflow tool for the development of applications for the [https://genode.org - Genode OS Framework]. It streamlines the work of cross-developing Genode application software using a GNU/Linux-based host system for development, in particular * Importing 3rd-party source code, * Building software using commodity build systems such as CMake, * Test-running software on the host system, * Crafting runtime configurations for deployment, and * Packaging and publishing the software for the integration into Genode systems like [https://genode.org/download/sculpt - Sculpt OS]. Goa is solely geared towards application software. It does _not_ address the following topics: * Integration of complete Genode systems, * Test automation, * Continuous testing and integration (CI). Those topics are covered by the tools that come with the Genode project and are described in Chapter 5 of the Genode Foundations book: [https://genode.org/documentation/genode-foundations/23.05/development/index.html] Installation ------------ # Install the latest Genode tool chain on a GNU/Linux OS on a 64-bit x86 PC. It is recommended to use the latest long-term support (LTS) version of Ubuntu. Make sure that your installation satisfies the following requirements. * _libSDL-dev_ needed to run system scenarios directly on your host OS, * _tclsh_ and _expect_ needed by the tools, * _xmllint_ for validating configurations, # Clone the Goa repository: ! git clone https://github.com/genodelabs/goa.git The following steps refer to the directory of the clone as '<goa-dir>'. # Enable your shell to locate the 'goa' tool by either * Creating a symbolic link in one of your shell's binary-search locations (e.g., if you use a 'bin/' directory in your home directory, issue 'ln -s <goa-dir>/bin/goa ~/bin/'), or alternatively * Add '<goa-dir>/bin/>' to your 'PATH' environment variable, e.g., (replace '<goa-dir>' with the absolute path of your clone): ! export PATH=$PATH:<goa-dir>/bin # Optionally, enable bash completion by adding the following line to your '~/.bashrc' file: ! source <goa-dir>/share/bash-completion/goa Usage ----- Once installed, obtain further information about the use of Goa via the 'goa help' command. Step-by-step tutorials ---------------------- The following article series gets you started with using Goa. :Streamlining the development of Genode applications: [https://genodians.org/nfeske/2019-11-25-goa] :Sticking together a little Unix (part 1): [https://genodians.org/nfeske/2019-12-13-goa-unix-bash] :Sticking together a little Unix (part 2): [https://genodians.org/nfeske/2019-12-19-goa-unix-terminal] :Sticking together a little Unix (part 3): [https://genodians.org/nfeske/2019-12-22-goa-unix-pipes] :Publishing packages: [https://genodians.org/nfeske/2020-01-16-goa-publish]
As of now, Goa is limited to the x86_64 architecture. Since Sculpt 21.10 has become routinely be available for AARCH64 in addition to x86_64, it would be nice being able to target both architectures, or even cross compile.
Goa unexpectedly requires an API version when exporting a regular src project.
This can be observed when removing the goa/examples/hello_make/version file and issuing the following commands:
goa/examples/hello_make$ rm version
goa/examples/hello_make$ touch /tmp/license
goa/examples/hello_make$ goa export --depot-user x --depot-overwrite --license /tmp/license
Error: version for archive x/api/hello_make undefined
Create a 'version' file in your project directory, or
define 'set version(x/api/hello_make) <version>' in your .goarc file.
In this case, Goa should only require version for x/src/hello_make, not for an API archive.
The packages are referenced in share/goa/lib/run/linux.tcl
but missing in share/goa/goarc
.
Cargo.toml
in the source directoryWhile using Goa for porting a variant of binutils, I encountered the issue that AR
and AS
where not explicitly passed as environment variables by Goa. So the project's Makefile.in was falling back to a default.
I'd suggest extending the assignments at https://github.com/genodelabs/goa/blob/master/share/goa/lib/build/autoconf.tcl#L40 to cover AS
and AR
as well.
When changing the version of an api archive used by a Goa project, the implemented symbols might change as well. However, the abi stubs stored in the var/abi/ directory are not regenerated automatically. Currently, we therefore need to remove the var/ directories when switching between Goa releases or when using --versions-from-genode-dir
.
Goa follows the scheme that (most) configuration variables can be overridden by command line arguments. However, for some configuration variables the corresponding command-line argument is missing. These are the variables with no equivalent command-line argument:
According to the documentation, only a single pkg shall be exported if --pkg <pkgname>
is provided to goa export
. Yet, this is only implemented for goa publish
.
By default, Goa creates a default depot at var/depot in the project directory. This is done "magically" when Goa tries downloading/extracting any depot archive and encounters a non-existing var/depot directory. However, when calling goa export
, the var/depot folder gets created but will not be populated with the default depot users. The same effect occurs for goa add-depot-user
.
By introducing a depot-dir
action similar to the build-dir
action, we could make the goa run
, goa export
and goa add-depot-user
dependent on this. Moreover, the action could be used separately to initialise a certain directory as a depot.
@nfeske I noticed that Goa only populates the depot directory with the default depot if the depot_dir
configuration has not been customised. I'm wondering whether we could change this. When working with several projects, the --depot-dir
argument or depot_dir
config variable can be used to use a shared depot directory. However, there is no way to conveniently set this up.
This allows access to ROM modules that are provided from outside of the project. The feature is activated as in the code snippet below:
<requires>
<rom label="name_of_rom"/>
</requires>
To provide the ROM a lx_fs
and a fs_rom
component are started. A file with the name name_of_rom
is expected in the sub directory roms
.
When trying to build the lvgl_support library without strict warnings disabled, I'm getting the following build error:
/home/johannes/repos/genode/depot/genodelabs/api/gui_session/2023-03-12/include/gui_session/gui_session.h: In instantiation of ‘Gui::Session::Command::Command(ARGS) [with ARGS = Gui::Session::Command::To_front]’:
/home/johannes/repos/genode/depot/genodelabs/api/gui_session/2023-03-12/include/gui_session/client.h:103:12: required from ‘void Gui::Session_client::enqueue(ARGS ...) [with CMD = Gui::Session::Command::To_front; ARGS = {Genode::Handle<Gui::View>, Genode::Handle<Gui::View>}]’
/home/johannes/repos/goa-pkgs/lvgl/lvgl_support/src/lvgl_support.cc:172:69: required from here
/home/johannes/repos/genode/depot/genodelabs/api/gui_session/2023-03-12/include/gui_session/gui_session.h:139:29: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
139 | reinterpret_cast<ARGS &>(nop) = args;
| ^~~
/home/johannes/repos/genode/depot/genodelabs/api/gui_session/2023-03-12/include/gui_session/gui_session.h:139:34: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
139 | reinterpret_cast<ARGS &>(nop) = args;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
cc1plus: all warnings being treated as errors
Apparently, this is due to a missing -fno-strict-aliasing
which is present in the Genode build system but not in Goa.
I tried to extend the unix shell a bit and noticed that some api archives are reported missing as soon as I run goa publish
. I believe the api archives are automatically downloaded, however, if goa is unable to pull a certain archive once it seems to forget some dependencies.
I'm able to reproduce this issue with the following modifications on the intro/unix_3rd
in goa-playground:
var
directory to start on a clean slatealex-ab/src/which
to pkg/unix_3rd/archives
set version(alex-ab/src/which) 2021-02-22
to .goarc
goa add-depot-user alex-ab --depot-url https://depot.genode.org --pubkey-file /path/to/alex-ab/pubkey
goa run
- this will fail on downloading https://depot.genode.org/alex-ab/api/libc/2021-02-22.tar.xzgoa run
again - this will succeedgoa publish
will report a bunch of missing api archivesSo far, goa has only added include paths to library headers to the CPPFLAGS variable, breaking any explicit C compilation rules only using CFLAGS because of unavailable (standard) library headers.
To support explicit use of (only) CFLAGS, add include directories to the CFLAGS variable as well.
Currently, Goa uses base-linux for its run stage. In principle, the tool is able to support other targets like qemu or real hardware. Since run/linux.tcl
serves as a blueprint for any future run-target implementation, it should undergo some restructuring. Moreover, we should add a bit of documentation about the limitations of the linux target and how custom targets can be added.
Updating a user's depot index with packages published with Goa for a new Sculpt release is still a manual task. With an increasing number of Goa packages to be managed and published, this becomes a significant effort. Adding support for publishing an index to a user's depot, will automate this task and eliminate the need to switch to the depot tools within the Genode repo.
Here is a collection of ideas regarding this:
goa export-index
and goa publish-index
.depot_user
and depot_overwrite
variables.index
file in the working directory that is similar to the exported/published file but lacks username and version information.goa publish-index
should call goa publish
for not-yet-published archives.Related to this, a command goa bump-versions <version>
could be helpful for setting the versions of all Goa projects present in the subdirectories.
We want to add a simple testing facility to Goa in order to identify regressions quickly. The current suggestion is that there will be a script in <goa>/share/goa/lib
that executes everything in the examples
directory and compares the actual output to the expected output (using expect
). Any objections or suggestions are welcome.
To simplify testing of projects that use multiple NIC interfaces we should start a nic_router
and anic_drv
for each required <nic label=""/>
.
This comes from the discussion on the mailing list see: https://lists.genode.org/pipermail/users/2023-June/008754.html
I have a test scenario, which uses the component vbox5
.
Running it on top of the linux
-runtime is not supported.
I would like to add a qemu
-runtime so the scenario can be tested like in the genode
-repository:
Would you like to implement this or if not, are there any valuable hints you might give me concerning the implementation?
Kind regards,
Lars
The libSDL2 and friends (like Mesa) are readily available at genode-world. However, to use the library from a Goa project, using CMake in particular, a few pieces are missing in Goa. Fortunately, @ssumpf happens to have those puzzle pieces in a branch https://github.com/ssumpf/goa/commits/doom3.
@ssumpf I would very much appreciate you upstreaming the libSDL2 support! Maybe even along with a simple example of using it with CMake? I'm asking because the latest version of Hatari relies on SDL2 and uses CMake.
I just noticed that the lib.so files generated by Goa are all ~17MB large. This also holds for the cmake_library example. I found that the -Wl,-Ttext=0x1000000
argument is passed as ldflags to cmake also for shared libraries, which inflates the resulting file by 16MB.
Currently, the files and directories specified by an 'api' file are searched in the build directory of the library. However, it is not very common for 3rd-party libraries to copy their header files to the build directory. In order to export an api archive with Goa, one thus needs to patch the CMakeLists.txt. I'm therefore thinking about adding support for searching the specified files and directories not only in the build directory but also in the src directory.
@trimpim Does this sound reasonable to you?
Now that Sculpt 21.10 is released, it is time to update Goa so that software can be built for the new Sculpt system.
I understand that goa import
supports populating the raw folder. However, this prevents keeping a manually managed raw/ folder in conjunction with the import feature.
If a Goa project has an empty used_apis
file, the exported src recipe is missing the used_apis
file. However, Genode's build system requires this file to be present when importing the recipe via import_from_depot
.
At the moment, the versions of components come from share/goa/goarc
. So at the moment, versions that my software is compiled with are changed when I update the tool. This is undesirable.
Ideally, Goa would write the current dependencies into a lock file when dependencies have been resolved. This lock file can then be checked into version control.
Good examples of this workflow are the Haskell Stack, or Cargo lock files, npm lock files, etc.
The linux_nic_drv previously had a nic_server-mode that was used to implement the NIC requirements. Since removal of this mode from the driver, the support for NIC sessions is broken in Goa. As the NIC driver solely operates as an Uplink client, we need to additionally instantiate a NIC router that we use as a NIC server for the runtime.
These are fixes for some problems I encountered while building OpenSC with GOA. The autoconf support had to be improved at some points and support for using the PCSC-lite API was required.
m-stein/goa@b9432d2 autoconf: improve appliance of verbosity settings
m-stein/goa@f231a3b autoconf: define CPP and CXXCPP for configure
m-stein/goa@9ea55a6 quirks: support using pcsc-lite api
m-stein/goa@7024fa9 autoconf: autoreconf if there's only configure.ac
Since its release, Goa has been augmented with support for additional build systems (autoconf, qmake, vivado) that require some sort of user documentation.
Due to ABI changes in FreeBSD 12, the libc crate used by rustc currently tries to link against versioned symbols of some library functions to preserve compatability with FreeBSD 11.
To make the standard library distributed with the x86_64-unknown-freebsd
target work, statically link versioned stat() and fstat() symbols from the the compat-libc
api package.
The cmake_step2
example relied on Github's Subversion support, which will be discontinued. In order to keep the example working, add a new repository type, sparse-git
, and convert the example to use a sparse checkout.
Analog to the extended validation of elements in the <parent-provides>
-node in runtime
-files worked on in #26 , I would like to extend the validation to throw an error on sessions from the <requires>
-node, which can not be provided.
For example the following vm
session reference would not trigger a validation error in the runtime
-file:
<requires>
<vm/>
<timer/>
</requires>
This would speed up debugging for users a lot and especially help eliminating first frustrations with the tooling.
Would you like to implement this or if not, are there any valuable hints you might give me concerning the implementation?
Kind regards,
Lars
Is Virtualbox supported for testing with goa run
?
I've mostly just tried to run the example from: https://github.com/genodelabs/genode/tree/master/repos/ports/recipes/pkg/vbox5
I currently use the following pkg/vbox_test/runtime
:
<runtime ram="4300M" caps="2500" binary="init">
<requires>
<file_system label="vm"/>
<file_system label="shared"/>
<vm/>
<timer/>
<gui/>
<nic/>
<rom label="capslock"/>
<report label="shape"/>
<report label="clipboard"/>
<rom label="clipboard"/>
<rm/>
<rtc/>
<rom label="usb_devices"/>
<usb/>
</requires>
<config verbose="yes">
<parent-provides>
<service name="ROM"/>
<service name="PD"/>
<service name="RM"/>
<service name="CPU"/>
<service name="LOG"/>
<service name="VM"/>
<service name="Gui"/>
<service name="Timer"/>
<service name="Rtc"/>
<service name="Report"/>
<service name="File_system"/>
<service name="Usb"/>
<service name="Nic"/>
</parent-provides>
<default-route> <any-service> <parent/> <any-child/> </any-service> </default-route>
<default caps="100"/>
<start name="vfs" caps="400">
<resource name="RAM" quantum="10M"/>
<provides><service name="File_system"/></provides>
<config>
<vfs>
<ram/>
<rom name="machine.vbox"/>
</vfs>
<default-policy root="/" writeable="yes"/>
</config>
<route>
<any-service> <parent/> <any-child/> </any-service>
</route>
</start>
<start name="vbox" caps="2000">
<binary name="virtualbox5" />
<resource name="RAM" quantum="4G"/>
<exit propagate="yes"/>
<config vbox_file="machine.vbox" xhci="yes" vm_name="linux" capslock="ROM">
<vfs>
<dir name="dev">
<log/>
<rtc/>
</dir>
<dir name="shared"> <fs label="shared" writeable="yes"/> </dir>
<fs writeable="yes"/>
</vfs>
<libc stdout="/dev/log" stderr="/dev/log" rtc="/dev/rtc"/>
</config>
<route>
<service name="Audio_out"> <parent/> </service>
<service name="File_system" label="shared"> <parent label="shared"/> </service>
<service name="File_system"> <child name="vfs"/> </service>
<service name="ROM" label="usb_devices"> <parent label="usb_devices"/> </service>
<service name="ROM" label="capslock"> <parent label="capslock"/> </service>
<service name="ROM" label="platform_info">
<parent label="platform_info"/> </service>
<service name="Nic"> <parent/> </service>
<service name="Report" label="shape"> <parent label="shape"/> </service>
<service name="ROM" label="clipboard"> <parent label="clipboard"/> </service>
<service name="Report" label="clipboard"> <parent label="clipboard"/> </service>
<service name="Gui"> <parent label=""/> </service>
<any-service> <parent/> </any-service>
</route>
</start>
</config>
<content>
<rom label="ld.lib.so"/>
<rom label="init"/>
<rom label="virtualbox5"/>
<rom label="libc.lib.so"/>
<rom label="libm.lib.so"/>
<rom label="libiconv.lib.so"/>
<rom label="libyuv.lib.so"/>
<rom label="qemu-usb.lib.so"/>
<rom label="stdcxx.lib.so"/>
<rom label="vfs.lib.so"/>
<rom label="jpeg.lib.so"/>
<rom label="platform_info"/>
<rom label="machine.vbox"/>
<rom label="vfs"/>
</content>
</runtime>
The following pkg/vbox_test/archives
:
TLC62CsRraK4hdw6/src/vbox5
TLC62CsRraK4hdw6/src/libc
TLC62CsRraK4hdw6/src/init
TLC62CsRraK4hdw6/src/posix
TLC62CsRraK4hdw6/src/zlib
TLC62CsRraK4hdw6/src/libiconv
TLC62CsRraK4hdw6/src/libyuv
TLC62CsRraK4hdw6/src/stdcxx
TLC62CsRraK4hdw6/src/vfs
TLC62CsRraK4hdw6/src/jpeg
The following raw/platform_info
(since I read it's not provided on Linux? taken from https://github.com/genodelabs/genode/blob/master/repos/os/run/vmm_x86.run#L75 :
<platform_info>
<kernel name="nova"/>
<acpi revision="2" rsdt="0x2ffe20c5"/>
<affinity-space width="2" height="2"/>
<boot/>
<hardware>
<features svm="true" vmx="true"/>
<tsc invariant="false" freq_khz="2555589"/>
<cpus>
<cpu id="0" package="0" core="0" thread="0" family="0x10" model="0x2" stepping="0x3" platform="0x0" patch="0x0"/>
</cpus>
</hardware>
</platform_info>
And a machine.vbox
from https://github.com/genodelabs/genode/blob/master/repos/gems/recipes/raw/download_coreplus/machine.vbox
The following is output by goa run
:
Genode sculpt-22.04 <local changes>
17592186044415 MiB RAM and 18997 caps assigned to init
[init -> nic_drv] MAC address 02:00:00:00:00:01
[init -> nic_drv] using tap device "tap0"
[init -> nic_drv] Error: could not configure /dev/net/tun: no virtual network emulation
terminate called after throwing an instance of 'Genode::Exception'
[init -> vbox_test] parent provides
[init -> vbox_test] service "ROM"
[init -> vbox_test] service "PD"
[init -> vbox_test] service "RM"
[init -> vbox_test] service "CPU"
[init -> vbox_test] service "LOG"
[init -> vbox_test] service "VM"
[init -> vbox_test] service "Gui"
[init -> vbox_test] service "Timer"
[init -> vbox_test] service "Rtc"
[init -> vbox_test] service "Report"
[init -> vbox_test] service "File_system"
[init -> vbox_test] service "Usb"
[init -> vbox_test] service "Nic"
[init -> vbox_test] child "vfs"
[init -> vbox_test] RAM quota: 9992K
[init -> vbox_test] cap quota: 366
[init -> vbox_test] ELF binary: vfs
[init -> vbox_test] priority: 0
[init -> vbox_test] provides service File_system
[init -> vbox_test] child "vbox"
[init -> vbox_test] RAM quota: 4194056K
[init -> vbox_test] cap quota: 1966
[init -> vbox_test] ELF binary: virtualbox5
[init -> vbox_test] priority: 0
[init -> vbox_test] child "vfs" announces service "File_system"
error: XDG_RUNTIME_DIR is invalid or not set in the environment.
Authorization required, but no authorization protocol specified
[init -> drivers -> fb_sdl] Error: fb_sdl works on X11 only. Your SDL backend is offscreen.
terminate called after throwing an instance of 'Sdl_videodriver_not_supported'
[init -> vbox_test -> vbox] Log created: 2023-01-21T15:24:00.002000000Z
[init -> vbox_test -> vbox] main Executable: /virtualbox
[init -> vbox_test -> vbox] Error: SystemPropertiesWrap : Cannot determine default Guest Additions ISO location. Most likely they are not available
[init -> vbox_test] child "vbox" requests resources: cap_quota=4
And then it's just stuck there.
Please either document that virtualization is not possible or provide an example in this repository.
Thanks for the great work!
I recently noticed a bunch of errors when running a test project using qt5_textedit. These errors are caused by unrouteable services such as clipboard ROM/Report. These can be easily resolved emulated by a report_rom component. Furthermore, by adding a black-hole component, we could easily satisfy service requirements such as Audio_in, Audio_out and state reports.
If a Goa project only has a src directory but no artifacts file, goa prints the following error:
$ goa build
Error: ... does not look like a goa project
However, in an initial project phase - e.g., while crafting an import file and trying to get some 3rd-party code compiled -
there exists no artifacts file yet because one hasn't got hold of any binaries yet (hence, no clue what to write into the artifacts file). It would be nice of Goa to account for this situation by giving an instructive message whenever a src directory is present but no artifacts file, like:
"The project has a 'src' directory but lacks an 'artifacts' file. You may start with an empty file."
Currently, there is no reference in Goa where to find documentation about the structure of 'import' files. Since Goa uses the port tools from the Genode repo, the guide at genode.org can be used as documentation. However, this is not immediately clear for newcomers.
Instead, I'd rather integrate documentation in terms of a goa help import
sub-command into Goa, similar to goa help artifacts
. This way Goa becomes more self-contained in terms of documentation.
This idea is spinning in my head for a while now, and I'm admittedly not quite sure about it.
My original intention behind the .goarc
files was to allow for user-specific settings, following the usual pattern (like .bashrc
).
However, by now, I have seen several instances where .goarc
files are checked in into Git repositories (goa-project repos), which in my opinion, makes things a little bit too obscure because those files are, well, hidden. So should we better change the naming to Goa
(with an upper-case first letter like in Makefile
)? Or should both variants be allowed?
I'm particularly worried since .goarc files can contain arbitrary Tcl code. So when a user clones a Git repo and invokes goa
commands, this code get executed. Hence, it would be fair to bring this code to the user's attention by not hiding it.
While building some library and binary archives on different computers we noticed, that the published archives contain symbolic links to absolute file paths from the machine creating the archive.
I see two ways to fix this.
extract_api_artifacts
do not create a symbolic link but copy the files.
If there are no objections, I would create and test a patch that implements the second solution.
@jschlatow noticed in 60ef785#r117846730 that there is an artifact that exports the API
to the wrong location.
The code in question looks like an artifact of integrating the possibility to build libraries. It is not needed to successfully build our test scenario, which uses libraries.
I know this isn't a huge deal, but when goa is run via a relative symbolic link, e.g. "../goa/bin/goa", it doesn't properly find its install directory. Either it should resolve relative links properly, or there should be a notice in the readme that goa cannot be added to the shell path via a relative symbolic link.
Commit nfeske@60ef785 breaks qmake (ldlibs_exe missing) and Genode dynamic binaries (Component::construct
gets garbage collected by ld
, leading to broken binaries).
Analog to the extended validation of elements in the <parent-provides>
-node in runtime
-files worked on in #26 , I would like to extend the validation to throw an error on files from the <content>
-node, which can not be provided.
For example the following inexistent
file reference would not trigger a validation error in the runtime
-file:
<content>
<rom label="existent">
<rom label="inexistent">
</content>
This would speed up debugging for users a lot and especially help eliminating first frustrations with the tooling.
Would you like to implement this or if not, are there any valuable hints you might give me concerning the implementation?
Kind regards,
Lars
Since the genodelabs depot archives needed for goa run
are only published for the official Sculpt releases, testing/developing for other Genode versions is quite cumbersome. Having a way to configure that the required depot archives shall be taken from another user than genodelabs will greatly simplify this.
Note this configuration is different from the depot_user
variable, which defines the depot user for exporting/publishing. The new variable should only affect the archives loaded by goa run
. The archives referenced by any Goa package will remain unaffected by this.
Regarding the naming, I am open to suggestions. Maybe something along the line of run_as
or run_as_depot_user
.
When Sculpt deploys a pkg, it looks up the declaration of ROM content of the new subsystem in all pkg dependencies (see genodelabs/genode#3981). Goa should do the same to avoid the duplication of <content>
nodes when customizing an existing pkg.
E.g., when crafting a new runtime based on https://github.com/nfeske/goa-playground/tree/master/unix, e.g., to supplement only make.tar
, one can create a new goa project as follows:
pkg/unix_with_make/archives
:
nfeske/pkg/unix
nfeske/src/gnumake
pkg/unix_with_make/runtime
:
<runtime ram="170M" caps="5000" binary="init" config="unix.config">
<requires>
...
</requires>
<content>
<rom label="make.tar"/>
</content>
</runtime>
raw/vfs.config
:
config ld_verbose="yes">
<vfs>
<tar name="make.tar" />
...
</vfs>
...
</config>
Note that one does not need to supply a copy of unix.config
because the ROM from the unix project can be used as is. Also the <content>
does not need to list any content that is already declared in unix's runtime.
When trying this with the current version of Goa, Goa complains that the unix.config
is missing. Since I have used Goa like that in the past (e.g., https://github.com/nfeske/goa-playground/tree/git/test/unix-git), I guess that the current behavior is a regression that was introduced with adding the sanity checks for the plausibility of the config.
The examples can be found at https://github.com/ssumpf/goa-projects and are used to demonstrate Genode's Phone SDK.
First of, I'm very excited to see the recent love and well deserved attention being given to Goa. It will help nurture a rich ecosystem in the Genode world.
I was happy to see the article on Rust support being improved in the Genode 23.08 release, such that the official Rust toolchains could be used unmodified with the Goa tool. This is an incredible improvement in user experience. Thanks for working on this @ssumpf, @atopia and others of the Genode team!
I followed the instructions of the https://genodians.org/atopia/2023-08-24-enabling-the-upstream-rust-toolchain article using the 23.08 release of Genode.
The example code worked perfectly using the official Rust toolchain and the Genode 21.05 toolchain:
$ rustup toolchain install stable-x86_64-unknown-linux-gnu
$ rustup target add x86_64-unknown-freebsd
However, I noticed that the Goa tool is yet to be updated to use the 23.05 Genode toolchain (as it still relies on the Genode 21.05 toolchain):
goa/share/goa/lib/command_line.tcl
Line 440 in 96c82ea
So, to test it out, I switched from 21.05 to 23.05.
if {![info exists cross_dev_prefix]} {
switch $arch {
- arm_v8a { set cross_dev_prefix "/usr/local/genode/tool/21.05/bin/genode-aarch64-" }
- x86_64 { set cross_dev_prefix "/usr/local/genode/tool/21.05/bin/genode-x86-" }
+ arm_v8a { set cross_dev_prefix "/usr/local/genode/tool/23.05/bin/genode-aarch64-" }
+ x86_64 { set cross_dev_prefix "/usr/local/genode/tool/23.05/bin/genode-x86-" }
default { exit_with_error "tool-chain prefix is not defined" }
}
}
The hello world Rust example gave a link time error when compiled with the 23.05 toolchain, whereas the example compiled and ran perfectly when using the Genode 21.05 toolchain.
$ time goa run -C examples/hello_rust --versions-from-genode-dir genode --depot-dir genode/depot
Error output:
error: linking with `/usr/local/genode/tool/23.05/bin/genode-x86-gcc` failed: exit status: 1
|
= note: LC_ALL="C" VSLANG="1033" "/usr/local/genode/tool/23.05/bin/genode-x86-gcc" "-m64" "/tmp/rustclRzYhg/symbols.o" ... "-L.../goa/examples/hello_rust/var/abi/x86_64" "-l:libc.lib.so" "-l:libm.lib.so" "-l:posix.lib.so" "-Wl,--dynamic-linker=ld.lib.so" "-Wl,--dynamic-list=.../goa/share/goa/ld/genode_dyn.dl" "-T" ".../goa/share/goa/ld/genode_dyn.ld" "-l:ld.lib.so"
= note: .../rustlib/x86_64-unknown-freebsd/lib/libstd-b98b1995c63467c0.rlib(std-b98b1995c63467c0.std.c179cdd7ef8a0dd5-cgu.0.rcgu.o): in function `std::sys::unix::fs::File::file_attr':
/rustc/1b198b3a196442e14fb06978166ab46a4618d131/library/std/src/sys/unix/fs.rs:1086: undefined reference to `fstat@FBSD_1.0'
.../rustlib/x86_64-unknown-freebsd/lib/libstd-b98b1995c63467c0.rlib(std-b98b1995c63467c0.std.c179cdd7ef8a0dd5-cgu.0.rcgu.o): in function `std::sys::unix::fs::stat::{{closure}}':
/rustc/1b198b3a196442e14fb06978166ab46a4618d131/library/std/src/sys/unix/fs.rs:1619: undefined reference to `stat@FBSD_1.0'
/usr/local/genode/tool/23.05/bin/../lib/gcc/x86_64-pc-elf/12.3.0/../../../../x86_64-pc-elf/bin/ld: /rustc/1b198b3a196442e14fb06978166ab46a4618d131/library/std/src/sys/unix/fs.rs:1619: undefined reference to `stat@FBSD_1.0'
/usr/local/genode/tool/23.05/bin/../lib/gcc/x86_64-pc-elf/12.3.0/../../../../x86_64-pc-elf/bin/ld: /rustc/1b198b3a196442e14fb06978166ab46a4618d131/library/std/src/sys/unix/fs.rs:1619: undefined reference to `stat@FBSD_1.0'
.../rustlib/x86_64-unknown-freebsd/lib/libstd-b98b1995c63467c0.rlib(std-b98b1995c63467c0.std.c179cdd7ef8a0dd5-cgu.0.rcgu.o): in function `std::sys::unix::fs::File::file_attr':
/rustc/1b198b3a196442e14fb06978166ab46a4618d131/library/std/src/sys/unix/fs.rs:1086: undefined reference to `fstat@FBSD_1.0'
collect2: error: ld returned 1 exit status
= note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `hello-rust` (bin "hello-rust") due to previous error
[hello_rust] Error: build via cargo failed: child process exited abnormally
Let me know if any additional information is required to troubleshoot this issue.
$ uname -a
Linux x1 6.4.11-arch2-1 #1 SMP PREEMPT_DYNAMIC Sat, 19 Aug 2023 15:38:34 +0000 x86_64 GNU/Linux
Cheerful summer wishes from Sweden,
Robin
Is Virtualbox supported for testing with goa run
?
I've mostly just tried to run the example from: https://github.com/genodelabs/genode/tree/master/repos/ports/recipes/pkg/vbox5
I currently use the following pkg/vbox_test/runtime
:
<runtime ram="4300M" caps="2500" binary="init">
<requires>
<file_system label="vm"/>
<file_system label="shared"/>
<vm/>
<timer/>
<gui/>
<nic/>
<rom label="capslock"/>
<report label="shape"/>
<report label="clipboard"/>
<rom label="clipboard"/>
<rm/>
<rtc/>
<rom label="usb_devices"/>
<usb/>
</requires>
<config verbose="yes">
<parent-provides>
<service name="ROM"/>
<service name="PD"/>
<service name="RM"/>
<service name="CPU"/>
<service name="LOG"/>
<service name="VM"/>
<service name="Gui"/>
<service name="Timer"/>
<service name="Rtc"/>
<service name="Report"/>
<service name="File_system"/>
<service name="Usb"/>
<service name="Nic"/>
</parent-provides>
<default-route> <any-service> <parent/> <any-child/> </any-service> </default-route>
<default caps="100"/>
<start name="vfs" caps="400">
<resource name="RAM" quantum="10M"/>
<provides><service name="File_system"/></provides>
<config>
<vfs>
<ram/>
<rom name="machine.vbox"/>
</vfs>
<default-policy root="/" writeable="yes"/>
</config>
<route>
<any-service> <parent/> <any-child/> </any-service>
</route>
</start>
<start name="vbox" caps="2000">
<binary name="virtualbox5" />
<resource name="RAM" quantum="4G"/>
<exit propagate="yes"/>
<config vbox_file="machine.vbox" xhci="yes" vm_name="linux" capslock="ROM">
<vfs>
<dir name="dev">
<log/>
<rtc/>
</dir>
<dir name="shared"> <fs label="shared" writeable="yes"/> </dir>
<fs writeable="yes"/>
</vfs>
<libc stdout="/dev/log" stderr="/dev/log" rtc="/dev/rtc"/>
</config>
<route>
<service name="Audio_out"> <parent/> </service>
<service name="File_system" label="shared"> <parent label="shared"/> </service>
<service name="File_system"> <child name="vfs"/> </service>
<service name="ROM" label="usb_devices"> <parent label="usb_devices"/> </service>
<service name="ROM" label="capslock"> <parent label="capslock"/> </service>
<service name="ROM" label="platform_info">
<parent label="platform_info"/> </service>
<service name="Nic"> <parent/> </service>
<service name="Report" label="shape"> <parent label="shape"/> </service>
<service name="ROM" label="clipboard"> <parent label="clipboard"/> </service>
<service name="Report" label="clipboard"> <parent label="clipboard"/> </service>
<service name="Gui"> <parent label=""/> </service>
<any-service> <parent/> </any-service>
</route>
</start>
</config>
<content>
<rom label="ld.lib.so"/>
<rom label="init"/>
<rom label="virtualbox5"/>
<rom label="libc.lib.so"/>
<rom label="libm.lib.so"/>
<rom label="libiconv.lib.so"/>
<rom label="libyuv.lib.so"/>
<rom label="qemu-usb.lib.so"/>
<rom label="stdcxx.lib.so"/>
<rom label="vfs.lib.so"/>
<rom label="jpeg.lib.so"/>
<rom label="platform_info"/>
<rom label="machine.vbox"/>
<rom label="vfs"/>
</content>
</runtime>
The following pkg/vbox_test/archives
:
TLC62CsRraK4hdw6/src/vbox5
TLC62CsRraK4hdw6/src/libc
TLC62CsRraK4hdw6/src/init
TLC62CsRraK4hdw6/src/posix
TLC62CsRraK4hdw6/src/zlib
TLC62CsRraK4hdw6/src/libiconv
TLC62CsRraK4hdw6/src/libyuv
TLC62CsRraK4hdw6/src/stdcxx
TLC62CsRraK4hdw6/src/vfs
TLC62CsRraK4hdw6/src/jpeg
The following raw/platform_info
(since I read it's not provided on Linux? taken from https://github.com/genodelabs/genode/blob/master/repos/os/run/vmm_x86.run#L75 :
<platform_info>
<kernel name="nova"/>
<acpi revision="2" rsdt="0x2ffe20c5"/>
<affinity-space width="2" height="2"/>
<boot/>
<hardware>
<features svm="true" vmx="true"/>
<tsc invariant="false" freq_khz="2555589"/>
<cpus>
<cpu id="0" package="0" core="0" thread="0" family="0x10" model="0x2" stepping="0x3" platform="0x0" patch="0x0"/>
</cpus>
</hardware>
</platform_info>
And a machine.vbox
from https://github.com/genodelabs/genode/blob/master/repos/gems/recipes/raw/download_coreplus/machine.vbox
The following is output by goa run
:
Genode sculpt-22.04 <local changes>
17592186044415 MiB RAM and 18997 caps assigned to init
[init -> nic_drv] MAC address 02:00:00:00:00:01
[init -> nic_drv] using tap device "tap0"
[init -> nic_drv] Error: could not configure /dev/net/tun: no virtual network emulation
terminate called after throwing an instance of 'Genode::Exception'
[init -> vbox_test] parent provides
[init -> vbox_test] service "ROM"
[init -> vbox_test] service "PD"
[init -> vbox_test] service "RM"
[init -> vbox_test] service "CPU"
[init -> vbox_test] service "LOG"
[init -> vbox_test] service "VM"
[init -> vbox_test] service "Gui"
[init -> vbox_test] service "Timer"
[init -> vbox_test] service "Rtc"
[init -> vbox_test] service "Report"
[init -> vbox_test] service "File_system"
[init -> vbox_test] service "Usb"
[init -> vbox_test] service "Nic"
[init -> vbox_test] child "vfs"
[init -> vbox_test] RAM quota: 9992K
[init -> vbox_test] cap quota: 366
[init -> vbox_test] ELF binary: vfs
[init -> vbox_test] priority: 0
[init -> vbox_test] provides service File_system
[init -> vbox_test] child "vbox"
[init -> vbox_test] RAM quota: 4194056K
[init -> vbox_test] cap quota: 1966
[init -> vbox_test] ELF binary: virtualbox5
[init -> vbox_test] priority: 0
[init -> vbox_test] child "vfs" announces service "File_system"
error: XDG_RUNTIME_DIR is invalid or not set in the environment.
Authorization required, but no authorization protocol specified
[init -> drivers -> fb_sdl] Error: fb_sdl works on X11 only. Your SDL backend is offscreen.
terminate called after throwing an instance of 'Sdl_videodriver_not_supported'
[init -> vbox_test -> vbox] Log created: 2023-01-21T15:24:00.002000000Z
[init -> vbox_test -> vbox] main Executable: /virtualbox
[init -> vbox_test -> vbox] Error: SystemPropertiesWrap : Cannot determine default Guest Additions ISO location. Most likely they are not available
[init -> vbox_test] child "vbox" requests resources: cap_quota=4
And then it's just stuck there.
Please either document that virtualization is not possible or provide an example in this repository.
Thanks for the great work!
Currently, goa run
will download and prepare dependencies before starting Genode. In some situations it might be desirable to separate these steps. For example: while testing my Goa projects I'd like to interact with the spawned Genode process via expect
and time out if a certain output is not produced. This does not work if I call goa run
because downloading the dependencies might take an arbitrarily long time.
One idea would be to create a new prepare-run
subcommand that does everything that run
currently does but does not call run_genode
. run
would then depend on this and only call run_genode
. Is there a specific reason why it does not currently work this way?
Hello Norman,
Today I've installed your goa program, I've followed every step you described in your installation file, but every time I try to execute it, (goa build), it appears the following error:
/usr/local/genode/tool/current/bin/genode-x86-gcc: 1: /usr/local/genode/tool/current/bin/genode-x86-gcc: Syntax error: "(" unexpected
while executing
"exec $cross_dev_prefix\gcc -print-libgcc-file-name"
(file "/home/sgrjlhgwl/goa/share/goa/lib/flags.tcl" line 14)
invoked from within
"source [file join $tool_dir lib flags.tcl]"
invoked from within
"if {$perform(build-dir)} {
#
# Prepare depot content for the used APIs and generate ABI stubs
#
# This must happen before assembling the compile ..."
(file "/home/sgrjlhgwl/goa/bin/goa" line 495)
Honestly I don't know how I can fix it, any helps?
Thanks.
PS: I've installed everything needed: from Genode toolchain to expect tool (& the other your wrote)
:)
When trying to run goa build
for the first time, it tries to copy the default depot into the source directory. If goa was installed in a read-only location, the resulting files will also be read-only. This issue came up during packaging of goa.
% find .
./src
./src/Makefile
./src/hello.cc
./used_apis
% goa build
download nfeske/api/base/2019-11-23.tar.xz
download nfeske/api/base/2019-11-23.tar.xz.sig
Error: could not verify 'nfeske/api/base/2019-11-23', signature does not match
public key '/home/julian/src/own/genode-hello/var/depot/nfeske/pubkey'
Error: failed to download the following depot archives:
nfeske/api/base/2019-11-23
The reason is that gpg cannot create the key in the read-only directories goa created:
gpg: can't create '/home/julian/src/own/genode-hello/var/depot/nfeske/pubkey.dearmored': Permission denied
% ls -lh var/depot
total 8.0K
dr-xr-xr-x 2 julian users 4.0K Jan 1 1970 genodelabs
dr-xr-xr-x 2 julian users 4.0K Jan 1 1970 nfeske
This can be hotfixed by running chmod -R +w var
.
The real fix seems to be fixing this Tcl code:
proc prepare_depot_with_archives { archive_list } {
global depot_dir public_dir tool_dir jobs
if {![depot_exists]} {
if {[customized_variable depot_dir]} {
exit_with_error "cannot install APIs because" \
"there is no depot at $depot_dir" }
# create default depot local to the project at var/depot/
file mkdir var
file copy [file join $tool_dir default_depot] $depot_dir # <- This needs to be fixed
}
Unfortunately, my Tcl skills are zero. :(
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.