rimerosolutions / entrusted Goto Github PK

Background

Since day one, a bunch of shell scripts has been used to create releases artifacts.

• It helped get started quickly, as I had already "old past experiences" building installers and software packages
• There was no need to understand GitHub actions or other older alternatives (Travis CI, CircleCI, etc.)
• It was fairly trivial for me to setup a build virtual machine (~25 GB of RAM allocated, ~100 GB of disk space)

Request

Migrate to GitHub actions for scalability and convenience.

• This saves local computing resources and network bandwidth
• It becomes more of a "professional" setting, without myself as a bottleneck

Overall changes

There are couple of things that need to happen. There were previous attempts to migrate to GitHub actions, so technical details and considerations are well understood.

• Workflow for building and pushing the "sandbox" container image
• Workflow for building and publishing "builder" container images
• Revisit binaries stripping process, it seems to have been missed for Linux aarch64/arm64
• Workflow for building main binaries (Windows, Linux and Mac OS)
• Trivy container vulnerability scans
• Workflow for building Live CD
• Refactor existing shell scripts to work with GitHub actions, but still be usable locally

Background

As much as possible everything is statically linked to avoid external dependencies, but that mostly possible with command-line applications.

Objectives

• Determine if there is some value added for having an AppImage
  • There are couple of libraries excluded from the GUI artifact which is an AppImage
  • Musl is not supported
• Determine if the DEB and RPM packages should include explicitly dependencies such as libXcb, libXfixes, etc.
• Revisit wayland support on Linux (mostly testing efforts)
  • Are separate binaries needed?
  • Does it work seamlessly on systems without wayland or x11 libraries available?

Background

A custom seccomp profile is utilized to reduce allowed system calls (syscalls) in the sandbox container image. This has proven to be challenging since the implementation, as any issue with it is "opaque" without much error details.

Expected Behavior

Conversions succeed under Fedora for the standard test dataset. Fedora is a good test generally because when it works, it's almost guaranteed to also work under Windows and Mac OS. Furthermore, Fedora has selinux enabled by default and that can occasionally result into slightly different problems that are not seen on other operating systems (file permissions issues while moving files from/to container volumes, etc.).

Actual Behavior

Tested conversions under a live CD of Fedora (amd64) and it failed abruptly without lots of information.
This is likely due to the custom seccomp profile again and we need to find yet again missing syscalls.

Notes

• Need to not rebuild original sandbox container image, as it might result into different syscalls possibly missing in the seccomp profile.
• Need to always build a debug strace aware sandbox container image to avoid spending too much time troubleshooting
• Need to find a long term solution later for custom seccomp profiles, this is very annoying

Update/enhance the existing RELEASING file.

All conversions are tested with and without OCR (English only) using files in the test_data folder.

• Linux testing (docker and podman)
  • On amd64
    • With Docker
      • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
      • Test DEB artifact on Ubuntu (Desktop clients: CLI and GUI)
      • Test RPM artifact on Fedora (Desktop clients: CLI and GUI)
    • With Podman
      • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
      • Test DEB artifact on Ubuntu (Desktop clients: CLI and GUI)
      • Test RPM artifact on Fedora (Desktop clients: CLI and GUI)
  • On arm64 (Podman)
    • This is only tested with podman for the CLI, on a Live CD of Alpine Linux.
• Windows testing (amd64 only, Docker)
  • Test Windows EXE installer
  • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
• Mac OS testing (amd64 only, Docker Desktop)
  • Test DMG image
  • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
• Live CD testing
  • amd64 testing
  • arm64 testing

Background

The Entrusted binaries provided in releases only run on Glibc systems.

I built a new virtual machine (Void Linux x86_64-musl) to assess a potential migration from xorg to Wayland.

• I then realized that none of my Rust-based Desktop applications work, including Entrusted
• I typically do not use Musl on the Desktop, so I never noticed

Request

Provide binaries that will run on Musl systems such as Alpine Linux, Void Linux Musl, Gentoo Musl builds, etc.

Overall changes

• Add new build steps in GitHub Actions to build for Musl systems (aarch64 and amd64)
• Update Linux CI/CD shell scripts for local machine builds
• Change Linux release artifacts naming (-glibc and -musl suffixes)

Notes:

• One solution is cloning the fltk-rs dependency at build time and do some search/replace magic (sed or perl)
• A better approach consists into providing prebuilt versions of FLTK-RS and fetch them by specifying the CFLTK_BUNDLE_URL environment variable to fetch remote artifacts

Background

There's a desire to transition from podman to nerdctl on the Live CD, because it provides graceful apparmor support for rootless containers. The Live CD ISO image size should not get out of control though.

• There was a successful preliminary investigation for enabling apparmor on the Live CD #18
• Sadly, the ISO image size grew too much (800 MB to 1.4 GB)
  • We added some new apparmor DEB packages
  • We added few nerdctl related binaries
  • We're using fuse-overlayfs as "containerd snapshotter"

Objectives

• Understand at a high level the nerdctl directory layout and typical storage size

• Understand at a high level nerdctl snapshotters and relevant disk storage requirements

• Evaluate opportunities to compress the container image "BLOB size" on the Live CD

Background

Even though there are plans to migrate to GitHub Actions, compile times are annoying with Rust.

• It seems that every year there are several articles about improving Rust compile times, I'm convinced that the trend will continue...
  • https://nnethercote.github.io/2022/02/25/how-to-speed-up-the-rust-compiler-in-2022.html
  • https://fasterthanli.me/articles/why-is-my-rust-build-so-slow
• I haven't seen anything interesting yet and I'm using a rather high end computer most of the time (but it's still not great)

Request

Reduce compile times both for local builds and later on GitHub Actions

Overall changes

• Drop software dependencies that involve too many macros
• Consider rewriting parts of libraries if only a tiny subset of their features is leveraged
• Avoid using generics?
• etc.

Background

axtloss found creative ways to package Entrusted 0.2.4 as Flakpak (based on the Live CD ISO image).

A lot has changed since 0.2.4, and there will be more improvements in 0.3.1

• Faster document processing
• Web interface visual improvements
• ISO image size optimization (~960 MB in 0.2.4, ~670 MB targeted for 0.3.1)
• Several security hardening options (kernel, services sandboxing, more secure configurations, etc.)
• gVisor implementation as container security platform on the Live CD in 0.3.1
• etc.

Request

Sync with axtloss to ensure that 0.3.1 is available as a Flatpak.

Overall changes

None anticipated so far, but there are 2 major updates in the Live CD (boot loader, custom kernel).

Background

See #56

Problem

Dependencies are not explicit in RPM or DEB packages beyond glibc.

Note: In the upcoming version, there will be also be a download artifact for systems that use musl instead of libc (in the form of a tar.gz archive).

Approach

• Update package descriptors
• Update README.txt in Linux tar.gz artifacts to mention required shared libraries

Background

The Live CD is based on Debian stable. The build process generates ISO images for amd64 and aarch64

Problem

The build process is a bit too complex to both maintain and support:

• There are temporary files and artifacts left upon build failure (see #65)
• It's difficult and too time consuming to provide user support in case of build failures:
  • Distribution specific issues and potential Unix/Linux user resourcefulness
  • Container runtime user customization concerns
  • User environment specific customization
  • Package dependencies name mappings to a specific Linux distribution
  • etc.

Proposed approach

Try containerizing most of the build process similarly to Lima Alpine ISO build or other "Live CD kits".

Main Challenge

We need a "ready-to-go" folder structure with the entire contents of $HOME/.local/share/containers in the Live CD.

• We need the container sandbox image contents
• We need all the remaining auxiliary Podman files and folders populated under $HOME/.local/share/containers

This is about requesting new builds (via GitHub Actions) with latest changes prior testing more 0.3.1.

Background

One of the key design principles for the Web interface was always to keep a familiar look and feel with the Desktop interface.

Not having "tabs" in the Web UI is an inconsistency. It has also other side effects as new conversion settings get introduced.

• Space feels wasted on the page
• UI differences with the Desktop client become more visible over time

Request

Implement tab containers for "Settings" and "Convert" (defaulting to "Convert") just like in the Desktop interface.

Overall changes

Old interface without tabs

New interface with tabs

Convert tab

This is the default selected tab, just like for the Desktop client.

Settings tab

The user can optionally customize defaults prior conversions in the "Settings" tab: there will likely be more settings over time.

Background

gVisor is a container security platform. It could be nice to leverage it on the Live CD for additional security hardening.

This has been tested few times a while back and the performance was really really bad with Podman under Linux ...

• One "rather small file" without gVisor, under Podman (Linux): roughly 30 seconds
• The same file with gVisor, under Podman (Linux), close to 28 minutes if I recall correctly
  • This might be beneficial for some people depending on their threat model
  • To become acceptable, this needs to be a bit faster in my opinion (but maybe not?)

Objectives

• Review gVisor performance and publish non-scientific benchmarks for now
  • a version of the Void Linux PDF handbook as test file with ~136 pages
  • A smaller file, with just few pages (potential average email attachment)
• Check if gVisor works in tandem with the seccomp security profile added in 0.3.0
• Assess whether or not gVisor could be an optional setting in the Web interface of the Live CD

Background

By default, conversion results are stored in the same folder as the original input files. There's also a button for customizing the output location on a per file basis (save icons in the "file conversion queue").

• When files are converted from multiple input folders, you either need to remember the input folder or mouse over a given input file (hint displaying the full file location)
• Sticking with the multiple input folders use-case, unless you had existing tab in your file explorer, then it might be a bit inconvenient. Either way, it feels right to have an option to open the result directly (on-demand) from Entrusted

Request

It would be nice to be able to click directly on a link to open the PDF result directly from Entrusted.

• This would match closely the look and feel of the Web interface ("Download trusted PDF" and "Logs" hyperlinks)
• This makes the "Settings" option "Open resulting PDF with" a safe fallback?.
  • The preferred PDF viewer for Entrusted is also likely the default PDF viewer for a given Desktop environment (Linux, Mac OS or Windows)
  • When the detection of the PDF viewer fails, it will be likely on Linux for non XDG compliant environments: minimal or heavily customized environments. We'll just inform the user that we cannot open the file with the default application...

Overall changes

Update settings tab

In the settings tab, the "Open resulting PDF with" option is removed.

Add new hyperlink in convert tab

In the file conversion queue, a new "Open" link is displayed upon successful processing. Please note that the existing "Logs" button is also streamlined to become more of an "hyperlink" widget.

Final result

The user interface seems less cluttered with buttons and is more streamlined with the Web interface look and feel.

Background

Few manual tests are run prior each release. In 0.3.0 a simple functional test was added in entrusted_client (Cucumber for Rust)

• The functional test covers the happy path for existing test data
• The functional test is still run manually

Request

Automate the existing functional test via GitHub actions.

Overall changes

Add new GitHub Actions workflow (amd64 only, excluding Windows due to execution errors on GitHub)

• Linux (Docker as container solution)
• Mac OS (Lima as container solution)

Background

There were previous efforts to reduce the ISO image size (#35). This can be taken a little bit further with a custom kernel for which the installation size is smaller than the stock Debian Linux image (linux-image-amd64 or linux-image-arm64).

There were few known challenges while attempting to build a tiny but functional kernel image

• Issues fully powering off the machine, likely due to disabled settings (APM, ACPI or something else)
• EFI boot problems with a hanged process in QEMU: "Existing boot services..."

The linux-image-cloud-* Debian packages are not suitable because they remove network drivers and we're stuck with only the loopback interface. We need a better middle ground for Entrusted.

Request

Use a custom kernel to further reduce the Live CD ISO image size.

Overall changes

• Code modifications in CI/CD pipeline scripts
• Kernel configuration files for amd64 and arm64 (ideally a single file for both) : This will be delivered via an external project kernel-deblive-smallserver, outside this repository

Background

In the early days of Entrusted, its file processing times was comparable to Dangerzone.

• As Entrusted became much faster, parallel conversions were then considered not essential
  • There is the programming language difference and how code is written to achieve certain goals directly
  • There are other design choices: 1 step container pass instead of 2 container passes (document to pixels, pixels to PDF)
• Faster feels always better, especially if the cost is not too high to pay (memory or CPU usage)

Objectives

Keep in mind that this would also require switching from "row indexes" to "document ids" when submitting file for processing... It is not technically difficult to implement a quick and dirty initial implementation (Fast mapping of IDs to widget for progress notifications, etc.).

• Assess if the software should decide how many conversions can run in parallel based on system specifications
• Assess if the user should be able to decide how to allocate computer resources, possibly with some guardrails
• Assess if parallel conversions would create few anticipated potential problems
  • File mounts disappearing and/or data corruption in the container solution storage
  • Weird container image crashes with little to non-existent feedback that can be captured easily
    • Segmentation faults
    • Container process killed for no apparent valid reason
    • etc.
  • Plethora of problems with differences per container solution and/or version
    • "podman 2.x does this and podman 3.x does that.."
    • "however in Docker, xywxs happens and the container crashes"

For every conversion I do, a cmd window opens (I believe to start the Docker container) and does not close, and the entrusted-gui begins Not Responding. I can then just close the cmd window and it opens a new one and continues with the page separation. At the end of the conversion, it says it has Failed and the Logs end with "Access is denied. (os error 5)." Even with this supposed failure, the PDF does come out the other end successfully.

I'm on Windows 10, using Docker (which I'm not very familiar with). I would like to say I appreciate entrusted because it handles very large PDFs and dangerzone does not seem to.

Background

For a while, application components have been built with rust 1.64.0. As improvements and security issues get incorporated, it's important to stay reasonably up to date.

Request

Build projects with rust 1.67.0.

Overall changes

• Update GitHub Actions workflows (livecd, main, supportimages)
• Update CI/CD shell scripts
• Update and publish builder container images

Make couple of updates to existing documentation

• New screenshots
• Obsolete statements
• TODO.org file
• CHANGELOG.org file
• etc.

Background

Application bundles are sandboxed on Mac OS, which means that they can't typically access most resources outside the application itself. Via entitlements, applications can get additional permissions.

Problem

Launching external programs such as Docker, Podman or Lima doesn't seem to work anymore via a wrapper script. Essentially the previous trick (script run through login shell) doesn't seem to work for launching applications that are outside the application bundle.

Error message

Even if an application is within the user PATH, due to the Apple sandbox, Entrusted is not be able to access it.

Converting /Users/me/Downloads/testdata/sample-odp.odp
No container runtime executable found!
Please install Docker or Lima, and make sure that it's running.

Additional information

The error message displayed by the application is confusing from a user perspective.

me-iMac-Pro$ which lima
/Users/me/Tools/lima-0.16.0/bin/lima

Solution for now

The solution for now is to only support Docker Desktop. The assumption is that we're dealing with a standard installation with Docker.app dragged to the /Applications folder.

• The docker application bundle is searched by its id (com.docker.docker)
• If the application bundle location is found, then we search for the docker command-line executable, relative to the resources folder of the application bundle.
• So far, this doesn't seem to result into permission issues or "executable not found" problems

Failed alternate solutioning attempts

• Leveraging NSTask doesn't help even with an absolute path to a given program. It also makes the code more complex.
• Leveraging NSWorkspace doesn't seem to allow low-level code (capture process output as it becomes available, etc.).

Once testing is complete and satisfactory

• Rebuild all release artifacts from GitHub Actions
• Perform quick testing
  • Live CD (arm64 and amd64)
  • Alpine arm64 (CLI only)
  • Fedora amd64 (Podman, CLI and GUI)
• Update documentation as needed
• Tag the current branch with the new release version.
• Create the release with the GitHub Web interface with the changelog

Problem statement

The program is unable to produce the final PDF result under Linux with Docker as container solution. The application fails with cryptic "permission denied errors".

Reproduction steps

This can be reproduce by following the testing instructions for arm64 under amd64, using "user-built dev snapshots" ("0.2.6 dev").

• Run entrusted-cli with any supported file type such as a PDF document (this will fail)
• Copy the logged Docker command and remove the seccomp profile option (wrong syscalls on arm64)
• Run the amended underlying Docker command
• This fails near the end of conversions after "Saving PDF", with some "permission denied errors"

Impacted versions

This has been detected in the upcoming 0.2.6 development builds and seems Docker specific under Linux. Most of the Linux testing is happening only with Podman since the early days of this project.

Known workarounds

• Change directory permissions of /tmp/entrusted/safe to make it write-able for all users:
  mkdir -p /tmp/entrusted/safe && chmod -R a+rw /tmp/entrusted/safe
• Or run Docker with the --privileged flag which is not desirable

Environment details

Additional information

• This happens after the "Saving PDF" message and fails while copying the final PDF result to the shared volume.
• The volume mount permissions are mapped to the current Linux user running entrusted, this differs from the user running insider the container solution (user ID permissions mismatch).

Potential root cause and solution

The root cause appears to be directory ownership issues and how they get mapped inside Docker. The simplest solution might ensure that the temporary directory mapped to the container volume is write-able by everybody (libc::chmod).

Background

In the upcoming 0.3.1 release, the plan is to utilize a single boot manager (grub) instead of previously a combination of 2 boot managers (isolinux for BIOS and grub for UEFI). This has apparently introduced a bug, maybe after shell scripts polishing.

Problem

The Live CD can no longer boot in BIOS mode on amd64/x86_64 systems.

Expected behavior

The system boots successfully after the grub boot manager prompt.

Actual behavior

The boot process fails with grub complaining about a missing prefix variable not being found.

Welcome to GRUB!

../../grub-core/kern/dl.c:944:variable 'prefix' isn't setBacktrace (.text 0xa058 .data 0x15e2c)

Background

As of now, there are 3 officially supported container solutions:

• Docker (Linux, Mac OS, Windows)
• Podman (Linux, possibly WSL???): This doesn't include podman-remote
• Lima (Mac OS): This is limited to the default instance for now

There's also hidden direct support for [nerdctl](https://github.com/containerd/nerdctl

• It's not advertised as it was never properly tested.
• The goal was to support Rancher Desktop but then it became obvious that few settings would need to be exposed to users (default mounted volumes vs. user customization for tinkerers, etc.).

Request

It is desirable to easily supporting additional container solutions such as Rancher Desktop and possibly much more. This request might not be worth the trouble:

• Supporting too many options creates lots of testing efforts
• Generic support will possibly complicate the code while making few parts more fragile or "user-error" prone

Overall changes

• New user settings for generic container solution command-line invocation?
  • Settings tab in the Desktop client
  • Configuration file changes
• Runtime preparation changes prior starting document processing

Background

As of now, the Linux GUI artifact is an AppImage file. This is achieved with linuxdeploy. The original goal was creating a fully self-contained binary, but this was never done properly (time constraints and current AppImage knowledge).

Request

Do not create an AppImage file for the GUI, as most of the dependencies are not currently embedded anyway.

• It complicates the build
• Fuse related issues
  • It may create dependency problems in certain distributions (FUSE related dependencies: i.e., fuse2 vs fuse3 vs fuse).
  • If the fuse libraries are not resolved, the program will silently failed to start without notice, but that's a general problem.
• It doesn't "naturally" support musl .

Anticipated changes

• This will impact local build scripts and the GitHub actions workflow.
• README files included in zip or tar release artifacts should be updated accordingly
• fuse should be removed as a dependency from any rpm spec file or deb control file.

Is it possible to build the iso only for Linux environment (as opposed to building also for Windows, etc.)?
If so, how to accomplish that?

Problem statement

Starting with release 0.2.4 (UI responsiveness "improvement"), a regression bug was introduced.
This translates into intermittent conversion failures because the PDF result might not be available anymore on container image volumes (process exits for a conversion and then the next conversion starts).

Reproduction steps

Please note that the issue seems to happen on Mac OS and Windows more often than under Linux (tiling window manager testing).

1. Use Mac OS (if possible) to ensure higher likelihood of reproducing the bug
2. Convert documents in batch couple of times, maybe 10 to 15 documents
3. You should observe some intermittent failures for few documents, without the full logs (i.e. summary of time spent at the end, log messages that appear to be truncated "Starting document conversion")

Impacted versions

0.2.4 and above, constrained to the graphical Desktop interface.

Known workarounds

Re-process individually documents that failed to process.

Environment details

This has been observed under Mac OS (recent versions), but it also happens on other operating systems such as Linux and probably Windows too.

Additional information

Below is a screenshot

Potential root cause and solution

• This seems to be introduced by concurrency handling that results into race conditions (entrusted_client module).
• A potential fix is to review the handling of UI updates and make them close SwingWorker in Java. That would ensure that all UI updates (and associated state modifications) are happening in the main UI thread.

From entrusted/ci_cd/live_cd, running the build.sh script gives this error:

Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:xxxxxxxxxxxxxxa02d4eb6bb": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/gshadow: invalid argument): exit status 1
+ retVal=125
+ '[' 125 '!=' 0 ']'
+ echo 'Could not build entrusted-cli and entrusted-webserver'
Could not build entrusted-cli and entrusted-webserver

Building from entrusted/ci_cd gives:

ERRO[0000] cannot find UID/GID for user $USER: no subuid ranges found for user $USER in /etc/subuid - check rootless mode in man pages. 
WARN[0000] Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user 
Trying to pull docker.io/uycyjnzgntrn/rust-windows:1.67.0...
Getting image source signatures
Copying blob 7139ad4c9de2 done  
Error: writing blob: adding layer with blob "sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcf9702c6ae68b8c": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/gshadow: invalid argument): exit status 1
WARN[0271] Failed to add pause process to systemd sandbox cgroup: Process org.freedesktop.systemd1 exited with status 1 
+ retVal=125
+ '[' 125 -ne 0 ']'
+ echo 'Failure to build Windows binaries'
Failure to build Windows binaries
+ exit 1
+ retVal=1
+ '[' 1 -ne 0 ']'
+ echo 'Windows build failure'
Windows build failure
+ exit 1

This is on branch main .
Linux x86_64

Any ideas how to debug?

Background

Since 0.3.1, releases artifacts are assemble via the GitHub infrastructure (GitHub Actions), instead of using a local Ubuntu virtual machine. Please note that few pre-existing shell scripts are also reused in GitHub actions.

Problems

Existing shell scripts were built with an emphasis on supporting multiple architectures quickly (x86_64 and arm64)

• It can take lots of effort to support people experiencing local builds challenges for multiple reasons
• The way that scripts were initially assembled (and rationale behind it) can be frustrating when build errors arise
• The fact that local builds are only tested under an Ubuntu virtual machine; Supporting all kinds of users with their Linux distributions isn't possible
• Some configurations/decisions might require specific Linux or tooling knowledge
• Some other things just "suck" and didn't get refactored yet

Tasks

On top of items below, it would be nice to come closer to "reproducible builds" (containerized builds only help with parts of that).

• Review existing shell scripts
  • Evaluate opportunities to simplify logic
  • Review error handling and retries
  • Evaluate opportunities to make builds more flexible
  • Move scripting logic inside containers as much as possible
• Review container builder images default programs
  • Evaluate opportunities to trim container size
  • Remove duplicate or unrequired commands (symlinks, etc.)

The current ISO CD release has a 30 second grub timeout at launch, having an optional ISO release that has a small or 0 timeout value so that it launches straight away would be very useful for when packaging it as an application. This is an issue with the flatpak being developed for this app: https://github.com/axtloss/flatpaks/tree/main/com.github.rimerosolutions.entrusted

Background

There was a discussion about Flatpak support and how to approach it going forward ( #42). This issue will actually implement the envisioned solution.

Request

• Simpler Flatpak support without embedding a virtual machine and a Web server
• Ability to easily keep the Flatpak version up to date for new releases

Overall changes

• Update the CI/CD pipeline (if needed)
• Discuss packaging and process invocation considerations (how to bundle a subset of tesseract and libreoffice???)
• Look at package publishing on Flathub
• Maybe more things to add (TODO update this)

References

• #42
• #43
• Flatpak main page
• Flathub portal

2 main problems

• Just too many permission issues with GitHub Actions
• For local builds, need to also check for permission issues... Need to verify the musl builds

Background

The release 0.3.0 of Entrusted runs a 5.x kernel for the Live CD. In development builds, a custom kernel has been introduced via the kernel-deblive-smallserver project but the version is 6.1.8.

Kernel 6.1 releases are now flagged as long term kernel releases and 6.1.8 is no longer the latest available version.

Request

When 0.3.1 is released, ensure that it ships with the latest available kernel 6.1.x version.

Overall changes

• In the kernel-deblive-smallserver project, generate Debian packages for the latest Kernel 6.1.x version
• In this project, update CI/CD scripts responsible for the Live CD creation (version variable updates)

Background

The current Live CD ISO image is roughly 800 MB. As functionality gets added, the ISO image size will grow.

For the Live CD, there's a correlation between the RAM allocated to the virtual machine and free space

• 75% of the RAM is allocated as free space on the Live CD (grub boot parameters: ramdisk-size, etc.)
• With 1 GB of RAM allocated to a virtual machine, the application still runs pretty well. The intent is to keep RAM requirements as low as possible.

Request

Reduce the ISO image size as much as possible while keeping it functional. Pay attention to aarch64/arm64 support to ensure that the ISO image doesn't hang as noticed in early testing stages.

Overall changes

• Replace podman Debian related packages by podman-static (version upgrade and unneeded files). (changed as part of #36).
• Replace openssh by dropbear. (changed as part of #36).
• Use a single boot manager for both UEFI and BIOS (Grub)
• Apply zstd compression for the initramfs generation and the Live CD squashfs data

References

• mvallim/live-custom-ubuntu-from-scratch#47
• https://bugs.launchpad.net/ubuntu-cdimage/+bug/1886148/+index?ss=1
• https://fedoraproject.org/wiki/Changes/FedoraCloudHybridBoot
• https://bugs.launchpad.net/ubuntu/+source/casper/+bug/1895131
• https://forum.openwrt.org/t/openwrt-19-07-7-for-vmware-esxi-arm-fling-rpi4-8gb/94814/6
• utmapp/UTM#4499

Background

Each time that the sandbox container image changes (different distribution or release update), the custom seccomp profile might break. The sandbox container base OS was changed from Debian buster to Debian bookworm (latest current stable Debian version).

Problem

The conversions of Office documents crashes unexpectedly on arm64/aarch64, likely due to missing required syscalls.

Expected behavior

The conversion succeeds for all supported documents on arm64, including Office documents.

Actual behavior

The conversion of Office documents terminates abruptly, with the usual vague error messages.

This is the output of the underlying entrusted-cli command invocation. podman is invoked directly to get more insights about the failure. It seems that a missing syscall is preventing LibreOffice from copying files around?

localhost:~$ /usr/bin/podman run --rm --network none --cap-drop all --userns kee
p-id --security-opt no-new-privileges --security-opt seccomp=/tmp/entrusted/secc
omp-entrusted-profile-0.3.1.json -v /home/entrusted/entrusted/test_data/sample-d
oc.doc:/tmp/input_file:Z -v /tmp/entrusted/safe:/safezone:Z -e ENTRUSTED_LANGID=
en docker.io/uycyjnzgntrn/entrusted_container:0.3.1 /usr/local/bin/entrusted-con
tainer --visual-quality medium --log-format json
{"percent_complete":5,"data":"Converting to PDF using LibreOffice"}
LibreOffice 7.4 - Fatal Error: The application cannot be started.
User installation could not be completed.

Background

Entrusted has implemented its own file type detection business logic

• This is based on magic bytes and doesn't rely on file name extensions or whatsoever
• It is not a robust implementation, but an acceptable one based on the time spent and the "value delivered to the people"
• No "great" libraries were found at the time (some libmagic bindings, few pure Rust libraries, etc.)
  • infer was used in early versions of the applications
    • it was not reliable for Office documents detection (OpenDocument, Microsoft Office 365 or legacy versions).
    • See also bojand/infer#46
  • tree_magic_mini looked promissing but doesn't support apparently Office documents and zip files generally speaking. It was previously experimented with.
  • There are several other libraries that need to be re-assessed (relevance, capabilities)

The libraries that we care about need to perform mime detection based on file bytes rather than relying on file extensions.

Objectives

• Assess if we can use a more robust mime type detection library in the sandbox container image
• See if we can detect many more file types
  • In theory we can support at least all documents that LibreOffice can open
  • There are also few more image formats that could be supported

Background

There was a discussion about Flatpak support and how to approach it going forward ( #42). This issue will actually implement the envisioned solution.

Request

• Simpler Flatpak support without embedding a virtual machine and a Web server
• Ability to easily keep the Flatpak version up to date for new releases

Overall changes

• Refactor the main "Document Processing Component" (entrusted-container) to accept command-line arguments
• Allow the UI to invoke directly the main document processing component without going through podman or docker
• Refactor new gvisor support for alignment (Live CD specific feature)

References

• #42
• Flatpak main page
• Flathub portal

Background

Even though the UI toolkit (FLTK) supports Wayland, this is not enabled at build time.

Request

Enable Wayland support in UI builds as more people are using Wayland nowadays.

Overall changes

• Update the entrusted-client project to enable the use-wayland feature
• Update to fltk-rs version 1.3.29

During the migration of the CI/CD pipeline to GitHub actions (#16 ), a random error has been observed for the amd64 ISO image.

• Check if the podman version installed via GitHub actions is the same as what is built locally
• Check apt repository differences
• Compare podman versions (version on 0.3.0, version on local dev build, version on GitHub actions dev build)
• Compare kernel versions too

If hardened_malloc is going to be a problem, consider a replacement such as mimalloc (secure mode).

Random error with GitHub actions dev build (amd64 ISO image)
There have been few iterations of the build pipeline on GitHub but it never led to errors related to hardened_malloc

After observing the above issue, it was confirmed that disabling hardened_malloc makes the problem go away...

As I'm in the middle of implementing foundational support that will enable easier Flatpak packaging, there are couple of things to reconsider.

• Is a container solution absolutely required with everything that comes along with it?
• Wouldn't it be much better if native OS capabilities could be leveraged instead? How difficult is it to get it reasonably right?

Issues with a container solution

• Additional software installation for the user

  • Issues with 3rd party software outside of my control
  • Issues related to specific operating systems or specific versions of the container solution
  • Disk space management that users might not be aware of in the long run (older container images to remove)

• Higher computing resources footprint

  • Memory usage
    • There's the memory required by the application itself, plus the memory required by the container solution
    • If the default/custom memory settings of the container solution are too low the application will terminate abruptly...
  • Slower processing due to indirection/translation layers

• Lack of service management

  • entrusted doesn't attempt to start/stop or configure services running on the user's machine
  • For some people this is fine, for non tech-savvy users it might not be "understandable/acceptable"
  • If those services fail (i.e. Docker) for whatever reason (i.e. on Windows Hyper-V and VirtualBox conflicts or something among those lines), it can be frustrating for non-tech savvy users

Pros of a container solution

• Temporary files get automatically cleaned up inside the container
• There's some layer of isolation with the main system
• Things are technically simpler to manage and streamline from a software implementation perspective
  • There's not too much operating system specific code to consider
  • There's no need for a strong understanding of everything operating system specific sandbox approach
    • on Apple software there's no proper sandbox API, sandbox-exec is "deprecated" without any clear flexible/good alternative)
    • on Windows, a correct software implementation might not be trivial. I also barely use Windows, there's been years where I didn't use a Windows machine at all
    • on Linux, Apparmor and Seccomp profiles is reasonably straightforward, but what if users have SELinux enabled instead of AppArmor???

Background

gVisor is a container sandbox developed by Google that focuses on security, efficiency and ease of use.

gVisor has been preferred to a combination of seccomp (existing) and apparmor profiles:

• Additional software packages to install for apparmor support
• Boot loader changes for apparmor settings
• Switch from podman to nerdctl, as nerdctl supports apparmor with rootless containers
  • Increased Live CD boot time for preparing nerdctl rootless support
  • Less overall testing with nerdctl than podman
  • Additional considerations with rootfs option, for avoiding duplicate storage("containerd snapshotters")

Request

Enable gVisor on the Live CD as it provides a rather complete out of the box container security solution.

Overall changes

• CI/CD pipeline updates
  • Update the build scripts and GitHub Actions workflow to account for gVisor
  • Update the entrusted-webservice systemd service for gVisor support
    • Disable seccomp support via environment variables
    • Enable gVisor support via environment variables
• Update the entrusted-client component for gVisor support
  • Setup a tmpfs filesystem (5 MB) to account for LibreOffice setup, as it needs to create data in XDG_CONFIG_HOME...). This excludes images and PDF files processing.
  • Disable the userns flag when gVisor support is desired

Background

There are couple of hardening options implemented on the Live CD starting with 0.3.0

• Kernel settings (Grub boot settings and sysctl.conf)
• Web interface started with hardened malloc as memory allocator
• Minimal systemd unit sandboxing options that still allows the Web interface to run correctly
• container seccomp security profile, just like in the CLI and Desktop interfaces
• Few SSH hardening options

It would be nice to build on top of previous efforts with AppArmor, especially for the Live CD which is a fully controlled environment. A custom AppArmor security profile could be loaded during file processing.

Objectives

• Assess complexity of defining and loading an AppArmor profile
• Assess ISO image resulting size after adding AppArmor
• Assess CI/CD pipeline changes (most of the work is here)
• Assess additional required changes in the entrusted-client component
  • We need the ability to pass additional security-opt flags to the container solution
  • Those additional flags would need to be a configurable setting (environment variable maybe??)

All conversions are tested with and without OCR (English only) using files in the test_data folder.

• Linux testing (docker and podman)
  • On amd64
    • With Docker
      • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
      • Test DEB artifact on Ubuntu (Desktop clients: CLI and GUI)
      • Test RPM artifact on Fedora (Desktop clients: CLI and GUI)
    • With Podman
      • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
      • Test DEB artifact on Ubuntu (Desktop clients: CLI and GUI)
      • Test RPM artifact on Fedora (Desktop clients: CLI and GUI)
  • On arm64 (Podman)
    • #71
    • Test with podman for the CLI, on a Live CD of Alpine Linux (emulation with QEMU).
• Windows testing (amd64 only, Docker)
  • Test Windows EXE installer
  • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
• Mac OS testing (amd64 only, Docker Desktop)
  • Test DMG image
  • Test artifacts in Zip file (webserver, webclient, Desktop clients: CLI and GUI)
• Live CD testing
  • amd64 testing
    • UEFI boot
    • BIOS boot
  • arm64 testing (UEFI boot)

Just noticed some permission denied issues with Podman under Linux (Ubuntu), while trying to run the functional test of the application in entrusted_client (via cargo test). Per comments, this is mostly requires documentation changes, but minor code changes will be reverted too (for clarity and to avoid forgetting about few technical details).

• I do have a very heavily customized development environment, additional testing should also be performed on another virtual machine.
• It seems that the problem (if not environment related) could be in entrusted_container (to be confirmed).

Reproducing the issue

• Fetch the current main or develop branch
• Build the CLI or Desktop UI
• Attempt to convert a Microsoft document from the test_data folder with the CLI or Desktop GUI
• The conversion fails with a permission denied message.

....
{"percent_complete":91,"data":"Collecting PDF pages"}
{"percent_complete":92,"data":"Updating bookmarks and page numbering"}
{"percent_complete":93,"data":"Processing PDF structure"}
{"percent_complete":94,"data":"Updating PDF dictionary"}
{"percent_complete":95,"data":"Combining PDF objects"}
{"percent_complete":96,"data":"Compressing PDF"}
{"percent_complete":98,"data":"Saving PDF"}
{"percent_complete":98,"data":"Failed to copy file from /tmp/safe-output-compressed.pdf to /entrusted/safe-output-compressed.pdf"}
{"percent_complete":99,"data":"Conversion failed with reason: Permission denied (os error 13)"}
{"percent_complete":100,"data":"Elapsed time: 0 hours 0 minutes 4 seconds"}

Expected behavior

The programs completes successfully.

Current behavior

The programs fails while converting the PDF result to the mounted volume.

Additional details

• I successfully run a conversion with 0.3.0 to confirm that this is a new issue.
• I run the 0.3.1 development build of the GUI
• I updated the container image to point to the 0.3.0 version, which required minor updates in entrusted_client
• Everything was fine...

Background

Originally, Entrusted only supported x86_64/amd64 architectures. Then releases artifacts for arm64/aarch64 became available as it was possible to generate relevant binaries "relatively easily".

The current naming conventions for release artifacts are a bit unorthodox, usually the version number comes right after the "artifactname" portion.

• Pattern: artifactname-osname-architecture-version.fileextension
• Example: entrusted-linux-aarch64-0.3.0.deb

Request

Adopt the following naming pattern going forward:

• Pattern: artifactname-version-osname-architecture.fileextension
• Example: entrusted-0.3.0-linux-aarch64.deb

Overall changes

• Update CI/CD release scripts
• Update documentation

Background

It seems that there's another FLTK regression for drag and drop under Linux and maybe under other operating systems (Mac & Windows).

• This has happened before but for multiple file selections
• Need to add some debugging code to see if it's a similar issue for drag and drop
  • Everything works fine under Ubuntu with pcmanfm as file manager
  • The problem seems to be isolated to Fedora and the "Gnome files" file manager. There could be similar issues under different Linux distributions and/or with other Linux file managers.

Expected behavior

It's possible to drag and drop multiple files in the "file drop zone" (Red background).
This was only observed under Fedora with "Gnome files".

Actual behavior

Apparently drag and drop only works for a single file with fltk-rs 1.4.8.
This was only observed under Fedora with "Gnome files".

Notes

Need to investigate to get a reasonable understanding of what could be wrong...

• Maybe the problem has been around for a while and went unnoticed?
• Maybe it's an issue specific to Fedora and potentially selinux???

Cannot use fltk-rs 1.4.9+ yet, as its requirements have changed and there were failures with the windows build (from an Ubuntu VM). Need to investigate what needs to be done with the local CI/CD pipeline and the flow in GitHub Actions.

Background

As software libraries and dependencies get upgraded, maintaining a custom seccomp profile has become a maintenance nightmare.

• Generating a stripped (and working) seccomp profile is neither quick nor trivial
• Sandbox container image upgrades are risky and often require regenerating the custom seccomp profile
  • Changing the base OS from Debian to Alpine or another distribution is problematic
  • Upgrading to a newer OS version of the Debian-based container image can lead to few hours or days of testing

Request

Disable custom seccomp profile, as minimizing required syscalls is both tedious and mistakes crash conversions.

Overall changes

• Remove the custom seccomp profile json file
• Disable custom seccomp profile checks
• Remove references to environment variables for enabling a custom seccomp profile

Background

The current user interfaces are mostly based on previous Desktop interface implementations (many years ago) and the initial UI capabilities of Dangerzone.

Request

Can the UI be simpler and yet flexible, and without adding too many screens?

I do not know the right answer at this time, as I built Entrusted only for myself originally.

• Significant performance compared to Dangerzone, among other features
• Personal preferences and opinionated choices that need to be revisited

Overall changes

There's a feeling that it's potentially adding too many screens for a rather simple application.

• Is the current state that messy?
• Would the effort be really worth it?
• The UI itself would be a little bit different than the mock-ups to follow.

User settings

For user settings, most users will never need the "advanced" settings.

Conversions

There could be a different screen for selecting files and then another one during conversions, with the option to go back to add files?

• Prior conversions, only the bare minimum widgets required are displayed
  • The space reserved for the "overall progress" information disappears
  • In the file conversion queue, there's no need for the progress, status or messages column
• During conversions, the area to drop or select files is not visible
  • There is also no need to display "file selection" buttons
  • The "password/key" and "save to custom location" icons can disappear in the file conversion queue
  • When you go back to add new files to convert, you loose access to previous conversion results

Background

Until the 0.3.2 release, all release builds were performed on an Ubuntu virtual machine. This relies on the rust-linux-darwin-builder container image for cross-compiling Mac OS binaries from Linux.

That build process doesn't seem to work anymore (arm64 only it seems):

• Maybe there are additional considerations to account for with rust-linux-darwin-builder
• For now, this implies having a real Mac OS machine around to build all Mac OS binaries (for both x86_64 and arm64).

Problem

The local build pipeline for macos fails for the arm64 architecture.

How to reproduce

Run the scripts ./ci_cd/macos/build.sh from an Ubuntu virtual machine.

Additional information

It seems that few symbols are missing when compiling for arm64.