I am considering removing util::actual_path() and just making all the utilities assume that the current working directory that we should operate in is the current working directory of the binary. The original point of util::actual_path() was to support the testing framework, but we won't need it for that once #24 is merged.

I can see it still being useful in some cases where mesabox is used as a library within another program as it allows operating in different directories without modifying the arguments or changing the host program's current working directory (which could be problematic if that program were to be multi-threaded). At the same time, removing util::actual_path() makes the utilities simpler and reduces the number of allocations we need to perform.

The only issue at the moment is getting the output. Running the program and waiting for it should work fine.

We should be doing:

1. tilde expansion (this, based on the value of $HOME), parameter expansion, command substitution, and arithmetic expansion
2. field splitting
3. pathname expansion (i.e. globbing)
4. quote removal (not sure if this needs to be handled explicitly or if Word::SingleQuote and Word::DoubleQuote are good enough

Firstly, getty shouldn't just panic on failure (it should instead return Results like the other utilities). Because of how getty affects the state of the system, I think the execve() call may be alright though.

getty should not be hard-coded to call /bin/ion. It should instead take an argument specifying what TTY to take control of and what program to start on said TTY (the current values can serve as defaults, however). We may want to implement gettytab or something similar as well.

Given echo \a in dash, a\n will be printed. In our shell, echo \a produces <alert (a beep)>\n.

I would vastly prefer to use something like LALRPOP or pest instead of nom, but right now neither support arbitrary bytes as input. There is this issue for LALRPOP and this one for pest that deal with that problem, but I'm not sure if it's fine to just say that invalid UTF-8 paths can't be given directly as arguments for now (they'd still work if a glob was given where the expanded part was invalid UTF-8). Switching to either LALRPOP or pest should help improve the error messages and make it simpler to fix some of the current parsing issues.

As a side note, this is actually the same issue that prevented me from just using pest in the first place.

I'm probably gonna use this issue to document differences that I've noticed between our shell and other shells, so it'll be edited/added to over time.

shift in function

Input

fn() { shift; echo hello; }
fn

Results

mesabox

illegal number "1"
hello

bash

hello

zsh

fn:shift: shift count must be <= $#
hello

dash

Notice that dash does not print hello at all. I didn't notice this in the specification, so I'm not actually sure if this is correct behavior (other built-ins like trap let the function continue normally).

dash: 1: shift: can't shift that many

test_chmod::test_chmod_ugoa seems to fail sometimes on Rust 1.26 for some reason. Additionally, the sleep tests can easily fail if the system takes too long to wake from sleep (or if threads take too long to spawn).

On Linux, we should try to use splice() rather than the standard read/write stuff for operations involving pipes. This mostly affects sh.

Because we use tar-rs, we will have this issue at the moment.

The above has been fixed, so it will no longer be an issue when using an up-to-date version of tar-rs.

We should see if there's anywhere where the new NonZero types would work/benefit us.

Hi,
I'm trying to contribute some code but get stuck in error handling.

One example I found for returning error is below. However, the progname and err are both None, which seems not very helpful.

Another example I found is below. This line is quite difficult to understand for new contributors.

Could you give some guildeline for error handling in mesabox?

We need to decide what documentation system to use. I assume we want support for man pages, which means we can either roll our own generator (like in ripgrep, which, last I checked, first generates an AsciiDoc file in build.rs) or use something like Sphinx, which is used in uutils/coreutils.

Not sure how commonly these are used, but they are needed for POSIX-compliance.

We should be doing:

1. tilde expansion, parameter expansion, command substitution, and arithmetic expansion
2. field splitting (this, based on the value of $IFS)
3. pathname expansion (i.e. globbing)
4. quote removal (not sure if this needs to be handled explicitly or if Word::SingleQuote and Word::DoubleQuote are good enough

We can probably remove the 1.26.2 targets on ARM, as if they work on x86_64 they'll likely work on ARM (we don't do much architecture-specific stuff at the moment). We can also probably combine the formatting target with the normal stable Linux target.

I don't know why the first tests are always blocking for over 60 seconds. However, after the blocking issue, all testcases can be run without any problem.

test gnu::arch::test_x86_64 ... test gnu::arch::test_x86_64 has been running for over 60 seconds
test gnu::base32::test_decode ... test gnu::base32::test_decode has been running for over 60 seconds

Switching from tarpaulin to kcov did not fix the issue as the problem (needing to disable ASLR) is inherent to the approach used by both tools. It seems like we either need to relax the Docker permissions or wait until rustc can emit coverage information (or use something other than Drone for CI).

For the time being I will continue to allow the build to succeed when the run-kcov.sh script fails.

I found that sleep is not consistent with the POSIX standard (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sleep.html). It's more like a GNU implementation.

Current test code coverage is pretty low (about 13.56%): https://codecov.io/gh/mesalock-linux/mesabox/tree/master

Our target coverage is around 80%.

Can we replace our self-maintained test utility with assert_cmd? (https://github.com/assert-rs/assert_cmd). Seems that assert_cmd provides most of functions we have, but I'm not sure what is still missing.

I didn't find init in the POSIX standard (http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html).

How about treat init itself as a "type"/"feature" (similar with "POSIX", "Netowrking", etc)?.

Also, I am trying to compile mesabox in OS X (x86_64-apple-darwin). init (in "posix" feature) and getty (in "loginutils" feature) are not compatible with OSX (which are not designed and supposed to work in OSX). Therefore, for people who want to use it in OSX can simply exclude "init" and "loginutils" feature.

Anyway, I don't think supporting OSX is the first priority.

It would be nice to be able to call the utilities from C code. We might limit the user to providing raw file descriptors/raw handles as there's no way to have generic bindings. We could also expose RawObject and have the user give those to the interface so we don't need to worry about the platform differences.

We should look at using cbindgen, but if our interface is small enough it might just be easier to maintain it by hand.

Due to parsing directly on the input, we have a few issues where stuff like forx iny; do echo $x; done will functions the same as for x in y; do echo $x; done. Tokenizing the input should fix this issue.

This should be very easy. All we need to do is check the exit codes returned from execute().

Given that I am planning on using all the mesabox utilities as built-ins for the shell, I thought I would use this issue to categorize any issues that come up.

One issue that I don't know how to solve until OOM can panic (instead of aborting) is that if one of the built-in utilities encounters an OOM error the entire shell will abort. Clearly, this is not intended behavior, so we should be careful with how much memory we allocate until OOM can panic (in which case we will just need to catch_unwind() the built-ins or something to make sure they don't crash the entire shell.

Everything works well except for files/functions under shell. I don't why the integration test cases for shell will not emit hit counts. Therefore, the test coverage for posix/sh is not accurate.

Because all the utilities are included dynamically using macros, cargo fmt only formats the stuff like src/setup.rs and src/util.rs.

I also think we should configure the formatting, as sometimes the output looks worse/is harder to read than the input with the current settings.

For some reason the parser accepts echo hi | echo hello (echo hi; cat) | cat, which causes hello to be displayed on stdout. Based on testing, it seems to stop parsing when it sees the (echo hi; cat) part.

It's pretty sketchy right now.

The current error messages are not consistent and nested together. We should have a better error message format.

mesabox head:

$ ./target/debug/mesabox head -c - tests/fixtures/head/lorem_ipsum.txt
head: error: Invalid value for '--bytes <NUMBER>': '-' is not a number or is too large

mesabox cat:

$ ./target/debug/mesabox cat fdsfs
cat: fdsfs: No such file or directory (os error 2)
cat: encountered 1 error(s)

Some examples for reference:

GNU coreutils:

$ head -c - tests/fixtures/head/lorem_ipsum.txt
head: invalid number of bytes: ‘’

BSD (OS X):

$ head -c - tests/fixtures/head/lorem_ipsum.txt
head: illegal byte count -- -

BusyBox:

$ ./busybox head -c - busybox_unstripped.out
head: invalid number ''

Cargo:

$ cargo build --release sss
error: Found argument 'sss' which wasn't expected, or isn't valid in this context

USAGE:
    cargo build --release

For more information try --help

We cannot use the globwalk crate as it does not support globbing for parent directories (i.e. paths like ../../* won't work). We will probably need to roll our own directory walker, either by using globset or modifying glob to support OsStrings.

The relevant sections are mostly in src/posix/sh/command.rs. We can't just use ChildStdin/ChildStdout/ChildStderr because we need to support other file descriptors as well, so we need to create the pipes manually using nix.

EDIT: dunno why I wrote the above, but ChildStdout should be passed into the next command as its stdin.

Not exactly sure what's going on, but I imagine a file descriptor isn't being dropped somewhere. This causes the cat within the subshell to hang (it might be the cat after the subshell, not sure yet).