mesalock-linux / mesabox Goto Github PK
View Code? Open in Web Editor NEWA collection of core system utilities written in Rust for Unix-like systems (and now Windows)
License: BSD 3-Clause "New" or "Revised" License
A collection of core system utilities written in Rust for Unix-like systems (and now Windows)
License: BSD 3-Clause "New" or "Revised" License
We should be doing:
$IFS
)Word::SingleQuote
and Word::DoubleQuote
are good enoughThe only issue at the moment is getting the output. Running the program and waiting for it should work fine.
Given that I am planning on using all the mesabox
utilities as built-ins for the shell, I thought I would use this issue to categorize any issues that come up.
One issue that I don't know how to solve until OOM can panic (instead of aborting) is that if one of the built-in utilities encounters an OOM error the entire shell will abort. Clearly, this is not intended behavior, so we should be careful with how much memory we allocate until OOM can panic (in which case we will just need to catch_unwind()
the built-ins or something to make sure they don't crash the entire shell.
Can we replace our self-maintained test utility with assert_cmd
? (https://github.com/assert-rs/assert_cmd). Seems that assert_cmd
provides most of functions we have, but I'm not sure what is still missing.
For some reason the parser accepts echo hi | echo hello (echo hi; cat) | cat
, which causes hello
to be displayed on stdout
. Based on testing, it seems to stop parsing when it sees the (echo hi; cat)
part.
Current test code coverage is pretty low (about 13.56%): https://codecov.io/gh/mesalock-linux/mesabox/tree/master
Our target coverage is around 80%.
Switching from tarpaulin
to kcov
did not fix the issue as the problem (needing to disable ASLR) is inherent to the approach used by both tools. It seems like we either need to relax the Docker permissions or wait until rustc
can emit coverage information (or use something other than Drone for CI).
For the time being I will continue to allow the build to succeed when the run-kcov.sh
script fails.
Due to parsing directly on the input, we have a few issues where stuff like forx iny; do echo $x; done
will functions the same as for x in y; do echo $x; done
. Tokenizing the input should fix this issue.
Not exactly sure what's going on, but I imagine a file descriptor isn't being dropped somewhere. This causes the cat
within the subshell to hang (it might be the cat
after the subshell, not sure yet).
I found that sleep
is not consistent with the POSIX standard (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sleep.html). It's more like a GNU implementation.
Because all the utilities are included dynamically using macros, cargo fmt
only formats the stuff like src/setup.rs
and src/util.rs
.
I also think we should configure the formatting, as sometimes the output looks worse/is harder to read than the input with the current settings.
I'm probably gonna use this issue to document differences that I've noticed between our shell and other shells, so it'll be edited/added to over time.
shift
in functionfn() { shift; echo hello; }
fn
illegal number "1"
hello
hello
fn:shift: shift count must be <= $#
hello
Notice that dash
does not print hello
at all. I didn't notice this in the specification, so I'm not actually sure if this is correct behavior (other built-ins like trap
let the function continue normally).
dash: 1: shift: can't shift that many
The current error messages are not consistent and nested together. We should have a better error message format.
mesabox head:
$ ./target/debug/mesabox head -c - tests/fixtures/head/lorem_ipsum.txt
head: error: Invalid value for '--bytes <NUMBER>': '-' is not a number or is too large
mesabox cat:
$ ./target/debug/mesabox cat fdsfs
cat: fdsfs: No such file or directory (os error 2)
cat: encountered 1 error(s)
Some examples for reference:
GNU coreutils:
$ head -c - tests/fixtures/head/lorem_ipsum.txt
head: invalid number of bytes: ‘’
BSD (OS X):
$ head -c - tests/fixtures/head/lorem_ipsum.txt
head: illegal byte count -- -
BusyBox:
$ ./busybox head -c - busybox_unstripped.out
head: invalid number ''
Cargo:
$ cargo build --release sss
error: Found argument 'sss' which wasn't expected, or isn't valid in this context
USAGE:
cargo build --release
For more information try --help
I would vastly prefer to use something like LALRPOP or pest
instead of nom
, but right now neither support arbitrary bytes as input. There is this issue for LALRPOP and this one for pest
that deal with that problem, but I'm not sure if it's fine to just say that invalid UTF-8 paths can't be given directly as arguments for now (they'd still work if a glob was given where the expanded part was invalid UTF-8). Switching to either LALRPOP or pest
should help improve the error messages and make it simpler to fix some of the current parsing issues.
As a side note, this is actually the same issue that prevented me from just using pest
in the first place.
On Linux, we should try to use splice()
rather than the standard read/write stuff for operations involving pipes. This mostly affects sh
.
It would be nice to be able to call the utilities from C code. We might limit the user to providing raw file descriptors/raw handles as there's no way to have generic bindings. We could also expose RawObject
and have the user give those to the interface so we don't need to worry about the platform differences.
We should look at using cbindgen, but if our interface is small enough it might just be easier to maintain it by hand.
We need to decide what documentation system to use. I assume we want support for man pages, which means we can either roll our own generator (like in ripgrep
, which, last I checked, first generates an AsciiDoc file in build.rs
) or use something like Sphinx, which is used in uutils/coreutils.
Firstly, getty
shouldn't just panic on failure (it should instead return Result
s like the other utilities). Because of how getty
affects the state of the system, I think the execve()
call may be alright though.
getty
should not be hard-coded to call /bin/ion
. It should instead take an argument specifying what TTY to take control of and what program to start on said TTY (the current values can serve as defaults, however). We may want to implement gettytab
or something similar as well.
This should be very easy. All we need to do is check the exit codes returned from execute()
.
We should be doing:
$HOME
), parameter expansion, command substitution, and arithmetic expansionWord::SingleQuote
and Word::DoubleQuote
are good enoughBecause we use tar-rs
, we will have this issue at the moment.
The above has been fixed, so it will no longer be an issue when using an up-to-date version of tar-rs
.
Given echo \a
in dash, a\n
will be printed. In our shell, echo \a
produces <alert (a beep)>\n
.
I didn't find init in the POSIX standard (http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html).
How about treat init itself as a "type"/"feature" (similar with "POSIX", "Netowrking", etc)?.
Also, I am trying to compile mesabox in OS X (x86_64-apple-darwin
). init
(in "posix" feature) and getty
(in "loginutils" feature) are not compatible with OSX (which are not designed and supposed to work in OSX). Therefore, for people who want to use it in OSX can simply exclude "init" and "loginutils" feature.
Anyway, I don't think supporting OSX is the first priority.
Not sure how commonly these are used, but they are needed for POSIX-compliance.
I am considering removing util::actual_path()
and just making all the utilities assume that the current working directory that we should operate in is the current working directory of the binary. The original point of util::actual_path()
was to support the testing framework, but we won't need it for that once #24 is merged.
I can see it still being useful in some cases where mesabox
is used as a library within another program as it allows operating in different directories without modifying the arguments or changing the host program's current working directory (which could be problematic if that program were to be multi-threaded). At the same time, removing util::actual_path()
makes the utilities simpler and reduces the number of allocations we need to perform.
We should see if there's anywhere where the new NonZero
types would work/benefit us.
The relevant sections are mostly in src/posix/sh/command.rs
. We can't just use ChildStdin
/ChildStdout
/ChildStderr
because we need to support other file descriptors as well, so we need to create the pipes manually using nix
.
EDIT: dunno why I wrote the above, but ChildStdout
should be passed into the next command as its stdin
.
Everything works well except for files/functions under shell. I don't why the integration test cases for shell will not emit hit counts. Therefore, the test coverage for posix/sh
is not accurate.
It's pretty sketchy right now.
I don't know why the first tests are always blocking for over 60 seconds. However, after the blocking issue, all testcases can be run without any problem.
test gnu::arch::test_x86_64 ... test gnu::arch::test_x86_64 has been running for over 60 seconds
test gnu::base32::test_decode ... test gnu::base32::test_decode has been running for over 60 seconds
We can probably remove the 1.26.2
targets on ARM, as if they work on x86_64 they'll likely work on ARM (we don't do much architecture-specific stuff at the moment). We can also probably combine the formatting target with the normal stable
Linux target.
We cannot use the globwalk
crate as it does not support globbing for parent directories (i.e. paths like ../../*
won't work). We will probably need to roll our own directory walker, either by using globset
or modifying glob
to support OsString
s.
test_chmod::test_chmod_ugoa
seems to fail sometimes on Rust 1.26 for some reason. Additionally, the sleep
tests can easily fail if the system takes too long to wake from sleep (or if threads take too long to spawn).
Hi,
I'm trying to contribute some code but get stuck in error handling.
One example I found for returning error is below. However, the progname
and err
are both None, which seems not very helpful.
mesabox/src/posix/chmod/mod.rs
Lines 161 to 165 in e906540
Another example I found is below. This line is quite difficult to understand for new contributors.
mesabox/src/networking/ping/mod.rs
Line 279 in e906540
Could you give some guildeline for error handling in mesabox?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.