open-education-hub / operating-systems Goto Github PK
View Code? Open in Web Editor NEWTeaching resources (OER) for Operating Systems
Home Page: https://open-education-hub.github.io/operating-systems/
License: Other
Teaching resources (OER) for Operating Systems
Home Page: https://open-education-hub.github.io/operating-systems/
License: Other
During local development is hard to write code following super-linter
rules and for each mistake, it will be necessary to run the corresponding action on GitHub, which leads to a waste of time.
Because of this, we need a method to be able to run the super-linter
locally.
My suggestion would be to add a script to check this inside util
folder. You can see here an example to run super-linter
locally.
In the demo/devices/hwattr-ioctl.c
demo, the interface name is hardcoded as the IFNAME
macro. Update the demo to pass the interface name as a command line argument, making the program easily work on different interface names.
The demos in demo/IPC/
feature a stream UNIX domain server and client (unix_socket_server.c
+ unix_socket_client.c
). Rename them to unix_stream_socket_server.c
and unix_stream_socket_client.c
and add a datagram UNIX domain server and client (unix_datagram_socket_server.c
and unix_datagram_socket_client.c
).
Porting asm program to AArch64
task (from System Calls
) states that we should be using skel files from the arm/
subfolder:
Use the skeleton files in the arm/ folder. Find information about the aarch64 system calls here.
However, this folder does not exist. The only related info that can be found is placed in the solutions/
file, which is not ideal if someone wishes to work on the item without checking the answers.
If, though, the student is expected to resolve the exercise independently, the task should be updated accordingly.
In the Data lab there are quizzes only for the Working with Memory section. The others have dedicated quiz sections which are marked with TODO
.
The write()
syscall is used multiple times in the Software Stack lab [1, 2, 3]. This is problematic because the students don't know what a file descriptor is at this point in the course. We need to add some lines here that briefly explain that file descriptors will be studied later in the course and that for now, it suffices to know what 0, 1 and 2 are.
[1] https://github.com/open-education-hub/operating-systems/blob/master/content/chapters/software-stack/lab/support/basic-syscall/hello.asm
[2] https://github.com/open-education-hub/operating-systems/blob/master/content/chapters/software-stack/lab/support/syscall-wrapper/syscall.s
[3] https://github.com/open-education-hub/operating-systems/blob/master/content/chapters/software-stack/lab/support/common-functions/syscall.s
Right now we don't have any PR templates and this leads to the fact that people create PR without a proper description.
It would be nice to have some templates as we use on Unikraft.
For more details, you can check the documentation from GitHub.
libult
is both buggy as described in #51 and difficult to understand. The purpose of this exercise and section is to visualise user-level threads. This can be done a lot easier using either:
Pick one user-level thread implementation and replace libult
with it. The new exercise should focus on the functionality of user-level threads and not on their implementation. The Scheduling section needs to be replaced with "User-level Threads" in accordance with the changes above.
The first proper section of the Compute lab is improperly called "Benchmarks". However, it is not about CPU benchmarks per see. It just wants to convey the message that while the hardware (CPU) may be powerful, it is the brain (OS) that needs to know how to make use of this power. A better name for this section may be "Hardware Perspective".
This subsection of the Data lab uses open()
for the destination file. Because it might not exist, the perm
argument is also given to open()
. The lab support contains no explanation about it, which may be confusing. The support needs to be updated to explain why this argument is needed.
In the demo/devices/filesystem-size.c
demo, the disk / partition name is hardcoded as the FS_PATH
macro. Update the demo to pass the disk / partition name as a command line argument, making the program easily work on different disk / partition names names.
The File Descriptors section only quotes read()
's man page as an argument to why this syscall must be performed inside a loop. This is likely not enough. It should be clearly stated that read()
and write()
may return fewer bytes than requested, which is why we call them inside loops.
With super-linter
as a workflow, support for cpplint
is provided. An additional static checker for C/C++ code is cppcheck
. There is a GitHub action that could be used.
I'm not certain if adding cppcheck
along cpplint
provides any extra benefits.
clang-format
is currently a no Go due to its strict adherence to non-configurable style rules, as described in PR #24 .
checkpatch.pl
is a Perl script used for coding style checking in the Linux kernel. We want to use it for C source code.
Add the checkpatch.pl
GitHub action to this repository.
These lines define BUFSIZE
only in case it's not already defined by libc. In most cases, however, it is defined and students try to redefine it here to no avail. Replace BUFSIZ
with BUFSIZE
and remove the #ifndef
block to avoid this confusion. Update the Markdown support accordingly.
All Makefile
s use this general build system.
If something changes with this we need to check if it will affect some Makefile
s.
The implementation of servers in demo/IPC/
is: receive connection, receive, send, end. Update the implementation to fit a proper server implementation, where the server runs an infinite loop (while(1)
) waiting for connections, receive,s end, end connection, loop.
This goes for all 4 servers in demo/IPC/
:
datagram_socket_server.c
stream_socket_server.c
unix_socket_server.c
The Memory Security section is incomplete. The skeleton needs to be completed with text, quizzes and exercises.
The code of most exercises in the Compute lab is written in D. However, this brings little to no benefit to students. It would be much better if exercises were mostly in C, with a few examples in Python here and there. The lab support (the Markdown content) should be updated accordingly.
2022-12-15 17:23:51 [ERROR] Found errors in [sqlfluff] linter!
2022-12-15 17:23:51 [ERROR] Error code: 2. Command output:
------
User Error: No dialect was specified. You must configure a dialect or specify one on the command line using --dialect after the command. Available dialects:
ansi, athena, bigquery, clickhouse, databricks, db2, exasol, hive, mysql, oracle, postgres, redshift, snowflake, soql, sparksql, sqlite, teradata, tsql
Also, running it locally with sqlfluff parse db.sql --dialect mysql
will fail with parse errors. So, even if the above error is solved, the check will still fail.
The .sql file was generated with mysqldump and wasn't modified afterwards in any way.
We use labels to classify pull requests. Labels titled area/...
or topic/...
could be automatically assigned, according to their contents.
Create a GitHub workflow to automate the assigning of labels to pull requests. Take inspiration from the labeler
GitHub action, or from the governance
repository for Unikraft.
The chapters are currently displayed in an incorrect order on Docusaurus:
The correct order is:
The order of chapters in config.yaml
has te be changed to reflect the one above.
The subsection on conditions does a bad job at explaining why the condition needs to be checked in a while
loop:
while len(messages) == 0:
event.wait()
Concretely, it doesn't mention that a spurious wakeup may happen. This explanation is required to clarify the way in which conditions work.
This subsection compares threads to processes but fails to specify the differences between them clearly. This table from the lecture needs to be added to the lab as well.
There is currently plenty of duplicate code between support/
and solution/
directories for labs. The best approach is to only provide the solution/
folder inside the repository. And the support/
files will be generated from that.
This approach is currently used by the linux-kernel-labs
GitHub organization, by the generate_skels.py
script. The script generates the skels/
directory with the support files from the corresponding tools/labs/templates/
directory.
This practice item only briefly mentions major and minor page faults without going into detail about them. Create a follow-up exercise in the Arena that showcases the differences between the two. Also add a quiz for this new exercise.
After compiling hello.s
provided in the solution/
folder and running the executable with
qemu-aarch64 -L /usr/aarch64-linux-gnu ./hello
the program crashes with a SEGFAULT
.
Further debugging (and surfing the net) proved that the aforementioned behavior is a bug from qemu
as documented here and on the qemu
upstream mailing list.
It would be nice to have the app running, so a possible hack would be adding some padding:
.section .bss
.lcomm buffer, 128
.p2align 12
Another solution would be getting rid off of the bss
section completely as it is not used by the program.
Note: The code is perfectly fine and runs alright on AArch64
machines (without emulation or qemu
being involved).
Right now we don't have any issue templates and this leads to the fact that people create issues without all the details and without any labels.
It would be nice to have some templates as we use on Unikraft.
For more details, you can check the documentation from GitHub.
The overview tabs for each chapter are inconsistent.
For example, for compute, io and app-interaction contains only a table of content, but the software-stack and data contain beside the table of content some introduction information.
I don't know how it's better, but I think it should be a difference between Overview and Introduction and this should be consistent throughout all chapters.
Also, the setup part for Git should be removed right now since we have the base support
directory for every lab.
Intro page is empty.
It doesn't contain the general information mentioned there.
The support code for conditions uses with event
constructs. They are confusing for students who don't have a good grasp on synchronisation mechanisms yet. They should be replaced with event.acquire()
. and event.release()
to make the code clearer. Also, update the corresponding Markdown file to remove the reference to the with
statement.
ltrace
works by extracting stuff out of the PLT, so if binutils
is optimizing a PLT
access into a GOT
access (by relying on now binding
) it's likely that ltrace
is going to stop working.
I've tested this on 2 quite different Debian based distros:
bin | version |
---|---|
gcc | 9.4.0 |
ld | 2.34 |
The main question is why are binaries linked with the -z now
flag (now binding
) on Ubuntu.
Even when trying to inspect /bin/ls
, there is no output for ltrace
:
$ scanelf -a /bin/ls
TYPE PAX PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW- - - NOW /bin/ls
$ ltrace /bin/ls
issues-ltrace.md
+++ exited (status 0) +++
bin | version |
---|---|
gcc | 10.2.1 |
ld | 2.35.2 |
On my ParrotOS machine (a Debian based rolling distro), binaries are linked by default with -z lazy
, ergo, ltrace
works like expected:
$ scanelf -a /bin/ls
TYPE PAX PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW- - - LAZY /bin/ls
$ ltrace /bin/ls
strrchr("/bin/ls", '/') = "/ls"
setlocale(LC_ALL, "") = "LC_CTYPE=en_US.UTF-8;LC_NUMERIC="...
bindtextdomain("coreutils", "/usr/share/locale") = "/usr/share/locale"
textdomain("coreutils") = "coreutils"
__cxa_atexit(0x56359a85d5a0, 0, 0x56359a873368, 0) = 0
isatty(1) = 1
[...]
+++ exited (status 0) +++
https://stackoverflow.com/questions/43213505/no-output-when-running-ltrace
https://sourceware.org/legacy-ml/binutils-cvs/2015-05/msg00106.html
https://stackoverflow.com/questions/58076539/plt-plt-got-what-is-different
This change will enable tracing library calls have no PLT. As PLT is needed to trace those libcalls, uftrace make up a trampoline and fill necessary instructions there.
The code is available at review/libcall-noplt-v1 branch.
I've found 2 alternative hacks:
uftrace
tool:$ sudo apt install uftrace
$ uftrace --force /usr/bin/pwd
/home/maria/Documents/school-work/SO/my-so-lab-2022/software-stack/libcall-syscall
# DURATION TID FUNCTION
2.766 us [ 26498] | getenv();
0.160 us [ 26498] | strrchr();
82.619 us [ 26498] | setlocale();
0.591 us [ 26498] | bindtextdomain();
0.742 us [ 26498] | textdomain();
0.241 us [ 26498] | __cxa_atexit();
4.511 us [ 26498] | getopt_long();
7.036 us [ 26498] | getcwd();
15.686 us [ 26498] | puts();
0.170 us [ 26498] | free();
0.241 us [ 26498] | __fpending();
0.261 us [ 26498] | fileno();
0.241 us [ 26498] | __freading();
0.100 us [ 26498] | __freading();
0.621 us [ 26498] | fflush();
1.273 us [ 26498] | fclose();
0.110 us [ 26498] | __fpending();
0.110 us [ 26498] | fileno();
0.111 us [ 26498] | __freading();
0.090 us [ 26498] | __freading();
0.120 us [ 26498] | fflush();
0.592 us [ 26498] | fclose();
diff --git a/content/chapters/software-stack/lab/support/libcall-syscall/Makefile b/content/chapters/software-stack/lab/support/libcall-syscall/Makefile
index bd91a72..a91df3f 100644
--- a/content/chapters/software-stack/lab/support/libcall-syscall/Makefile
+++ b/content/chapters/software-stack/lab/support/libcall-syscall/Makefile
@@ -1,4 +1,5 @@
CFLAGS = -Wall
+LDFLAGS = -z lazy
.PHONY: all clean
$ ltrace ./call
fopen("a.txt", "wt") = 0x5603b16fb2a0
strlen("Hello, world!\n") = 14
fwrite("Hello, world!\n", 1, 14, 0x5603b16fb2a0) = 14
fflush(0x5603b16fb2a0) = 0
+++ exited (status 0) +++
The second alternative does half of the work, as we cannot change binaries that are already compiled (e.g. /bin/pwd
).
The sockets opened by each browser tab can be inspected using lsof
. This behaviour is merely mentioned here, but no demo is provided. Both the lab and the lecture would benefit from having a demo where lsof
is running while opening several firefox tables.
The Basic Syscalls chapter in the Software Stack lab starts very abruptly by showcasing the x86_64 syscall API without saying what a syscall is, why it needs to be performed etc.
This should be addressed by adding a paragraph (together with some diagrams if possible) stating the reason syscalls are important, what they do and why we need them before the Basic System Calls
section. These details should be added to a new section.
I think in this section 1 you should explain better this sentence.
So using the hardware support is more efficient, but it can only be leveraged for simple, individual instructions, such as loads and stores.
Hardware support is a bit unclear to me.
A good example is presented in this blog post 2. That example with assembly-code explained very well the difference between Mutex and Atomic and what is with that hardware support.
The demo in the data lab shows allocations of 8KB, but ALLOC_SIZE_KB
is set to 256 (corresponding to 256KB allocations), as requested by this practice item. It should be set to 8.
The Now We Pipe exercise asks students to simply send some data from the child process to the parent. Add a sample console output that shows input being written (to the child process) and then echoed by the parent.
This section about multithreaded programs is misplaced in the Data lab because it proceeds the Compute chapters, which is where students first learn about threads. This section together with its supporting code and quiz should be placed in the Compute lab, at the end of the "Threads" section.
Docusaurus uses the name of the file instead of the first heading as the title page. This way we end up with sections looking like this where the title is set to process-memory
instead of Process Memory
:
The Markdown files for each section have to be updated and the topmost heading changed from ##
to #
. The rest of the headings probably need to be bumped one level higher as well since Markdownlint doesn't allow adjacent headings to differ by more than one #
. Look into lab content files as well as quizzes.
Building the website locally is tedious, takes a very long time, and 20GB+ of disk space. This makes it difficult to test changes to the documentation.
The solution is to add a GitHub action triggered by a new commit to a PR that runs the builder, publishes the rendered content, and then adds a comment to the PR containing a link to the published website that contains the changes within the PR. You can draw inspiration from the GitHub action used by the OS Internals class. An example of a comment created by this action is linux-kernel-labs/linux#316 (comment).
Ideally, this action would only be triggered by changes to the Markdown documentation or quizzes and not just code. For this, PRs should first be labelled automatically as per #89. According to this issue, the action should only be executed when the area/documentation
label is present.
The repository features super-linter
as a worfklow in the .github/workflow/
directory. This is enabled at each push and it runs markdownlint-cli
.
We need to update the README.md
file and / or the CONTRIBUTING.md
file with information on installing and using markdownlint
locally, before creating a PR.
Installing is done using:
$ npm install -g markdownlint-cli
Checking a Markdown file is done using:
$ markdownlint --config .github/workflows/config/config.json --rules .github/workflows/rules/rules.js content/chapters/data/lab/quiz/*.md
The buffer cache massively improves the performance of both mmap_copy
and copy
. Because the buffer cache is taught during the I/O chapter, students might get confused why the running time of each program drops so much. To avoid this, use this script from the I/O lab here as well and update the lab support to briefly explain why it's needed.
socketpair()
can be used to create a pair of unnamed UNIX sockets (AF_UNIX
family). It creates a bidirectional connection between two related processes. It's similar to a a pipe (and to the pipe()
call - it returns two file descriptors), except that it's bidirectional.
Add an exercise as part of the I/O lab to use socketpair()
to make to related processes communicate. The socket pair can use both SOCK_STREAM
and SOCK_DGRAM
as socket types.
This is an example of using socketpair()
, could also be referenced in the exercise.
For this lab there are 2 images missing in the web interface.
Capitolu SO-Cloud este vizibil strict din pagina Overview, din pagina generala a laboratorului nu poate fi vazuta.
De asemenea, nu se poate da next din capitolul anterior.
slides.mk
is currently using ffmpeg
to create a GIF file from multiple SVG images (see the videos
target. This causes the image quality of some resulting GIFs to drop unexpectedly. In addition, text and black lines get some unwanted yellowish shadows. I've included an example GIF and one of the original SVGs used for creating it below.
Using convert
instead of ffmpeg
seems to result in higher-quality GIFs. An example of how to create a GIF from SVGs using convert
is:
student@os:/.../app-interact/lecture$ convert -delay 200 -loop 0 media/synchronization/* out.gif
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.