phil-opp / blog_os Goto Github PK
View Code? Open in Web Editor NEWWriting an OS in Rust
Home Page: http://os.phil-opp.com
License: Apache License 2.0
Writing an OS in Rust
Home Page: http://os.phil-opp.com
License: Apache License 2.0
Because Intel CPUs don't remember "not present" pages, documentation from Intel may say that you don't need to invalidate when changing a page from "not present" to "present". Intel's documentation is only correct for Intel CPUs. It is not correct for all 80x86 CPUs. Some CPUs (mostly Cyrix) do remember when a page was "not present" and because of those CPUs you do have to invalidate when changing a page from "not present" to "present".
From this stackoverflow answer: http://stackoverflow.com/a/28384866/866447
Should be:
test_long_mode:
mov eax, 0x80000000 ; Set the A-register to 0x80000000.
cpuid ; CPU identification.
cmp eax, 0x80000001 ; Compare the A-register with 0x80000001.
jb .no_long_mode ; It is less, there is no long mode.
mov eax, 0x80000001 ; Set the A-register to 0x80000001.
cpuid ; CPU identification.
test edx, 1 << 29 ; Test if the LM-bit, which is bit 29, is set in the D-register.
jz .no_long_mode ; They aren't, there is no long mode.
ret
But is:
check_long_mode:
mov eax, 0x80000000 ; Set the A-register to 0x80000000.
cpuid ; CPU identification.
cmp eax, 0x80000001 ; Compare the A-register with 0x80000001.
jb .no_long_mode ; It is less, there is no long mode.
mov eax, 0x80000000 ; Set the A-register to 0x80000000.
cpuid ; CPU identification.
cmp eax, 0x80000001 ; Compare the A-register with 0x80000001.
jb .no_long_mode ; It is less, there is no long mode.
ret
http://os.phil-opp.com/modifying-page-tables.html
It looks like the P4 table entry goes to bit 48 and overlaps bit 47, but 48 is a copy of 47 and part of the signed extension.
When you mention the write!/writeln! macros can be now used, the example uses Writer::new(...) although Writer doesn't have a
new
method.
It seems like grub-mkrescue
causes some issues. See this comment.
Update: We should add some notes regarding
grub-mkrescue
in some distros (grub2-mkrescue
)mtools
(to create an EFI image) or -d /usr/lib/grub/i386-pc
(to avoid EFI) on EFI systemsxorriso
is contained in libisoburn
for some distros@cmr created a bitflags-core fork, so we should switch to it.
Update: It seems like the official bitflags crate has a no_std feature now. See bitflags/bitflags#29.
Replace
qemu-system-x86_64 -drive format=raw,file=os.iso
with the simpler
qemu-system-x86_64 -cdrom os.iso
Add a note in Entering Longmode that stack_bottom
needs to be above stack_top
in the source code because swapping these symbols causes ugly bugs.
Currenly we use redcarpet, which won't be available after May 1st.
Alternative: switch to hugo.
https://github.com/phil-opp/blog_os/blob/entering_longmode/src/arch/x86_64/boot.asm#L57
This line uses imul, when unsigned semantics are required, in general.
Currently the compiler issues the following warnings:
src/memory/paging/table.rs:90:1: 90:15 warning: enum is never used: `Level3`, #[warn(dead_code)] on by default
src/memory/paging/table.rs:90 enum Level3 {}
^~~~~~~~~~~~~~
src/memory/paging/table.rs:91:1: 91:15 warning: enum is never used: `Level2`, #[warn(dead_code)] on by default
src/memory/paging/table.rs:91 enum Level2 {}
^~~~~~~~~~~~~~
src/memory/paging/table.rs:32:27: 32:44 warning: private trait in public interface (error E0445), #[warn(private_in_public)] on by default
src/memory/paging/table.rs:32 impl<L> Table<L> where L: HierarchicalLevel
^~~~~~~~~~~~~~~~~
src/memory/paging/table.rs:32:27: 32:44 warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
src/memory/paging/mapper.rs:88:5: 93:6 warning: method is never used: `map`, #[warn(dead_code)] on by default
src/memory/paging/mapper.rs:88 pub fn map<A>(&mut self, page: Page, flags: EntryFlags, allocator: &mut A)
src/memory/paging/mapper.rs:89 where A: FrameAllocator
src/memory/paging/mapper.rs:90 {
src/memory/paging/mapper.rs:91 let frame = allocator.allocate_frame().expect("out of memory");
src/memory/paging/mapper.rs:92 self.map_to(page, frame, flags, allocator)
src/memory/paging/mapper.rs:93 }
src/memory/paging/mapper.rs:102:44: 102:53 warning: unused variable: `allocator`, #[warn(unused_variables)] on by default
src/memory/paging/mapper.rs:102 pub fn unmap<A>(&mut self, page: Page, allocator: &mut A)
^~~~~~~~~
src/memory/paging/mapper.rs:112:13: 112:18 warning: unused variable: `frame`, #[warn(unused_variables)] on by default
src/memory/paging/mapper.rs:112 let frame = p1[page.p1_index()].pointed_frame().unwrap();
^~~~~
The warnings in mapper.rs
are ok (we don't use these values yet), but the table.rs
warnings need to get fixed.
HierachicalLevel
, Level3
, and Level2
publicLevel3
and Level2
(why is it even dead code?)[deleted]
Some Mac OS users seem to have tool problems in the first post. So it might be a good idea to create a separate page that lists all needed extra steps (such as updating nasm
).
Unfortunately, I have zero Mac experience and can only copy instructions from the post's comments. So if someone got it working, I would really appreciate a list of required commands!
The "Some Tests" section states that a screen character consists of an 8-byte color code and 8-byte character. This should read as 8-bit for both.
See intermezzOS/kernel#8 for advantages of dual licensing.
Are there any drawbacks?
See monocasa's comment on reddit and the OSDev forum.
At the moment you define PhysicalAddress
and VirtualAddress
as synonyms via the type
keyword, but this has the sideeffect that either can be passed to functions that take the other since they're both usize
. It might be worth defining them via the newtype pattern (struct PhysicalAddress(usize);
) instead.
Ahmed Charles noted in a comment that we set the write protect
bit when enabling paging. It's bit 16 in the CR0
register, Wikipedia describes it like this:
When set, the CPU can't write to read-only pages when privilege level is 0
The question is wether we should keep it (and add an explaining comment) or remove it (since it's not required for the switch to long mode).
I think it's reasonable to set it but maybe it's better to do it in a later post when it's required. When we remove it, it must not cause problems for people with the old version.
Any opinions are appreciated :).
src/vga_buffer.rs:1:5: 1:22 error: use of unstable library feature 'unique': needs an RFC to flesh out design (see issue #27730)
src/vga_buffer.rs:1 use core::ptr::Unique;
^~~~~~~~~~~~~~~~~
src/vga_buffer.rs:1:5: 1:22 help: add #![feature(unique)] to the crate attributes to enable
The setup_SSE
function can be moved to the boot.asm
file. Then we don't need an error function in the long_mode_init.asm
file.
Hey Phil,
Thanks again for a rad tutorial. I've been trying to do stuff on my own, while sort of peeking back at your stuff for hints. Given that 'print to the screen' is mostly Just Rust, I tried doing it myself. I figured opening an issue might be a decent way to discuss the differences, I'm not sure it's the right thing, let me know if I should have emailed you. 😄
Here's where I'm at: https://github.com/steveklabnik/nucleus/blob/cec875df64d988d707537c8058aa4862a532ef36/vga/src/lib.rs
The big difference, looking back, is in the way we built the buffer: I made mine a single, long buffer, and did calculations to get the x/y going. I also wrote a flush()
that ends up doing the writing of the buffer to the screen, which is just a copy, since the buffer now has the same representation as the VGA memory itself.
Your code also handles long lines correctly; I'll have to fix mine to take this into account. I also don't need Unique
, due to flush()
handling it.
And I think I spotted another bug, I'm making the arrays twice as long as they need to be, now that they hold the struct rather than the bytes they did before...
Anyway. Would be interested in hearing your thoughts, or if I've done something else silly somewhere.
Most of the posts are pretty long, so a table of contents might be a good idea.
It must be called only once.
Some issues that were brought up by rylev on IRC:
cmp
and jmp
Update: below @mtnygard mentioned some issues with Printing to Screen:
impl
method?)lib.rs
and Cargo.toml
changes required for Unique
and spin
write_str
For post 1, Would be nice to point out that grub-mkrescue
needs xorriso
installed. The command throws a cryptic error about xorriso
version otherwise. Would be nice to save a (albeit quick) Google search.
Include changes from phil-opp/phil-opp.github.io#8
in ubuntu 14.04 after installing multirust and running the command multirust update nightly
I ran into the following error:
#[feature] may not be used on the stable release channel
which suggested I was still using stable. running multirust override nightly
did the trick, may be worth throwing into your document for other newbie readers such as myself.
Thanks for making a really excellent tutorial by the way!
Your blog series is normally easy to follow, but I got a bit lost when I started reading "Frame Allocation" because you leapt straight in without saying what a frame is. Could you dedicate a paragraph or two to this so the reader knows where they're standing?
Linux uses the buddy allocator for frame allocations, not for virtual memory. Thanks to @DSMan195276 for pointing it out!
We should fix the last paragraph of the introduction and also remove the wrong “kernels normally don't do heavy concurrent allocations” claim.
On http://blog.phil-opp.com/rust-os/multiboot-kernel.html you have used multiboot_header but in the git source and the page http://blog.phil-opp.com/rust-os/setup-rust.html you refer to the section multiboot . little confusing if following along step by step.
As a foot-note, thank you so much for this series, it is a great educational piece.
Instead of merging .multiboot_header
into . rodata
, we could just specify it as noalloc
.
Support for 1GB pages was introduced in QEMU 2.1 in 2014. Intel CPUs support it since Westmere (2010).
Either add a test through CPUID 0x80000001, EDX bit 26 or use 2MB pages instead.
…and not a nightly from the time the post was written. Probably link the travis build, too.
Also check the linked Vagrantfile.
reported by @mmalone on IRC
dd 0x100000000 - (0xe85250d6 + 0 + (header_end - header_start))
produces the warning:
multiboot_header.asm:7: warning: numeric constant 0x100000000 does not fit in 32 bits
The following version compiles without warnings (but causes a warning on Linux):
dd - (0xe85250d6 + 0 + (header_end - header_start))
The AreaFrameAllocator
is unsafe to use after kernel remapping because it references the multiboot information structure, which is not remapped. I'm not sure what your next steps are (it's possible this is by design) but this took me a hot second to debug once I got through your last post and started off on my own adventures. If other people are having similar problems you have a couple choices:
remap_the_kernel
, like you did the VGA bufferYou already have a reference to the boot_info
object in remap_the_kernel, so you can use that for the remap. It's a bit ghetto, but here's a quick implementation of the latter option:
let boot_info_frame = Frame::containing_address(boot_info as *const _ as usize);
mapper.identity_map(boot_info_frame, PRESENT, allocator);
It would be nice to have some kind of introduction on the front page, which describes the scope of the project and provides some links (e.g. to the github repo). Then we could also remove the ugly Latest Post banner.
Maybe just Page Tables
...
➜ os-rust nasm -f elf64 multiboot_header.asm
nasm: fatal: unrecognised output format 'elf64' - use -hf for a list
This is the error I get when going through part one of your tutorial when trying to specify elf64 as the format for the linked executable.
nasm -hf lists:
valid output formats for -f are (`*' denotes default):
* bin flat-form binary files (e.g. DOS .COM, .SYS)
aout Linux a.out object files
aoutb NetBSD/FreeBSD a.out object files
coff COFF (i386) object files (e.g. DJGPP for DOS)
elf ELF32 (i386) object files (e.g. Linux)
as86 Linux as86 (bin86 version 0.3) object files
obj MS-DOS 16-bit/32-bit OMF object files
win32 Microsoft Win32 (i386) object files
rdf Relocatable Dynamic Object File Format v2.0
ieee IEEE-695 (LADsoft variant) object file format
macho NeXTstep/OpenStep/Rhapsody/Darwin/MacOS X object files
Right now an Entry
is just an u64
. Maybe it should be an AtomicUsize
instead.
The reason is that we're modifying the tables through recursive mapping and thus have some magic addresses. The compiler and the CPU don't know this and could maybe reorder instructions in a wrong way.
For example, it would be bad if the following is reordered (pseudo code):
create_p3_table(p4[0x42]);
modify 0xffffffffffe42000; // the magic address to the created table
If some optimizer would cause modification of the magic address before the p3 is created, it would cause a page fault.
The question is: Is it possible that an optimizer does such things?
Rust complains that most of the VGA colour constants are dead code. This creates a big pile of compiler warnings to wade through; perhaps this enum should have the #[allow(dead_code)]
attribute?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.