ChrisOS

Because like logging frameworks for java, we need more hobby OSs.



Licensing:

Some files have been copied from other sources:

kprintf.c is derived from BSD

multiboot.h and multiboot2.h are from grub.

boot.S is inspired by examples available on osdev.org. The intention is that
code on that site is in the public domain, but the actual provenance of
each example is harder to determine.

syscalls, GDT, IDT and interrupts code is based on examples from James Molloy
at www.jamesmolloy.co.uk and is openly licensed in this forum post:
http://forum.osdev.org/viewtopic.php?p=201752#p201752
basically: ownership retained, all other rights waived, but with a disclaimer.

toolchain.sh is based on the one from HelenOS.

uthash is included in the libs folder. it's slightly modified to add some minor
features.

utils/sort.c is based on code from wikipedia

In each case, any required copyright info is inside the file.
All other files are my own work, see the license in this directory.



Getting Started:

For compiling, start by creating a cross compiler.
Currently I'm using just gcc and friends, targeting 32bit x86 only.
See tools/toolchain.sh

Then edit the tools/ia32.py to set the location and target triple.
I've used i686-pc-elf when building gcc (and for any tools which use llvm,
i'll use i686-linux-gnu for now).

Run make to build.

Run run.sh to test in qemu (it has code for direct multiboot or cdrom iso boot)
and gdb.sh to attach remotely gdb to running qemu.
mkiso.sh builds a bootable cdrom for use with qemu, virtualbox, vmware etc.



Current problems:
interrupt handling is really messy
paging code is too complicated
no slab allocators yet
too reliant on kprintf
how to make io nice or at least useable?

Features done:

boot from syslinux and also directly from qemu (with -kernel) on ia32
tested on qemu, bochs and vmware fusion.
requires multiboot, reads info from multiboot params
jump to higher half
handle interrupts
printf to 80x25 vga screen
basic buggy paging
iffy virtual memory management
basic local apic support
model specific register support
cpuid support
pic support
pit support
apic timer calibration
round-robin scheduler
outline of processes and threads
basic uart driver
bare minimum acpi library integration
ps2 driver
ioapic setup
acpi table loading using libacpica
acpi apic config
ps2 keyboard interrupt handling
system clock with really weird frequency
asynchronous timer callbacks
minimal tty system
basic write-only serial console for kprintf
interrupt blocking spinlocks
kthread blocking mutexes
ring0 -> ring3 switching
ring3 -> ring0 interrupt
syscalls

Wishlist:
Next:
fix system clock frequency 1 second != 1 second
framebuffer
audit everything so far for races and concurrency bugs/flaws
  now that we're messing with interrupts and interleaved execution.
  can we miss wakeup?
  do we correctly hold locks across context switches?
  is scheduler_yield really correct?
object system
completion ports
rewrite memory management + paging. make it work with more than one process
figure out how microkernel arch might look

Later:
make more stuff in kernel/ia32/ useable via HAL
revisit distinction between IRQs and interrupts
audit ring3/ring0 separation
port newlib libc
add i686-chrisos-elf target support to gcc
interactive console
rust support. needs either a libc or patches to rust build
c++ support. needs stack unwinding for exceptions, and global initialisation
Evaluate micro or monolithic designs.
(best effort) Realtime features?
fat16
vfs?
better scheduler
userspace
ramdisk
elf file loading
multiprocessor support
more complete process control blocks
more complete bug-free paging
more complete virtual memory management

Much later:
userspace threads
plug and play
more acpi
more drivers
usb?
port to arm (raspberry pi)?
lots of possibilities

Much Much later and in no particular order:
investigate porting drivers from BSD or minix
port to amd64
networking stack
shell
dtrace



Memory Layout:

initial boot:
loaded as one blob as per linker script at 1MB physical address
immediately loads boot gdt tables to remap first 4MB to 3G linear address
this captures both bios area and kernel

virtual and physical memory management:
setup the ptd with just two pt entries filled in
one for first 4MB region and one for the single table starting at 3G
load gdt with cs and ds ring0 entries starting at linear address 0,
replacing hacky ones which started at 3G
linux uses 3 regions: dma (0-16MB), normal (16MB-1GB), and high (1-4GB),
although the 16MB region may not be needed for PCI DMA?
fetch memory layout from multiboot info. do we need to check this with acpi?
calculate regions of available physical memory


Memory Management:


physical memory management is currently just gdt and paging code
low level paging code is in kernel/ia32/paging.c
it doesn't fail fast enough on bad inputs,
and has some very suspicious logic in places.
needs a rewrite.


virtual memory management is in kernel/mm and 
is mostly in kmem.{c,h}, allocator.{c,h}, vm_map.{c,h} and range.h
create a generic physical map of memory with a list of regions for each useable area of memory
need a physical page allocator that can allocate pages from a region

we use a set of range manipulation macros to add and remove pieces of memory or address space.
pieces are merged and split as we go. i think linux has something similar somewhere.
allocating memory involves allocating a chunk of virtual address space, then one-by-one
allocating physical pages of memory to patch into the page tables.

todo: consider changing the range code to use [start,end) instead of [start,start+size].
as it is, the code can't be used for a single range which covers the entire 32bit space,
since size would wrap around. this won't be a problem for memory (either physical or virtual)
since it's all carved up. could be a problem for other things later.

todo: higher level slab allocators and or better malloc implementation

todo: use a separate allocator for each distinct zone of memory/address space,
instead of relying on flags stored inside the allocator's regions.
that will remove the messy situation with merging regions with different flags set,
will need to store the extent of the regions in the allocator itself so we can check
which allocator some free memory needs to go back to.
won't need to store the flags in the allocated memory piece since we can do a range
check instead of flags check when freeing.

todo: make sure we track used memory adequately. 
will become necessary when we're forking, paging to disk, and mmap'ing

todo: process memory
kernel threads don't need their own address space -- the kernel's memory,
which lives above 3GB is mapped to all processes's address spaces.
kernel threads don't belong to a separate kernel process.
user processes have their own address space, which allocate areas between 0GB and 3GB
each thread gets its own stack
each process needs an initial amount of heap large enough to hold the program binary code+data
except the kernel threads which run code directly from the kernel area.

create kernel threads:
1. no process setup.
2. create main kernel thread. allocate stack page(s).
3. initialise registers and set EIP to entry point, ESP to stack, code and data segments set for ring 0
4. additional threads copy initial thread, but separate stack and entry point

create pid 1:
1. init process structs
2. exec init process

fork process:
1. duplicate address space, add to page reference counts
2. duplicate threads, copy stack, copy registers.

exec user process:
1. create new address space, copying from old one
2. destroy threads
3. create new main thread with new stack and registers. set code and data segments for ring 3

exit user process:
1. set exit code
2. mark as dead
3. signal parent SIGCHLD
4. reparent surviving child processes
5. remove dead child processes
6. destroy threads
7. destroy address space, decrementing page reference counts
8. close file handles etc.

collect process (cleanup dead process):
1. remove process from data structures

TTY system:
there's a ps2 keyboard driver which is capable of translating 
scancodes to ascii key up/down events, including modifier keys.
the events are sent to a tty system with only one tty device.
there's basic line buffering and backspace support but no cursor.
getc and gets are supported