Xen3 ARMv7/Tegra2 work
======================

This is a an effort to bring up Samsung's Xen3 tree on
Tegra2.

"Bring up" is a very relative word... most of the code
I end up having to (re)write isn't platform specific.
On the other hand, writing an OS is much more exciting
than making an existing one work on $FOO chip, so
I can't complain ;-).

Once the "basics" are in, the more interesting work
of figuring out the correct subset of hypercalls
and porting the Tegra2 Linux kernel to be PV can begin.
I am also musing adding self-hosting as a target - 
certainly it would be a boon for debugging and help
with SMP bring-up.


Status
======

The current TODO, in probable order:

* (critical) fix broken DABT/PABT handlers
* (critical) investigate DABT in __divdi3
* (minor) Make FB and FB console optional through configuration options.
* (minor) push more init code and 16bpp support into tegrafb.
* (critical) Rewrite dom0 bring-up code and switching to VM and back.
* (major) Better crash debugger.
* (nice) L2 cache integration.
* (major) Synchronizing hypervisor mappings across different address spaces.
* (major) Blocking synchronization primitives.
* (nice) USB ACM gadget support.
* (major) Full preemption for xen domains.
* (major) Hypercall definitions.
* (major) Tickless operation.
* (major) Endboot callback list to cleanup devices before Dom0 boot.
* (nice) Fix up the GDB bits.


Differences
===========

Compatibility with Samsung's tree (or other Xen ARM efforts) is
not a goal. Compatibility at hypercall interface is a non-goal
as well. Perhaps I should rename my effort to "Nex" ;-).

Current differences from Samsung's tree:
1) ARMv7-only support.
2) Dom0/U configuration is not hardcoded.
3) Boot through ATAG-compatible bootloader, with all images passed
   through a "boot volume".
4) Kernel threads ("xen domains").
5) No ACM, given likely hypercall changes.
6) A FIQ-based extensible serial kernel debugger with useful commands to help
   debugging and bring-up.
7) In progress work on a framebuffer debug console.
8) VMSAv7-driven rework of page table management.

Full kernel preemption is a design goal. Kernel threads are cooperative at the moment.

Building
========

$ make menuconfig
$ make


Boot volume
===========

To get anywhere, you need to create a boot volume. This is a CPIO archive that at least
should contain a 'dom0' and 'initrd0' files.

Assuming the current directory contains the boot volume contents, the following
command will create a valid Xen boot volume.

$ find . | cpio --create --format='newc' > ../bootvol.img


Running
=======

Assuming you have a Xoom with the serial port through the headphone h^Hjack.

$ sudo fastboot boot  /home/andreiw/src/xen3-arm-tegra/dist/install/boot/xen-bin-3.0.2-2 bootvol.img

ATAG    (P): 0x00000100
Linked  (V): 0xff004000
Actual  (P): 0x00a00800
Desired (P): 0x00104000

Handing off to C...
[XEN]  __  __            _____  ___   ____    ____  
[XEN]  \ \/ /___ _ __   |___ / / _ \ |___ \  |___ \ 
[XEN]   \  // _ \ '_ \    |_ \| | | |  __) |__ __) |
[XEN]   /  \  __/ | | |  ___) | |_| | / __/|__/ __/ 
[XEN]  /_/\_\___|_| |_| |____(_)___(_)_____| |_____|
[XEN]                                               
[XEN] Xen/ARMv7 virtual machine monitor for ARM architecture
[XEN] Copyright (C) 2012 Andrei Warkentin <andreiw@msalumni.com>
[XEN] Copyright (C) 2007 Samsung Electronics Co, Ltd. All Rights Reserved.
[XEN]  http://www.cl.cam.ac.uk/netos/xen
[XEN]  University of Cambridge Computer Laboratory
[XEN] 
[XEN]  Xen version 3.0.2-2 (andreiw@localdomain) (gcc version 4.4.3 (GCC) ) Wed Sep 19 17:15:46 EDT 2012
[XEN]  Platform: arm-tegra
[XEN]  GIT SHA: ab918d38f19258b8a03f8871d3dacbac45dab161
[XEN]  Kernel command line: <NULL>
[XEN]  Boot volume: 0x5000000-0x5022400
[XEN] Virtual memory map:
[XEN] ===============================================
[XEN] VECTORS_BASE:          0xffff0000
[XEN] DIRECTMAP_VIRT_END:    0xfff00000 (PA 0x01000000)
[XEN] M pages end:           0xfff00000 (PA 0x01000000)
[XEN] M pages start:         0xff54f000 (PA 0x0064f000)
[XEN] Frame table end:       0xff046000 (PA 0x00146000)
[XEN] Frame table start:     0xff546000 (PA 0x00646000)
[XEN] Kernel ends at:        0xff045074 (PA 0x00145074)
[XEN] Kernel linked at:      0xff004000 (PA 0x00104000)
[XEN] Idle page tables:      0xff000000 (PA 0x00100000)
[XEN] DIRECTMAP_VIRT_START:  0xff000000
[XEN] MAPCACHE_VIRT_START:   0xfec00000
[XEN] SHARED_INFO_BASE:      0xfeb00000
[XEN] IOREMAP_VIRT_START:    0xfc000000
[XEN] HYPERVISOR_VIRT_START: 0xfc000000
[XEN] looking at bank 0
[XEN]         base - 0x0
[XEN]         end  - 0x40000000
[XEN] IVT page is PA 0x01fff000
[XEN] FIQ debugger enabled
[XEN] Using scheduler: Simple EDF Scheduler (sedf)
[XEN] fb: no framebuffer support
[XEN] Waiting for FIQDB
fiqdb> 
fiqdb> 
fiqdb> ?

FIQ debugger commands:
bt - backtrace
regs - register dump
xregs - extended register dump
reset - reset
continue - continue execution after trap

Xen debugger commands:
doms - List domains
ver - hypervisor version
hexdump - hexdump VA
v2p - VA to PA translation
fiqdb> bt
sp ff03cf98 fp ff03cfac pc ff02e788 (debugger_trap_immediate+0x44/0x70)
sp ff03cfb0 fp ff03cffc pc ff024f50 (start_xen+0x218/0x2e4)
sp ff03d000 fp 0000c090 pc 00000000
fiqdb> regs
r0 00000020  r1 00000000  r2  ff036dbc  r3 00000001
r4 001042d0  r5 000000df  r6  00c5107f  r7 00335e84
r8 00000003  r9 00a00800  r10 0000c090  fp ff03cfac
ip ff03cf48  sp ff03cf98  lr  ff024018  pc ff02e788
cpsr 20000093 mode SVC
spsr 00000000 mode ???
fiqdb> xregs
r0 00000020  r1 00000000  r2  ff036dbc  r3 00000001
r4 001042d0  r5 000000df  r6  00c5107f  r7 00335e84
r8 00000003  r9 00a00800  r10 0000c090  fp ff03cfac
ip ff03cf48  sp ff03cf98  lr  ff024018  pc ff02e798
cpsr 20000093 mode SVC
spsr 00000000 mode ???
svc: sp ffefeed0  lr ff02e828  spsr 00000000  mode ???
abt: sp ff03f000  lr 00000000  spsr 00000000  mode ???
und: sp ff040000  lr 00000000  spsr 00000000  mode ???
irq: sp ff03e000  lr 00156c83  spsr 6000001f  mode SYS
fiq: r8 ff02ec8c  r9 ff03b8ac  r10  00000000  r11  ff02e798 r12 20000093
fiq: sp ffefefb8  lr ff02e79c  spsr 20000093  mode SVC
fiqdb> ver
Xen version 3.0.2-2 (andreiw@localdomain) (gcc version 4.4.3 (GCC) ) Wed Sep 19 17:17:39 EDT 2012
Platform: arm-tegra
GIT SHA: ab918d38f19258b8a03f8871d3dacbac45dab161
fiqdb> hexdump  
Usage: hexdump 1|2|4|8 <hex size>@<hex VA address>
fiqdb> hexdump 1 0x100@0xff546000
ff546000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff546020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff546040: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff546060: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff546080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff5460a0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................................
ff5460c0: ff ff ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................
ff5460e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................
fiqdb> v2p
Usage: v2p u|p r|w <hex VA address>
fiqdb> v2p u r 0xffff0000
Unpriviledged read VA 0xffff0000 to PA: 0x01fff000
PAR: 0x081fff0f4 flags: shareable 
Outer: write-back, write-allocate
Inner: write-back, no write-allocate
fiqdb> continue
fiqdb> 
[XEN] Continuing...
Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!

fiqdb> 
fiqdb> 
fiqdb> doms
1: 0x21 priv xen
2: 0x21 priv xen
fiqdb> [XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 1!
[XEN] Hello from a Xen domain 2!
[XEN] Hello from a Xen domain 1!


Hacking away
============

You're welcome! I'd love to hear about and see your changes.

I use emacs, and have a special editing mode that I tend to like.

    (defun xen-c-mode ()
      "C mode with adjusted defaults for use with my Xen source."
      (interactive)
      (c-mode)
      (c-set-style "K&R")
      (setq tab-width 3)
      (setq indent-tabs-mode nil)
      (setq c-basic-offset 3)
      (setq show-trailing-whitespace t))

      ;; Draw tabs with the same color as trailing whitespace  

      (add-hook 'font-lock-mode-hook  
                (lambda ()  
                  (font-lock-add-keywords  
                   nil  
                   '(("\t" 0 'trailing-whitespace prepend)))))


Contact Info
============

Andrei Warkentin <andreiw@vmware.com>
Andrei Warkentin <andreiw@msalumni.com>
Andrei Warkentin <andrey.warkentin@gmail.com>