Xen Kria SOM Starter Kits Support¶
Xen is a free and open-source hypervisor, providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently. The 2023.2 Kria Starter Kit Petalinux BSP as well as Yocto contains support for XEN. AMD specific details about XEN hypervisor can be found on its wiki page.
This page documents how to generate XEN artifacts (based on Kria Starter Kit embedded Linux) for Kria and gives an example on how to boot Ubuntu as guest OS (aka, a DOMU) on a XEN DOM0. This particular combination requires about 4GB of memory, which K26 has. Therefore this can be done on either KV260 or KR260. K24 only has 2 GB of memory, therefore it does not support Xen.
Prerequisites¶
Experience and equipments booting a regular Kria Starter Kit embedded Linux on Kria SOM Starter Kit (you can follow getting started with PetaLinux on KV260, but use a wic image from 2023.2 BSP for your Starter Kit on a minimum 16GB SD Card)
A Linux Desktop to create and program partitions
Yocto (preferred) or PetaLinux tool
Generating Xen Artifacts¶
Xen artifacts can be generated by native Yocto tool flow or found in PetaLinux BSP. You can choose to use one of the tools to create Xen artifacts.
Generating Xen Artifacts in Native Yocto Tool Flow¶
First, set up a Yocto project on a host with Yocto support using the instructions here, stop after source setupsdk step.
Then append the following to the end of build/conf/local.conf to enable Xen:
BOOTMODE = "xen"
IMAGE_INSTALL:append = "packagegroup-petalinux-xen"
ENABLE_XEN_DTSI = "1"
ENABLE_XEN_QEMU_DTSI = "1"
ENABLE_XEN_UBOOT_SCR = "1"
XEN_SERIAL_CONSOLES = "serial1"
Then build the image the usual way:
To generate Xen support for KV260, use k26-smk-kv as the <machine name>:
MACHINE=k26-smk-kv bitbake kria-image-full-cmdline
To generate Xen support for KR260, use k26-smk-kr as the <machine name>:
MACHINE=k26-smk-kr bitbake kria-image-full-cmdline
There is no Xen support for combined MACHINE name k26-smk.
Yocto generated wic that contains XENs support can be found in <yocto_project>/build/tmp/deploy/images/<machine name>/kria-image-full-cmdline-<machine name>.wic in the Yocto project.
Xen Artifacts in PetaLinux tool flow¶
Users are encouraged to use Yocto flow, as it is verified and steps are simpler in Prepare SD card stage. If user still require to use PetaLinux, download a 2023.2 or later Kria SOM Starter Kit BSP. Extract the PetaLinux project:
petalinux-create -t project -s xilinx-<hardware>-<version>-<timestamp>.bsp
cd xilinx-<hardware>-<version>
The wic image required is in pre-built/linux/images/petalinux-sdimage.wic.xz, and the XEN artifacts required can be found in pre-built/linux/xen.
Prepare SD Card¶
Flash the embedded Linux wic image from either Yocto or PetaLinux flow to an SD card using Balena Etcher. This will set up the SD card to contain 2 partitions - partition 1 with boot images (accessible by Windows and Linux) that we will discard and partition 2 with Kria Starter Kit embedded Linux rootfs (not accessible by Windows) that will also be used for XEN DOM0.
Partition 1¶
If you are using Yocto generated wic image - partition 1 already contain XEN artifacts as the wic is generated with XEN support. Below step is for PetaLinux tool Flow only.
PetaLinux tool flow only: Open the SD card on a host computer (can be Windows or Linux), go to partition 1, remove the boot images on that partition and copy all the XEN artifacts in pre-built/linux/xen into partition 1.
We then need to copy over the Ubuntu image that we want to boot as DOMU. Download the iot-limerick-kria-classic-desktop-2204-x07-20230302-63-system-boot.tar.gz file from Kria Ubuntu download site, un-compress it, and copy image.fit file into partition 1 for DOMU later. This file is not being used in XEN booting, but we use SD partition 1 as a place to transfer the file onto target.
Partition 2¶
Partition 2 was programmed with a Kria Starter Kit embedded Linux rootfs from setting SD card image with the Kria Starter Kit embedded Linux .wic image. This can be re-used for XEN.
Partition 3¶
This step needs to be done a Linux host.
Download the Ubuntu rootfs for DomU. Ubuntu image can be downloaded from Kria Ubuntu download site. For this example, we will be using iot-limerick-kria-classic-desktop-2204-x07-20230302-63-rootfs.ext4.xz file.
Unzip the downloaded rootfs file to .ext4 using following:
xz -d iot-limerick-kria-classic-desktop-2204-x07-20230302-63-rootfs.ext4.xz
Plug the SD card into a Linux host machine, in this example we assume the sd card is found in /dev/sda but you may find it in different file path dependent on your machine. Observe the 2 partitions with sudo fdisk -l /dev/sda, it should look like the following with two partitions:
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 8 4194311 4194304 2G c W95 FAT32 (LBA)
/dev/sda2 4194312 12582919 8388608 4G 83 Linux
Enter command sudo fdisk /dev/sda, Enter n to create a new partition. It will ask about partition type, enter p, next it will ask about Partition number, please hit enter or enter the number 3. Finally, it will ask for First and Last Sector - hit enter to automatically create partition with all remaining storage on sd card.
Now it should look like this:
Command (m for help): n
Partition type
p primary (2 primary, 0 extended, 2 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (3,4, default 3): 3
First sector (12582920-31116287, default 12584960):
Last sector, +sectors or +size{K,M,G,T,P} (12584960-31116287, default 31116287):
Created a new partition 3 of type 'Linux' and of size 8.9 GiB.
Next, enter w to write the partition table and exit.
Next, format the newly created partition to support ext4 rootfs. Use following command to format it to a .ext4 format:
sudo mkfs.ext4 -L root /dev/sda3
/dev/sda3 is the path for new partition that we will use for rootfs for DOMU. Run “sudo fdisk -l” again to verify all the partition. It should look similar to what is shown below with 3 partitions:
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 8 4194311 4194304 2G c W95 FAT32 (LBA)
/dev/sda2 4194312 12582919 8388608 4G 83 Linux
/dev/sda3 12584960 31116287 18531328 8.9G 83 Linux
lastly, copy Ubuntu rootfs that we download and extracted earlier to partition /dev/sda3:
sudo dd if=iot-limerick-kria-classic-desktop-2204-x07-20230302-63-rootfs.ext4 of=/dev/sda3
Quick tip: if dd gives error that “No space left on device” yet the file is smaller than space allocated, do the following:
sudo rm /dev/sda3
sudo fdisk /dev/sda # enter "D" and then "3" to delete partition 3
#then remove the SD card and reinsert and go back to the beginning of this section to start from creating partition 3 again.
Now partition 3 is ready as well.
Boot XEN On Target¶
Plug the SD card into Starter Kit and turn on the power. If you used Yocto to generate the Xen artifact - it will automatically boot Xen- you can move to the next section.
If you used PetaLinux to generate Xen artifacts, you should automatically get to u-boot command prompt. Enter this to load xen_boot_sd.scr to 0xc00000 and source the script to boot xen:
For KV260, SD card is on mmc, therefore use the following commands to boot:
load mmc 1:1 0xc00000 xen_boot_sd.scr #KV260 only!
source 0xc00000
for KR260, SD Card is on USB, therefore use the following commands to boot:
load usb 0 0xc00000 xen_boot_sd.scr #KR260 only!
source 0xc00000
This will boot Kria Starter Kit embedded Linux XEN aka DOM0.
Boot UBuntu as DOMU¶
First, change to root:
sudo -s
Locate image.fit file in /boot/image.fit, but location may vary dependent on release and tool used. One can search for it using find / -iname "image.fit".
extract the Ubuntu kernel image:
dumpimage -T flat_dt -p 0 /boot/image.fit -o /home/petalinux/ubuntu_Image # for KV260
dumpimage -T flat_dt -p 0 /boot/image.fit -o /home/petalinux/ubuntu_Image # for KR260
Next, check how much memory is dom0 consuming and available memory using xl info. In our example we should have a little more than 1.9G left to use for DOMU and therefore we will allocate 1900MB for DOMU in next step.
Create a guest0.cfg file with the following content:
If running Xen on KV260:
name = "guest0"
kernel = "/home/petalinux/ubuntu_Image" # Ubuntu Kernel extracted
disk = ['/dev/mmcblk1p3,,xvda'] # Ubuntu Rootfs in partition 3
extra = "console=hvc0 root=/dev/xvda"
memory = 1900 # This can be increased if there is more free memory.
vcpus = 2
If running Xen on KR260:
name = "guest0"
kernel = "/home/petalinux/ubuntu_Image" # Ubuntu Kernel extracted
disk = ['/dev/sda3,,xvda'] # Ubuntu Rootfs in partition 3
extra = "console=hvc0 root=/dev/xvda"
memory = 1900 # This can be increased if there is more free memory.
vcpus = 2
Next, check if /dev/mmcblk1p3 or /dev/sda3 is mounted automatically by dom0 using df -h, if its mounted, unmount it so that DOMU can use it:
umount /dev/mmcblk1p3 # KV260 only
umount /dev/sda3 # KR260 only
Start guest0/DOMU using:
xl create -c guest0.cfg
This will boot to root@ubuntu.
To get out of DOMU, press ctrl-].
After getting out of DOMU, in XEN DOM0 prompt, Enter xl list to see a list of OSes running, you should see something like this:
xilinx-kv260-starterkit-20231:/home/petalinux# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 2048 1 r----- 17916.8
guest0 1 1899 2 r----- 28894.7
To access the guest0 DOMU again, do xl console <ID>
To kill the guest0 DOMU, identify the xl destroy <ID> to kill the session.
Another DOMU session can be started without having to re-boot XEN.
Kria Starter Kit Embedded Linux as DOMU¶
If you would like to run Kria Starter Kit embedded Linux instead of Ubuntu as DOMU, the steps are similar to that of Ubuntu OS as DOMU except for kernel image and root fs artifacts for the perspective OS, as well as the config file. The only 2 artifacts required in this example is in the image folder, file Image and file rootfs.cpio.gz, copy them into boot partition 1. For partition 3, skip the dd command and leave the partition empty.
Once booted up and in root, copy the two artifacts to partition 3. To find out where the partitions are mapped to, use command df -h
k26-smk-kv:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 1.5G 955M 408M 71% /
devtmpfs 563M 4.0K 563M 1% /dev
tmpfs 693M 0 693M 0% /dev/shm
tmpfs 277M 11M 267M 4% /run
tmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup
tmpfs 693M 4.0K 693M 1% /tmp
tmpfs 693M 136K 692M 1% /var/volatile
/dev/mmcblk1p1 511M 147M 365M 29% /boot
/dev/mmcblk1p3 113G 5.2G 103G 5% /run/media/writable-mmcblk1p3
tmpfs 139M 0 139M 0% /run/user/1000
In the example output above (for KV260), boot partition (mmcblk1p1) is mapped to /boot, and third partition (mmcblk1p3) is mounted to /run/media/writable-mmcblk1p3
In this case, this would be the command to copy:
cp /boot/Image /run/media/writable-mmcblk1p3
cp /boot/rootfs.cpio.gz /run/media/writable-mmcblk1p3
Use the following as guest0.cfg (adjusting kernel and ramdisk path accordingly):
name = "guest0"
kernel = "/run/media/writable-mmcblk1p3/Image"
ramdisk = "/run/media/writable-mmcblk1p3/rootfs.cpio.gz"
extra ="console=hvc0 init=/bin/sh root=/dev/ram0"
memory = 1801
vcpus = 1
Known Limitations¶
With current XEN hypervisor configuration, DOMU does not have access to peripherals. Peripheral access are mapped to dom0. Therefore, you will see that ethernet is not accessible for DOMU. Upon DOMU initialization (tested with Kria Starter Kit embedded Linux 2023.2 XEN + Ubuntu 22.04 x07 image) you will see cloud init errors.
XEN on Kria currently do not save its state across reboot - e.g. the files in rootfs or password will not persist over power cycles or reboots.
Copyright© 2023 Advanced Micro Devices, Inc