Enabling KVM Support on ARM FVP and Running ARM64 KVM

ARM64 KVM Boot Failure on Fixed Virtual Platform (FVP)

The core issue revolves around the inability to boot ARM64 KVM (Kernel-based Virtual Machine) on the ARM Fixed Virtual Platform (FVP). The user has successfully built a Linux-on-FVP environment but encounters issues when attempting to enable and verify KVM functionality. Despite ensuring that the KVM configuration flags (CONFIG_KVM=y) are set in the relevant kernel configuration files, the KVM functionality does not appear to be active. Diagnostic commands such as kvm-ok, dmesg | grep KVM, and grep -E -o 'vmx|svm' /proc/cpuinfo yield negative results, indicating that KVM is not operational. This suggests a potential misconfiguration in the FVP setup or an unsupported CPU configuration for KVM.

The ARM FVP is a versatile platform for simulating ARM-based systems, but it requires precise configuration to support advanced features like KVM. KVM is a critical component for virtualization, allowing the Linux kernel to function as a hypervisor. Enabling KVM on FVP involves not only configuring the Linux kernel but also ensuring that the FVP itself is set up to emulate a CPU that supports virtualization extensions. The absence of explicit documentation on KVM support in FVP adds to the complexity, making it challenging to identify the root cause of the issue.

Misconfigured FVP CPU Model and Missing Virtualization Extensions

The most likely cause of the KVM boot failure is the use of an FVP CPU model that does not support virtualization extensions. KVM relies on hardware virtualization support, which is typically provided by CPU features like ARM’s Virtualization Extensions (ARM-VE). If the FVP is configured to emulate a CPU model that lacks these extensions, KVM will not function, regardless of the kernel configuration.

Another potential cause is the incomplete or incorrect configuration of the Linux kernel. While the user has set CONFIG_KVM=y in the kernel configuration files, there may be additional dependencies or configuration options that are not enabled. For example, KVM on ARM64 requires support for specific CPU features, such as the Generic Timer and GIC (Generic Interrupt Controller) virtualization. If these features are not properly configured, KVM will fail to initialize.

Additionally, the FVP itself may require specific parameters to enable virtualization support. The FVP command-line options and configuration files must be carefully reviewed to ensure that all necessary components for KVM are enabled. This includes setting the correct CPU type, enabling virtualization extensions, and configuring the memory and interrupt controllers to support virtualization.

Configuring FVP for KVM and Enabling ARM64 Virtualization

To resolve the KVM boot failure on FVP, a systematic approach is required to ensure that all components are correctly configured. The following steps outline the process of enabling KVM support on FVP and verifying its functionality.

Step 1: Verify FVP CPU Model and Virtualization Extensions

The first step is to ensure that the FVP is configured to emulate a CPU model that supports ARM Virtualization Extensions. The FVP provides several CPU models, and not all of them support virtualization. The recommended CPU model for KVM is the Cortex-A57 or Cortex-A72, as these models include the necessary virtualization extensions.

To configure the FVP with a supported CPU model, modify the FVP command-line parameters to specify the appropriate CPU type. For example:

FVP_Base_RevC-2xA57x4 -C bp.secure_memory=0 -C cluster0.cpu0.CONFIG64=1 -C cluster0.cpu1.CONFIG64=1 -C cluster0.cpu2.CONFIG64=1 -C cluster0.cpu3.CONFIG64=1

This command configures the FVP to emulate a system with four Cortex-A57 CPUs, which support ARM Virtualization Extensions. The CONFIG64 parameter ensures that the CPUs operate in 64-bit mode, which is required for ARM64 KVM.

Step 2: Enable KVM in the Linux Kernel Configuration

While the user has already set CONFIG_KVM=y in the kernel configuration files, it is important to verify that all related KVM options are enabled. The following table lists the key kernel configuration options that must be set to enable KVM on ARM64:

Configuration Option	Description
CONFIG_KVM	Enables KVM support in the kernel.
CONFIG_KVM_ARM_HOST	Enables KVM support for ARM hosts.
CONFIG_KVM_ARM_VGIC	Enables support for the ARM Virtual GIC.
CONFIG_KVM_ARM_TIMER	Enables support for the ARM Generic Timer.
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT	Enables CPU relaxation in KVM.
CONFIG_KVM_MMIO	Enables support for MMIO-based devices in KVM.

Ensure that these options are set in both the kernel configuration files (defconfig and tc0/defconfig). After modifying the configuration, rebuild the kernel and verify that the new configuration is applied.

Step 3: Configure FVP Memory and Interrupt Controllers

KVM requires specific memory and interrupt controller configurations to function correctly. The FVP must be configured to provide sufficient memory for the guest virtual machines and to support virtualization of the interrupt controller.

To configure the FVP memory, set the bp.secure_memory parameter to 0 to disable secure memory, as KVM does not require it. Additionally, ensure that the memory size is sufficient for both the host and guest operating systems. For example:

FVP_Base_RevC-2xA57x4 -C bp.secure_memory=0 -C cluster0.cpu0.CONFIG64=1 -C cluster0.cpu1.CONFIG64=1 -C cluster0.cpu2.CONFIG64=1 -C cluster0.cpu3.CONFIG64=1 -C bp.memory.size=4096

This command configures the FVP with 4GB of memory, which is sufficient for most KVM use cases.

For the interrupt controller, ensure that the GIC is configured to support virtualization. The FVP provides a GICv2 or GICv3 interrupt controller, both of which support virtualization. To enable GICv3, use the following parameter:

-C cluster0.gicv3.support=1

This command enables GICv3 support in the FVP, which is required for KVM on ARM64.

Step 4: Verify KVM Functionality

After configuring the FVP and rebuilding the Linux kernel, verify that KVM is functioning correctly. Use the following commands to check KVM status:

1. kvm-ok: This command checks if the system is capable of running KVM. If KVM is enabled, the output should indicate that KVM acceleration can be used.

2. dmesg | grep KVM: This command checks the kernel log for KVM-related messages. If KVM is initialized successfully, the output should include messages indicating that KVM is enabled and ready.

3. grep -E -o ‘vmx|svm’ /proc/cpuinfo: This command checks for CPU virtualization support. On ARM64, the output should include vmx or svm if the CPU supports virtualization extensions.

If these commands yield positive results, KVM is successfully enabled on the FVP. If not, review the FVP configuration and kernel build process to ensure that all steps were followed correctly.

Step 5: Debugging and Troubleshooting

If KVM is still not functioning, additional debugging may be required. Check the following areas for potential issues:

• Kernel Logs: Review the kernel logs (dmesg) for any errors or warnings related to KVM or virtualization. Look for messages indicating missing CPU features or misconfigured components.

• FVP Logs: Check the FVP output for any errors or warnings. The FVP may provide additional information about unsupported configurations or missing components.

• Kernel Configuration: Double-check the kernel configuration to ensure that all required KVM options are enabled. Use make menuconfig or make nconfig to review the kernel configuration interactively.

• FVP Parameters: Verify that all FVP command-line parameters are correct and that the FVP is configured to emulate a CPU model that supports virtualization extensions.

By following these steps, the KVM boot failure on FVP can be resolved, enabling ARM64 virtualization on the platform. This process requires careful attention to both the FVP configuration and the Linux kernel build process, but with the correct setup, KVM can be successfully enabled and verified on the ARM Fixed Virtual Platform.