GICv3 Group 1 Interrupt Enable Bit Not Propagating to ICC_IGRPEN1_EL3.EnableGrp1NS
The issue revolves around the inability of the Non-secure ICC_IGRPEN1_EL1.Enable bit to propagate its value to the ICC_IGRPEN1_EL3.EnableGrp1NS bit when written from EL1. According to the ARM GICv3 specification, the Non-secure ICC_IGRPEN1_EL1.Enable bit is a read/write alias of the ICC_IGRPEN1_EL3.EnableGrp1NS bit. This means that any write to ICC_IGRPEN1_EL1.Enable should directly reflect in ICC_IGRPEN1_EL3.EnableGrp1NS. However, in the observed scenario, writing to ICC_IGRPEN1_EL1.Enable from EL1 updates the local view of the bit but fails to update ICC_IGRPEN1_EL3.EnableGrp1NS, leaving group 1 interrupts effectively disabled for EL1.
The problem manifests in a baremetal application running atop ARM Trusted Firmware-A (TF-A) on the ARM Foundation Platform. The application transitions from EL2 to EL1 upon startup, and attempts to enable group 1 interrupts by writing to ICC_IGRPEN1_EL1.Enable. Despite the write operation appearing successful when viewed from EL1, the corresponding ICC_IGRPEN1_EL3.EnableGrp1NS bit remains unchanged. This discrepancy prevents group 1 interrupts from being enabled at EL1, leading to a failure in interrupt handling.
Interestingly, the issue can be temporarily resolved by making an SMC call to TF-A after setting ICC_IGRPEN1_EL1.Enable. Upon returning from the SMC call, the ICC_IGRPEN1_EL3.EnableGrp1NS bit is correctly updated, and group 1 interrupts become enabled. This suggests that the act of entering EL3 via the SMC call triggers the update of the ICC_IGRPEN1_EL3.EnableGrp1NS bit. Additionally, when the ICC_IGRPEN1_EL1.Enable bit is set through the debugger register view, regardless of the exception level, the ICC_IGRPEN1_EL3.EnableGrp1NS bit is correctly updated. This behavior indicates that the issue may be related to the specific context in which the write operation is performed from EL1.
EL1 to EL3 State Transition and Virtualization Configuration Impact
The root cause of the issue lies in the interaction between the exception level state transitions and the virtualization configuration of the GICv3 CPU interface. The ARM GICv3 architecture defines a complex relationship between the physical and virtual views of the interrupt controller registers, particularly when transitioning between different exception levels. In this case, the failure of the ICC_IGRPEN1_EL1.Enable bit to propagate to ICC_IGRPEN1_EL3.EnableGrp1NS can be attributed to several potential causes related to the state of the system during the write operation.
One possible cause is the virtualization configuration at EL2. The GICv3 specification states that virtual accesses to the ICC_IGRPEN1_EL1 register update the ICH_VMCR_EL2.VENG1 bit. However, in the observed scenario, the ICH_VMCR_EL2.VENG1 bit does not change when ICC_IGRPEN1_EL1.Enable is written from EL1. This suggests that the virtualization configuration at EL2 may not be correctly propagating the write operation to the physical register. This could be due to an incomplete or incorrect setup of the virtual GIC at EL2, leading to a failure in the translation of the virtual access to the physical register.
Another potential cause is the state of the system during the write operation. When the system is in EL1, the GICv3 CPU interface may not be fully aware of the need to propagate the write to the EL3 register. This could be due to a lack of synchronization between the exception levels, particularly if the system has not been properly configured to handle such transitions. The fact that the issue is resolved by making an SMC call suggests that the act of entering EL3 forces the system to synchronize the state of the interrupt controller registers, thereby updating the ICC_IGRPEN1_EL3.EnableGrp1NS bit.
Additionally, the debugger’s ability to correctly update the ICC_IGRPEN1_EL3.EnableGrp1NS bit regardless of the exception level indicates that the issue may be related to the specific context in which the write operation is performed. When the debugger writes to the register, it may bypass certain checks or synchronization mechanisms that are in place when the write is performed from EL1. This could explain why the debugger is able to correctly update the bit while the EL1 write operation fails to do so.
Implementing Exception Level Synchronization and Virtual GIC Configuration
To resolve the issue, it is necessary to ensure that the system is correctly configured to handle the propagation of the ICC_IGRPEN1_EL1.Enable bit to the ICC_IGRPEN1_EL3.EnableGrp1NS bit when writing from EL1. This involves several steps, including configuring the virtual GIC at EL2, ensuring proper synchronization between exception levels, and verifying the state of the system during the write operation.
First, it is essential to verify that the virtual GIC at EL2 is correctly configured. This includes ensuring that the ICH_VMCR_EL2.VENG1 bit is properly set up to reflect the state of the ICC_IGRPEN1_EL1.Enable bit. If the virtual GIC is not correctly configured, the write operation from EL1 may not be properly propagated to the physical register. This can be achieved by carefully reviewing the virtualization configuration at EL2 and ensuring that all necessary bits are set according to the GICv3 specification.
Next, it is important to ensure that the system is properly synchronized between exception levels. This can be achieved by implementing the necessary synchronization mechanisms, such as barriers or explicit state transitions, to ensure that the state of the interrupt controller registers is correctly propagated across exception levels. In particular, it may be necessary to explicitly transition to EL3 and back to EL1 to force the system to synchronize the state of the ICC_IGRPEN1_EL3.EnableGrp1NS bit. This can be done by making an SMC call, as observed in the initial scenario, but it is important to understand why this is necessary and whether it can be avoided by properly configuring the system.
Finally, it is crucial to verify the state of the system during the write operation. This includes checking the state of the exception levels, the virtualization configuration, and the state of the interrupt controller registers. By carefully analyzing the state of the system, it is possible to identify any potential issues that may be preventing the write operation from being properly propagated. This may involve using a debugger to step through the code and inspect the state of the system at various points during the write operation.
In conclusion, the issue with the ICC_IGRPEN1_EL1.Enable bit not propagating to the ICC_IGRPEN1_EL3.EnableGrp1NS bit when written from EL1 is likely due to a combination of virtualization configuration issues and a lack of synchronization between exception levels. By carefully configuring the virtual GIC at EL2, ensuring proper synchronization between exception levels, and verifying the state of the system during the write operation, it is possible to resolve the issue and ensure that group 1 interrupts are correctly enabled at EL1.
