ARM Cortex-A35 Cache Coherency and System-Wide Flush for Multi-Core Systems

Cache Coherency Challenges in Multi-Core ARM Cortex-A35 Systems

In multi-core ARM Cortex-A35 systems, such as the i.MX8DX platform, cache coherency is a critical aspect of ensuring data integrity and consistency across cores. The Cortex-A35 cores (Core 0 and Core 1) share a common L2 cache but maintain separate L1 caches. This architecture introduces complexities when one core modifies data that may be cached by another core, especially when the data needs to be accessed by a different processor, such as the Cortex-M4 core in the i.MX8DX. The primary challenge arises when Core 0 modifies a memory region that is also cached by Core 1. If Core 0 performs a cache clean and invalidate operation (DC CIVAC) on its L1 and L2 caches, it is unclear whether this operation will propagate to Core 1’s L1 cache. This uncertainty can lead to data inconsistency, particularly when the Cortex-M4 core attempts to access the modified memory region.

The Cortex-A35 architecture supports cache coherency between its cores, but this coherency is limited to the L1 caches of the Cortex-A35 cores. The Cortex-M4, being a separate processor, does not participate in this coherency mechanism. Therefore, any data modified by the Cortex-A35 cores and intended for the Cortex-M4 must be explicitly flushed from the Cortex-A35 caches to ensure the Cortex-M4 accesses the most recent data. The key question is whether a cache maintenance operation performed by Core 0 can propagate to Core 1’s L1 cache and whether this operation is sufficient to ensure data consistency across the entire system.

Broadcast Cache Maintenance Operations and Their Limitations

The ARMv8-A architecture, which the Cortex-A35 implements, provides cache maintenance operations that can be broadcasted to other cores. The DC CIVAC (Data Cache Clean and Invalidate by Virtual Address to Point of Coherency) instruction is one such operation. When executed, this instruction cleans and invalidates the cache line corresponding to the specified virtual address in the L1 and L2 caches of the executing core. Importantly, the ARMv8-A architecture specifies that this operation is broadcasted to other cores, meaning that it should also clean and invalidate the corresponding cache lines in the L1 caches of other Cortex-A35 cores.

However, the effectiveness of this broadcast mechanism depends on the implementation of the cache coherency protocol in the specific processor. In the case of the Cortex-A35, the broadcast mechanism ensures that the cache maintenance operation propagates to the L1 caches of other Cortex-A35 cores. This means that if Core 0 executes a DC CIVAC operation, Core 1’s L1 cache will also be cleaned and invalidated for the specified address range. However, this mechanism does not extend to the Cortex-M4 core, as it operates outside the cache coherency domain of the Cortex-A35 cores.

The L2 cache, being shared between the Cortex-A35 cores, is also affected by the DC CIVAC operation. Cleaning and invalidating the L2 cache ensures that any modified data is written back to main memory, making it accessible to the Cortex-M4. However, the Cortex-M4 does not have direct access to the L2 cache, so the data must be explicitly flushed to main memory. This raises the question of whether cleaning and invalidating the L2 cache alone is sufficient or if the L1 caches of both Cortex-A35 cores must also be cleaned and invalidated.

Implementing System-Wide Cache Flush and Testing Strategies

To ensure data consistency across the entire system, including the Cortex-M4 core, a system-wide cache flush must be performed. This involves cleaning and invalidating the L1 and L2 caches of both Cortex-A35 cores. The following steps outline the recommended approach:

1. Execute DC CIVAC on Core 0: Core 0 should execute the DC CIVAC instruction for the specified address range. This operation will clean and invalidate the corresponding cache lines in Core 0’s L1 cache and the shared L2 cache. Due to the broadcast mechanism, Core 1’s L1 cache will also be cleaned and invalidated for the same address range.

2. Verify Cache Maintenance Operation Propagation: To ensure that the cache maintenance operation has propagated to Core 1’s L1 cache, a synchronization mechanism should be employed. This can be achieved using a Data Synchronization Barrier (DSB) instruction, which ensures that all cache maintenance operations are completed before proceeding.

3. Flush L2 Cache to Main Memory: Cleaning and invalidating the L2 cache ensures that any modified data is written back to main memory. This step is crucial for making the data accessible to the Cortex-M4 core. The DC CIVAC operation performed by Core 0 will handle this for the shared L2 cache.

4. Testing the Cache Flush Mechanism: To verify the effectiveness of the cache flush mechanism, a test scenario can be implemented. Core 0 modifies a specific memory region, performs the cache flush, and then the Cortex-M4 reads the same memory region. The Cortex-M4 should observe the modified data, confirming that the cache flush was successful. If the Cortex-M4 reads stale data, it indicates that the cache flush mechanism was not fully effective, and further investigation is required.

5. Handling Cortex-M4 Cache: Since the Cortex-M4 does not participate in the cache coherency mechanism of the Cortex-A35 cores, its cache must be managed separately. If the Cortex-M4 has its own cache, it should be invalidated before accessing the modified memory region to ensure it fetches the most recent data from main memory.

The following table summarizes the cache maintenance operations and their effects:

By following these steps, a system-wide cache flush can be effectively implemented, ensuring data consistency across the Cortex-A35 cores and the Cortex-M4 core. Testing the mechanism is essential to confirm its correctness and to identify any potential issues in the cache maintenance operations.