[dhalbert]

Code and measurements by Claude, reviewed and edited by @dhalbert.

Improve free RAM utilization for the GC heap on zephyr-cp, and allow larger objects.

Problems

Some zephyr-cp boards assign the entire amount of unused RAM to Zephyr's malloc arena. This includes nordic and RP2. On these boards, port_heap_get_largest_free_size() always returned 64 kB. The CircuitPython VM heap grows by adding areas from the arena, and it uses that number as the size limit for a new area. So on these boards:

• No single Python object could be larger than about 63 kB.
• Once the heap reached 64 kB, every new area was exactly 64 kB, which could leave up to 64 kB of the arena unusable, because the last chunk might be just under 64 kB and would be rejected.
• Because the GC heap did not take Zephyr's own runtime allocations into account, such as for BLE pairing and TLS, those allocations might or might not succeed, depending on how much free RAM was left at the time. Letting the GC heap use the whole arena, without a reserve, made BLE pairing fail. See Testing.

A more minor problem was that gc.mem_free() was inaccurate on those zephyr-cp boards. It adds the free space in the existing GC areas to the value of port_heap_get_largest_free_size(). Because that value was a fixed 64 kB, gc.mem_free() never read below 64 kB, even when no free RAM was available.

Changes

• Changed port_heap_get_largest_free_size():
  • Now it reports a value actually based on the arena's free bytes, instead of just the fixed 64 kB value. To do this we turn on Zephyr's CONFIG_SYS_HEAP_RUNTIME_STATS option.
  • It also sets aside a reserve for Zephyr's own allocations. The size is a new setting: CONFIG_CIRCUITPY_ZEPHYR_MALLOC_RESERVE. It's 4096 by default, and a board can change it in its board.conf. This is enough for BLE. TLS needs much more, about 37 kB per connection, so it can still fail when the heap is nearly full. We might adjust this in the future.
• gc_try_add_heap() in py/gc.c asks for a new area as large as the whole heap so far, capped by port_heap_get_largest_free_size(). Previously, if that allocation failed, it gave up, even when a smaller area would have fit. Now it retries, halving the size each time, down to the size the failed allocation needs plus area overhead.

Note that the info enabled by CONFIG_SYS_HEAP_RUNTIME_STATS only reports total free bytes, which could be the sum of fragmented regions, not the arena's largest free block. So when it is fragmented the first request could be too large, and it will try again. On TLSF ports like espressif, where the size limit is an exact free-block size, the first request should not fail and the retry loop does not run.

gc.mem_free() is now more accurate on zephyr-cp boards, with no change to its code, because port_heap_get_largest_free_size() now reports the arena's actual free space. The value is still somewhat approximate because:

• It counts all free arena bytes as room for Python data, but a new area spends about 3% of its size on tables.
• It can't show fragmentation, so it doesn't tell whether one large object will fit.
• On boards with several RAM regions, it will count only the larger of the TLSF pools' largest free block and the arena's free bytes, not all free memory.

Testing

• Zephyr Feather nRF52840: filled the Python heap until MemoryError, then added a BLE service, advertised, and paired from a Linux host.
  • Without the 4 kB reserve, the BLE pairing failed.
  • With the 4 kB reserve, it worked.
• Zephyr Pico 2 W: made one TLS connection to www.adafruit.com while holding a given amount of Python data:

Python data	TLS
none	OK
100 kB	OK
full heap, no free heap	MemoryError, no crash
full, then freed	OK

• Zephyr Pico 2 W: gc.mem_free() went from about 285 kB at the start to about 6 kB with the heap full.
• make test on native_sim passed.

A regression test was also done on the classic raspberrypi port, Pico 2 W, since the py/gc.c change applies to all ports. The same TLS test worked as above, and TLS also worked while holding 130 to 255 kB of Python data.

Later: gentler heap growth

gc_try_add_heap() makes each new area as large as the whole heap so far, capped by port_heap_get_largest_free_size(). So the area sizes double, and near the end of RAM one growth step can take most of what's left in the arena, even when Python doesn't need that much. Then TLS can fail even though the GC heap has plenty of free space.

A fix, not in this PR, is to cap each new area at half of what's left in the arena, unless the failed allocation needs more. Early growth would be unchanged, but near the end the arena's free space would shrink gradually instead of all at once.

Later: more detailed heap reporting

I have an idea for a more comprehensive supervisor.mem_info() that would report on all the GC heap pools, giving the free space and largest available space in each one, and how much more free RAM is still available for pools.

Alternative: one TLSF heap for everything

An alternative approach, not done here, is to replace Zephyr's malloc() with CircuitPython's TLSF allocator, so that the VM heap and Zephyr's own allocations share one heap across all RAM regions. That is similar to how espressif works: ESP-IDF and CircuitPython already share one TLSF-based heap. This approach would:

• Give the exact size of the largest free block, so no retry would be needed.
• Let free memory in any region serve either the VM or Zephyr, which matters on boards with RAM in regions outside zephyr,sram, like the i.MX RT1011 boards, once they move to zephyr-cp.

This PR doesn't do that, because replacing libc malloc() is a larger and riskier change. Every Zephyr allocation would go through it, and it would have to be set up before main() and be thread-safe. It could be considered in the future, for example if the arena on WiFi boards fragments enough to cause early MemoryErrors.

[tannewt]

Thank you!