Repository navigation
Conversation
Add extended telemetry sections to Memory Window 2 (TEL2 v2.0) and
Memory Window 3 (PMW2 v2.0) to support real-time host monitoring tools
such as sof-top without invasive IPC polling during active audio
streaming:
- Memory Window 2 (TEL2, 0x0100..0x0AFF):
- Directory header with seqlock, section descriptors, and timestamps
- Cache-line-aligned hot-path counters (LL overruns, DP deadline
misses, total/per-pipeline XRUNs, IPC latency, active LL tasks)
- Per-gateway hot counters (16 slots x 8B) and cold descriptors
(DAI type/index/direction/DMA channel/buffer size)
- Memory allocators section (L2 HP-SRAM heaps, LPSRAM, fast-get pool,
vpage physical pool, blob pool, shared memory, Zephyr system heap)
- Virtual regions section (up to 20 vregions with committed/allocated
bytes, free block counts, and peak HWM)
- Pipelines and per-core scheduler section (up to 16 pipelines and 4
cores with active KCPS budget, LL/DP task counts, and XRUN status)
- Zephyr RTOS threads section (up to 24 threads with core affinity,
priority, state, stack size/used/HWM, and runtime cycles)
- Memory Window 3 (PMW2):
- Extend performance measurements window with per-module cold resource
table (heap usage, heap HWM, configured CPC) and per-module hot
counters (CPC violations and XRUN counts)
- Fix telemetry_update() to accumulate elapsed active LL tick cycles
instead of inter-tick wall-clock ccount deltas, and reset idle core
tick metrics when active LL task count drops to zero.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Telemetry still permits MW3 table overflow and unsynchronized collection of objects that may be freed concurrently, creating critical memory-corruption and use-after-free risks.
17 open issues
What changed in this PR
Extends SOF’s shared telemetry windows to support host monitoring tools such as sof-top without additional IPC polling during audio streaming.
Changes:
- Adds TEL2 sections for memory, virtual regions, pipelines, gateways, and Zephyr threads.
- Adds PMW2 module resource records and hot counters, and relocates I/O performance data.
- Changes systick accounting to measure active LL execution cycles and adds idle-metric resets.
| File | Description |
|---|---|
zephyr/lib/vregion.c |
Registers virtual regions and collects their statistics. |
zephyr/lib/vpage.c |
Tracks page allocation usage and failures. |
zephyr/lib/regions_mm.c |
Exports virtual-heap block statistics. |
zephyr/lib/fast-get.c |
Collects cache-pool usage and allocation counters. |
zephyr/lib/alloc.c |
Collects heap usage, failures, and SRAM statistics. |
zephyr/include/sof/lib/regions_mm.h |
Declares the virtual-heap telemetry interface. |
src/ipc/ipc4/helper.c |
Adds pipeline lifecycle notifications. |
src/include/sof/debug/telemetry/telemetry.h |
Defines TEL2 layouts and collection interfaces. |
src/include/sof/debug/telemetry/performance_monitor.h |
Defines PMW2 partitions and module records. |
src/debug/telemetry/telemetry.c |
Implements collection, publication, and revised tick accounting. |
src/debug/telemetry/performance_monitor.c |
Implements PMW2 resource tables and I/O storage. |
src/debug/telemetry/Kconfig |
Adds telemetry feature and capacity settings. |
src/audio/pipeline/pipeline-xrun.c |
Records pipeline and component XRUN events. |
src/audio/pipeline/pipeline-stream.c |
Notifies telemetry when triggers are submitted. |
src/audio/module_adapter/module/generic.c |
Publishes module resource usage after allocations. |
src/audio/host-zephyr.c |
Adds host gateway registration and transfer updates. |
src/audio/dai-zephyr.c |
Adds DAI gateway registration and transfer updates. |
src/audio/component.c |
Updates module resources and CPC violation counters. |
Review effort: Balanced
Comment on lines
+108
to
+110
| range 32 186 | ||
| default 88 if SOF_TELEMETRY_MODULE_RESOURCES | ||
| default 186 |
Comment on lines
+173
to
+174
| list_for_item(clist, &ipc->comp_list) { | ||
| struct ipc_comp_dev *icd = container_of(clist, struct ipc_comp_dev, list); |
Comment on lines
+320
to
+321
| if (!s->is_host && s->dd) { | ||
| struct dai_data *dd = s->dd; |
Comment on lines
+647
to
+649
| for (int i = 0; i < SOF_TEL_MAX_VREGIONS; i++) { | ||
| struct vregion *vr = tel_vregions[i]; | ||
| struct sof_tel_vregion_info *info = &vsec->vregions[i]; |
| res->heap_high_water_mark = res->heap_usage; | ||
|
|
||
| k_mutex_unlock(&mod_res_lock); | ||
| perf_data_update_module_resources(mod->dev); |
| if (!hot) | ||
| return; | ||
|
|
||
| hot->total_kb_or_frames = (uint32_t)(hd->total_data_processed >> 10); |
|
|
||
| tsec->thread_count = 0; | ||
| tsec->dropped_threads = 0; | ||
| k_thread_foreach_unlocked(telemetry_thread_cb, tsec); |
Comment on lines
+926
to
+930
| if (prid == 0) { | ||
| #ifdef CONFIG_SOF_TELEMETRY_HOT_CACHE_STAGING | ||
| if (hot_line0_dirty) { | ||
| wnd_data->hot_line0 = hot_line0_cache; | ||
| hot_line0_dirty = false; |
| uint8_t index; /* dai_index or stream_tag */ | ||
| uint8_t direction; /* PLAYBACK=0, CAPTURE=1 */ | ||
| int8_t dma_chan; /* chan_index */ | ||
| uint16_t comp_id; /* Owning Copier component ID */ |
Comment on lines
+985
to
+987
| mem->sram.hp_sram_free_pages = | ||
| hp_banks ? (uint16_t)((mem->sram.hp_sram_total_pages * active_hp_banks) / hp_banks) | ||
| : mem->sram.hp_sram_total_pages; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds extended telemetry sections to Memory Window 2 (
TEL2v2.0) and Memory Window 3 (PMW2v2.0) to support real-time host monitoring tools such assof-top(see thesofproject/sof-utils#2) without invasive IPC polling during active audio streaming.Memory Window 2 (
TEL2,0x0100..0x0AFF)0x0100, 64B):TEL2magic, v2.0, seqlock counter, microsecond timestamp, and 5 section descriptors.0x0140..0x023F):0x0140, 64B): per-core LL overruns, DP deadline misses, total/per-pipeline XRUNs, IPC count/max latency, and active LL task count.0x0180..0x01FF, 16 x 8B): per-gateway active bytes/frames, XRUN counts, and error flags.0x0240, 256B):vpagepool, blob pool, shared memory, and Zephyr system heap (used, free, peak HWM, and largest free block).0x0340, 512B):vregiondescriptors reporting virtual size, committed bytes, allocated bytes, free blocks, and peak HWM.0x0540, 384B):0x06C0, 256B):0x07C0, 832B):Memory Window 3 (
PMW2)PMW2trailer, per-module cold resource table (sof_perf_mod_cold_item: heap usage, heap HWM, configured CPC), and per-module hot counters (sof_perf_mod_hot_item: CPC violations and XRUN counts).Systick / Idle Core Cycle Fix
telemetry_update()to accumulateelapsedactive LL tick cycles (current_stamp - begin_stamp) rather than inter-tick wall-clockccountdeltas, and resets idle core tick metrics whenll_task_countdrops to zero.