Repository navigation
Improve personal memory: extraction, reconcile, procedures, hardening - #51
Open
Aayush Kataria (aayush3011) wants to merge 11 commits into
Open
Aayush Kataria (aayush3011) wants to merge 11 commits into
Aayush Kataria (aayush3011) wants to merge 11 commits into
Conversation
Extraction - extract_memories-v3 is the default prompt. Extraction sees recent turns, the thread summary and the user's known facts as context. - Facts can record assertion, subject, attribute, entities and source turns. Reconcile - reconcile() folds duplicate facts and combines facts about one specific thing before the contradiction pass. - Passes accept a deadline, and the Function App stops them before the function timeout. Procedural memory - Procedures are learned incrementally from evidence as instructions, habits and skills (breaking reshape, no migration). Search - Optional preference and procedure slots in search_cosmos(). - Keyword and hybrid thread searches no longer send a partition key (Azure/azure-sdk-for-python#42241). Hardening - Credentials are redacted on write, and deleting a memory removes the records folded into it. - Logs and Function App failures no longer quote exception, query, memory or model text. - Empty and refused model replies are handled, plus smaller Function App config fixes. Infra - Bicep: the summaries container gets the vector and full-text index, and dedupPoolSize, activityTimeBudgetSeconds, chatCapacity and embeddingCapacity are new parameters. See CHANGELOG.md for the breaking changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Reduce the extraction note to the new prompt and the variable that selects an earlier one. - Shorten the procedural memory heading. - Drop the supersedes note, the keyword-search bug-fix entry and the requirements pin entry. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Tool-only replies with blank content are incorrectly retried as transient empty replies, causing persistent failures to defer extraction instead of being treated as non-retryable.
1 open issue
What changed in this PR
This PR improves the Agent Memory Toolkit’s personal-memory pipeline, from contextual extraction and reconciliation to evidence-based procedures, safer storage, and Function App processing.
Changes:
- Adds contextual extraction, duplicate folding and combining, procedural learning, and optional search slots.
- Hardens credential handling, logs, model-response handling, and time-budgeted processing.
- Updates infrastructure, configuration, documentation, and regression coverage.
| File | Description |
|---|---|
tests/unit/test_utils.py |
Tests preferred-region resolution. |
tests/unit/test_thresholds.py |
Expands boolean-token coverage. |
tests/unit/test_reconcile.py |
Adapts reconciliation tests to dedup counts. |
tests/unit/test_query_builder.py |
Tests parameterized condition support. |
tests/unit/test_procedural_synthesis.py |
Removes legacy procedural-synthesis tests. |
tests/unit/test_no_exception_text_in_logs.py |
Adds source checks against exception-text logging. |
tests/unit/test_logging.py |
Updates assertions for safe exception details. |
tests/unit/test_log_describe.py |
Tests privacy-safe diagnostic descriptions. |
tests/unit/test_fact_merge_fields.py |
Tests merge provenance and retirement reasons. |
tests/unit/test_cosmos_memory_client.py |
Tests regions, counter expiry, and search defaults. |
tests/unit/test_combine_thresholds.py |
Pins consolidation limits. |
tests/unit/test_chat.py |
Tests empty replies, terminal failures, and safe logs. |
tests/unit/store/test_secret_redaction.py |
Tests redaction before embedding and persistence. |
tests/unit/store/test_search_routing.py |
Tests keyword and vector query routing. |
tests/unit/store/test_memory_store.py |
Updates thread-scoped search expectations. |
tests/unit/services/test_prompty_loader.py |
Tests prompt versions and contextual inputs. |
tests/unit/services/test_procedural_retrieval.py |
Updates retrieval tests for the procedure schema. |
tests/unit/services/test_pipeline_service.py |
Updates reconciliation fixtures and removes legacy procedure tests. |
tests/unit/services/test_parse_llm_json.py |
Checks parsing errors omit model output. |
tests/unit/services/test_extraction_batching.py |
Tests terminal and retryable reply classification. |
tests/unit/services/test_extract_prompt_selection.py |
Tests v3 defaults and earlier-prompt overrides. |
tests/unit/services/test_extract_dry.py |
Adapts extraction fixtures for context and retries. |
tests/unit/services/test_embedding_redaction.py |
Tests sanitized embedding inputs and summaries. |
tests/unit/services/test_absorbed.py |
Tests cascading deletion of folded records. |
tests/unit/prompts/test_memory_writing_prompts.py |
Checks secret-handling rules and prompt rendering. |
tests/unit/prompts/test_consolidation_prompts.py |
Tests consolidation prompt contracts. |
tests/unit/processors/test_inprocess.py |
Tests combined-fact reconciliation counts. |
tests/unit/function_app/test_synthesize_procedural_orchestrator.py |
Tests deadlines and reduced activity outputs. |
tests/unit/function_app/test_orchestrators.py |
Tests shared reconciliation deadlines. |
tests/unit/function_app/test_failure_logs.py |
Tests safe trigger and orchestration failures. |
tests/unit/function_app/test_config_time_budget.py |
Tests activity-budget defaults and bounds. |
tests/unit/function_app/test_config_bool_parity.py |
Checks Function App and SDK boolean parity. |
tests/unit/function_app/test_config_batch_size.py |
Tests rejection of empty batch windows. |
tests/unit/aio/test_reconcile_telemetry.py |
Updates async reconciliation result expectations. |
tests/unit/aio/test_procedural_synthesis.py |
Removes legacy async procedure tests. |
tests/unit/aio/test_cosmos_memory_client.py |
Tests async preferred regions and counter expiry. |
tests/unit/aio/test_chat.py |
Tests async reply handling and safe logs. |
tests/unit/aio/store/test_secret_redaction.py |
Tests async write-path redaction. |
tests/unit/aio/store/test_search_routing.py |
Tests async keyword and vector routing. |
tests/unit/aio/store/test_memory_store.py |
Updates async thread-search expectations. |
tests/unit/aio/services/test_procedural_retrieval_async.py |
Updates async procedure retrieval tests. |
tests/unit/aio/processors/test_inprocess.py |
Tests async combined-fact counts. |
Samples/Scenarios/scenario_tagging_and_filtering.py |
Formatting only. |
Samples/Scenarios/scenario_remote_processor.py |
Formatting only. |
Samples/Scenarios/scenario_remote_processor_async.py |
Formatting only. |
Samples/Scenarios/scenario_rag_with_memory.py |
Formatting only. |
Samples/Scenarios/scenario_multi_agent.py |
Formatting only. |
Samples/Scenarios/scenario_memory_reconciliation.py |
Updates reconciliation guidance and cascade-aware cleanup. |
Samples/Scenarios/scenario_customer_support.py |
Formatting only. |
Samples/Scenarios/scenario_counter_tuning.py |
Formatting only. |
Samples/Scenarios/scenario_chat_memory.py |
Formatting only. |
Samples/Quickstarts/quickstart_local.py |
Formatting only. |
Samples/Processing/processing_thread_summary.py |
Formatting only. |
Samples/Notebooks/Demo_function_app.ipynb |
Notebook formatting only. |
Samples/Notebooks/Demo_function_app_async.ipynb |
Notebook formatting only. |
Samples/Advanced/advanced_search_patterns.py |
Formatting only. |
infra/README.md |
Documents indexes, capacity, and processing controls. |
infra/modules/functions.bicep |
Passes reconciliation and time-budget settings. |
infra/modules/cosmos.bicep |
Adds summary vector and full-text indexes. |
infra/main.parameters.json |
Binds new deployment environment variables. |
infra/main.bicep |
Exposes processing budgets and model capacity. |
function_app/triggers/change_feed.py |
Sanitizes trigger logs and propagated failures. |
function_app/shared/error_redaction.py |
Adds safe activity-failure wrappers. |
function_app/shared/counters.py |
Replaces exception text with safe diagnostics. |
function_app/shared/config.py |
Shares boolean parsing and validates processing settings. |
function_app/orchestrators/user_summary.py |
Wraps summary activities with error redaction. |
function_app/orchestrators/thread_summary.py |
Wraps thread activities with error redaction. |
function_app/orchestrators/synthesize_procedural.py |
Adds deadlines and reduced result payloads. |
function_app/orchestrators/extract_memories.py |
Adds safe failures and reconciliation deadlines. |
function_app/orchestrators/extract_episodes.py |
Wraps episode extraction with error redaction. |
function_app/local.settings.json.template |
Adds reconciliation pool and budget settings. |
Docs/troubleshooting.md |
Explains search filtering, safe failures, and budgets. |
Docs/operations.md |
Documents expiry, deletion, and pipeline controls. |
Docs/local_testing.md |
Documents default counter expiry. |
Docs/azure_testing.md |
Updates procedure and reconciliation validation guidance. |
Docs/architecture/orchestrators.md |
Documents processing passes, cursors, and redaction boundaries. |
CHANGELOG.md |
Summarizes features, fixes, and breaking changes. |
azure/cosmos/agent_memory/thresholds.py |
Defines context, consolidation, and counter-expiry limits. |
azure/cosmos/agent_memory/store/_search_helpers.py |
Adds result fields, procedure filters, and keyword routing. |
azure/cosmos/agent_memory/services/_absorbed.py |
Implements cascading deletion of folded records. |
azure/cosmos/agent_memory/prompts/user_summary.prompty |
Adds secret-exclusion instructions. |
azure/cosmos/agent_memory/prompts/user_summary_update.prompty |
Forbids retaining secrets in updated profiles. |
azure/cosmos/agent_memory/prompts/synthesize_procedural.prompty |
Removes the legacy synthesis prompt. |
azure/cosmos/agent_memory/prompts/summarize.prompty |
Adds secret-exclusion instructions. |
azure/cosmos/agent_memory/prompts/summarize_update.prompty |
Forbids retaining secrets in updated summaries. |
azure/cosmos/agent_memory/prompts/extract_memories.prompty |
Adds credential-exclusion instructions. |
azure/cosmos/agent_memory/prompts/extract_memories-v2.prompty |
Adds credential-exclusion instructions. |
azure/cosmos/agent_memory/prompts/extract_episode.prompty |
Prioritizes secret exclusion over exact capture. |
azure/cosmos/agent_memory/prompts/dedup.prompty |
Corrects the explanation of duplicate-pass ordering. |
azure/cosmos/agent_memory/prompts/dedup_verify.prompty |
Adds conservative duplicate and containment checks. |
azure/cosmos/agent_memory/prompts/combine_memories.prompty |
Adds detail-preserving memory combination. |
azure/cosmos/agent_memory/processors/inprocess.py |
Includes combined facts in reconciliation counts. |
azure/cosmos/agent_memory/chat.py |
Adds empty-reply retries and safe failure logging. |
azure/cosmos/agent_memory/auto_trigger.py |
Sanitizes automatic-processing failure reports. |
azure/cosmos/agent_memory/aio/processors/inprocess.py |
Mirrors combined-fact counting asynchronously. |
azure/cosmos/agent_memory/aio/chat.py |
Mirrors reply handling and safe logging asynchronously. |
azure/cosmos/agent_memory/aio/auto_trigger.py |
Sanitizes async automatic-processing failures. |
azure/cosmos/agent_memory/_utils.py |
Resolves ordered Cosmos read regions. |
azure/cosmos/agent_memory/_slots.py |
Plans and fills typed search-result slots. |
azure/cosmos/agent_memory/_query_builder.py |
Supports prebuilt parameterized conditions. |
azure/cosmos/agent_memory/_counters.py |
Sanitizes failure-stamping diagnostics. |
azure/cosmos/agent_memory/_base/base_client.py |
Stores resolved preferred-region configuration. |
.env.template |
Documents region and extraction-prompt overrides. |
Review effort: Balanced
A reply that finished with tool calls but had empty or whitespace-only content was retried as a transient empty reply. It now raises unexpected_tool_response, like a reply with no content, so extraction does not keep retrying it. This applies to the sync and async chat clients. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Improves the personal memory pipeline: smarter extraction, duplicate folding in
reconcile(), evidence-based procedural memory, safer writes and logs, and the matching Bicep changes. User-facing notes are inCHANGELOG.md.What changed
extract_memories-v3is the default. It sees recent turns, the thread summary and the user's known facts as context, and facts can recordassertion,subject,attribute,entitiesand source turns.deadline, and the Function App stops them before the function timeout (ACTIVITY_TIME_BUDGET_SECONDS).build_procedural_context()returns the standing rules.memories_summariesgets the vector and full-text index, anddedupPoolSize,activityTimeBudgetSeconds,chatCapacityandembeddingCapacityare new parameters.Breaking changes
exc_msgis nowexc_detail, and a failed Function App activity reports only a short code.memories_summariescontainer created again, because Cosmos DB sets a vector policy only at creation.Validation
ruff checkandruff format --checkare clean, and CI is green.azd provision --previewagainst an environment created from the old template.function_app/requirements.txtstill pins SDK 0.3.0b2. Bump it with the SDK release before deploying the updated Function App code.