Skip to content

The mem0 capability never gives Mem0 the session's date, so its facts carry the day of the run #12

Open

Description

@ttachon

Priority

medium

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • catalogue (benchmarks, agents, capabilities)
  • the standard (ports, kits, relay, runs)
  • CI, workflows and skills

Evidence

LongMemEval, oracle file, the 78 knowledge-update questions, the mem0 capability (run with a local FastEmbed embedder in place of OpenAI embeddings; its add path is the bundled one), reader and judge claude-sonnet-4-6: 66 of 78. In 7 of its 12 misses (0e4e4c46, 1cea1afa, 69fee5aa, 830ce83f, b01defab, cc5ded98, d7c942c3), the facts carry the day of the run instead of the session's.

Question 1cea1afa, "How many Instagram followers do I currently have?", expected 600; its sessions are dated 2023/05/21 and 2023/05/25. Recalled:

[2023/05/25 (Thu) 22:45] User's Instagram follower count grew from 500 to 600 followers by November 9, 2026
[2023/05/21 (Sun) 16:22] User is working on growing their Instagram presence and reached 500 followers around early November 2026, ...

The bracketed date is the adapter's and is right; "November 9, 2026" is the day the benchmark ran. The reader answered: "The current date is June 8, 2023, which is well before November 2026 when the follower milestones (500 and 600) are mentioned ... the follower counts of 500 and 600 are future projections, not current figures."

Cause: capability.py:114 passes the session date to mem.add only as metadata, which is printed back at recall (capability.py:133) but never reaches Mem0's extraction. In mem0ai 2.2.1 (the pinned version), the extraction prompt has an "Observation Date", described as when the conversation took place and "your ONLY temporal anchor". Memory.add builds that prompt without a date (mem0/memory/main.py:950), so _resolve_dates (mem0/configs/prompts.py:1007) sets it to today. add(timestamp=...) is refused in the open-source package as platform-only.

Screenshot

No response

Reproduction

  1. nullius standard run longmemeval mem0 --dataset longmemeval_oracle ... on questions whose sessions are older than today, for example 1cea1afa.
  2. Read the recall in the unit's trajectory: the facts are dated on the day of the run.

Expected

Mem0 extracts each session's facts as of that session's date, so "just reached 600 followers" in a session of 25 May 2023 is dated May 2023.

Acceptance

  • The mem0 capability gives Mem0 the session's date as its Observation Date for each add (for example by patching mem0.configs.prompts._resolve_dates, or freezing the clock at the session's date during mem.add, since timestamp is platform-only in the open-source package).
  • On 1cea1afa, the recalled facts carry May 2023, not the day of the run.

Dependencies and related

The cause sits between this adapter and mem0ai 2.2.1 (https://github.com/mem0ai/mem0), whose open-source add has no way to pass the Observation Date.

Activity

  1. aitianshi commented on Nov 10, 2026

    @aitianshi
    Collaborator

    Upstream

    The cause is a known bug in mem0, already tracked there. No new upstream issue was filed, to avoid a duplicate.

    What it means here

    Both open fixes read the observation date from metadata["timestamp"], then metadata["created_at"]. Our adapter (capability.py:114) passes the session date as metadata["date"], so even after the fix lands, a pin bump alone would not reach the extraction prompt.

    Keep open until a mem0 release carries the fix; then bump the pin, pass the session date as metadata["timestamp"], and rerun the 78 knowledge-update questions.

  2. self-assigned this
    on Nov 10, 2026
  3. aitianshi commented on Nov 10, 2026

    @aitianshi
    Collaborator

    @ttachon Extraction is supported in Nullius, but it needs to be supported in mem0 first.
    Have a try with the PR #29 + import a new mem0 version with the needed patch.
    You may keep this issue open waiting to see if the PR fixes your problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions