Repository navigation
Description
Priority
medium
Components
- app/nullius (backend)
- app/web (routes)
- templates and static files
- docs and the site
- catalogue (benchmarks, agents, capabilities)
- the standard (ports, kits, relay, runs)
- CI, workflows and skills
Evidence
LongMemEval, oracle file, the 78 knowledge-update questions, the mem0 capability (run with a local FastEmbed embedder in place of OpenAI embeddings; its add path is the bundled one), reader and judge claude-sonnet-4-6: 66 of 78. In 7 of its 12 misses (0e4e4c46, 1cea1afa, 69fee5aa, 830ce83f, b01defab, cc5ded98, d7c942c3), the facts carry the day of the run instead of the session's.
Question 1cea1afa, "How many Instagram followers do I currently have?", expected 600; its sessions are dated 2023/05/21 and 2023/05/25. Recalled:
[2023/05/25 (Thu) 22:45] User's Instagram follower count grew from 500 to 600 followers by November 9, 2026
[2023/05/21 (Sun) 16:22] User is working on growing their Instagram presence and reached 500 followers around early November 2026, ...
The bracketed date is the adapter's and is right; "November 9, 2026" is the day the benchmark ran. The reader answered: "The current date is June 8, 2023, which is well before November 2026 when the follower milestones (500 and 600) are mentioned ... the follower counts of 500 and 600 are future projections, not current figures."
Cause: capability.py:114 passes the session date to mem.add only as metadata, which is printed back at recall (capability.py:133) but never reaches Mem0's extraction. In mem0ai 2.2.1 (the pinned version), the extraction prompt has an "Observation Date", described as when the conversation took place and "your ONLY temporal anchor". Memory.add builds that prompt without a date (mem0/memory/main.py:950), so _resolve_dates (mem0/configs/prompts.py:1007) sets it to today. add(timestamp=...) is refused in the open-source package as platform-only.
Screenshot
No response
Reproduction
nullius standard run longmemeval mem0 --dataset longmemeval_oracle ...on questions whose sessions are older than today, for example1cea1afa.- Read the recall in the unit's trajectory: the facts are dated on the day of the run.
Expected
Mem0 extracts each session's facts as of that session's date, so "just reached 600 followers" in a session of 25 May 2023 is dated May 2023.
Acceptance
- The
mem0capability gives Mem0 the session's date as its Observation Date for eachadd(for example by patchingmem0.configs.prompts._resolve_dates, or freezing the clock at the session's date duringmem.add, sincetimestampis platform-only in the open-source package). - On
1cea1afa, the recalled facts carry May 2023, not the day of the run.
Dependencies and related
The cause sits between this adapter and mem0ai 2.2.1 (https://github.com/mem0ai/mem0), whose open-source add has no way to pass the Observation Date.
Activity
aitianshi commentedon Nov 10, 2026
Upstream
The cause is a known bug in mem0, already tracked there. No new upstream issue was filed, to avoid a duplicate.
bug,P2-medium)._add_to_vector_storenever passes a timestamp togenerate_additive_extraction_prompt, so## Observation Datedefaults to today.Closes #4963. Earlier attempts (#4964, #5000, #5032, #5222, #6331) were closed unmerged; extraction prompt date anchors to datetime.now() instead of created_at/timestamp metadata mem0ai/mem0#5894 was closed as a duplicate.What it means here
Both open fixes read the observation date from
metadata["timestamp"], thenmetadata["created_at"]. Our adapter (capability.py:114) passes the session date asmetadata["date"], so even after the fix lands, a pin bump alone would not reach the extraction prompt.Keep open until a mem0 release carries the fix; then bump the pin, pass the session date as
metadata["timestamp"], and rerun the 78 knowledge-update questions.fix(catalogue): pass the mem0 session date as the timestamp mem0's fi…
aitianshi commentedon Nov 10, 2026
@ttachon Extraction is supported in Nullius, but it needs to be supported in mem0 first.
Have a try with the PR #29 + import a new
mem0version with the needed patch.You may keep this issue open waiting to see if the PR fixes your problem