LumenMem v0.4.1

LumenMem is now a build-only memory engine. It converts ordered dialogue input into four connected artifacts:

1. Episodes and multi-granularity Segments
2. Rescenes (theme, diary, time, and source spans)
3. Atomic facts with embeddings and evidence
4. Entity profiles and a Neo4j entity-relation graph

Rescene is the platform-derived episodic memory layer. It is deliberately distinct from the caller's raw Scene in the function protocol.

Install from a source checkout

pip install -e .

Set model credentials in .env. Entity construction requires NEO4J_URL, NEO4J_USER, and NEO4J_PASSWORD; each workflow uses an isolated database name generated from NEO4J_DATABASE_TEMPLATE.

LLM credentials are kept in separate provider blocks. The active provider is selected by the configuration of each workflow, not by a global environment switch:

# .env
OPENAI_API_KEY=
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o-mini

DEEPSEEK_API_KEY=
DEEPSEEK_BASE_URL=https://api.deepseek.com
DEEPSEEK_MODEL=deepseek-v4-flash

For memory construction, select the block in config/memory_build.yaml:

llm:
  provider: deepseek  # or openai
  model: ""           # empty: use the selected provider's *_MODEL value
  temperature: 0.0

An explicit non-empty llm.model overrides the selected provider's environment model. The embedding block is resolved separately, so selecting DeepSeek for chat does not change the configured embedding service.

Input

The command accepts a JSON list of dialogues, {"dialogues": [...]}, or a Scene JSONL file. Each canonical dialogue contains ordered turns:

{
  "dialogues": [
    {
      "dialogue_id": "dialogue-001",
      "turns": [
        {
          "speaker": "user",
          "text": "I started learning pottery today.",
          "timestamp": "2026-08-11T10:00:00+08:00"
        }
      ]
    }
  ]
}

Additional turn fields are preserved through Episode, Segment, Rescene, and fact construction.

For Scene JSONL, one file is one cross-transaction Dialogue. Every non-empty JSON record becomes one turn, ordered by seq rather than timestamp. Runtime entry types are rendered into the text seen by memory prompts:

Stimulus_X -> [observation_X] content
Thought    -> [thinking] content
Action     -> [tool_use_<tool_name>] content

Source identifiers such as append_id, transaction_id, delegate_id, and payload_ref are retained on the turn for traceability. Nested JSON content is rendered as compact valid JSON; user-facing prose remains unchanged.

This command is the construction engine's internal batch-ingestion entry point. The v0.4.1 Memory_build protocol is a subsystem-boundary draft, not the current CLI request/response contract.

Build

lumen-mem-build \
  --workflow-id demo \
  --input path/to/dialogues.json \
  --config config/memory_build.yaml

Useful controls:

• --clean: remove local artifacts for this workflow before building.
• --force-rebuild: clear and rebuild the workflow's derived entity state.
• --force-rescene: regenerate existing Rescenes.
• --force-facts: regenerate fact payloads.

Concurrency is controlled by LUMEN_MEM_EPISODE_MAX_WORKERS, LUMEN_MEM_RESCENE_MAX_WORKERS, and LUMEN_MEM_RESCENE_FACT_MAX_WORKERS.

Output

By default, artifacts are written under data/memory/<workflow-id>/:

dialogues/
episodes/
rescene/                 # Rescene JSON; atomic facts are embedded with provenance
local_store/entity_library/
local_store/entity_profile/

The main configuration is config/memory_build.yaml; all LLM templates used by construction live under config/prompts/memory_build/.

The open semantic questions are documented in docs/v0.4.1/memory-semantics-open-questions-zh-CN.md. The semantics-independent persistence and build-path plan is maintained separately in docs/v0.4.1/memory-data-persistence-engineering-plan-zh-CN.md.