[Bug]: remember() into a previously forget()-ed dataset name always fails with 409 RetryError[ProgrammingError] (full deletion tombstones the name; memory_only=True does not)

[devanshug2307]

What happened

After fully deleting a dataset with forget(dataset=X), any later remember(..., dataset_name=X) on Cognee Cloud fails with:

Remote remember failed (409): {"error":"An error occurred during remember: RetryError[<Future at 0x... state=finished raised ProgrammingError>]"}

The dataset name becomes permanently unusable for the tenant. Deleting with forget(dataset=X, memory_only=True) does NOT have this problem — re-remembering works fine.

Minimum viable code to reproduce

import asyncio, cognee

async def main():
    await cognee.serve(url=TENANT_URL, api_key=API_KEY)

    # A) full forget -> re-remember FAILS
    await cognee.remember("v1", dataset_name="repro_409_full")
    await cognee.forget(dataset="repro_409_full")            # {'status': 'success'}
    await cognee.remember("v2", dataset_name="repro_409_full")  # 409 RetryError[ProgrammingError]

    # B) memory-only forget -> re-remember WORKS
    await cognee.remember("v1", dataset_name="repro_409_memonly")
    await cognee.forget(dataset="repro_409_memonly", memory_only=True)
    await cognee.remember("v2", dataset_name="repro_409_memonly")  # OK

asyncio.run(main())

Ran both paths back-to-back on a fresh tenant today: A fails 100% of the time, B succeeds. Standalone repro script: https://github.com/devanshug2307/blackoutops/blob/main/scripts/repro_forget_409.py

Environment

• Cognee Cloud tenant, server version 1.2.2.dev0 (from /health), AWS us-east
• cognee SDK 1.2.2, Python 3.12.13, macOS (Darwin 25.5.0)
• Connected via cognee.serve(url=..., api_key=...)

Expected behavior

forget(dataset=X) followed by a later remember(..., dataset_name=X) should recreate the dataset (or return a documented error explaining the name is reserved). A deleted dataset name should not be tombstoned forever.

My guess at the cause

Dataset IDs appear deterministic per (tenant, name) — the same name maps to the same dataset_id before and after deletion (same UUID in receipts). If full deletion soft-deletes the relational row, the re-remember INSERT would collide with the tombstoned row → ProgrammingError → retried → surfaced as 409 RetryError. memory_only=True resets data but keeps the row, which would explain why it survives.

Found while building a hackathon project whose demo forgets and re-seeds incident datasets. Happy to help verify a fix — I have a tenant that reproduces this reliably.

[github-actions]

Hello @devanshug2307, thank you for your interest in our work!

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

[devanshug2307]

The minimum viable repro is in the issue body above; here it is executed end-to-end just now against my tenant, via the standalone script (scripts/repro_forget_409.py):

Connected to Cognee (remote) at https://tenant-ad159b59-****.aws.cognee.ai
A (full forget):        re-remember FAILED: Remote remember failed (409): {"error":"An error occurred during remember: RetryError[<Future at 0x... state=finished raised ProgrammingError>]"}
B (memory-only forget): re-remember OK

Same run, back to back: full forget() → remember() into the same dataset name fails with the 409 every time; forget(memory_only=True) → remember() succeeds. Fresh tenant, server 1.2.2.dev0, SDK 1.2.2, connected via cognee.serve(). Happy to re-run with any debug flags that would help.

[devanshug2307]

Since I reported this, I'd like to take the fix as well @Vasilije1990 @dexters1.

Spent the evening in the OSS code and I believe it's three separate residues left behind by a full forget, not one bug:

1. PGVectorDatasetDatabaseHandler.delete_dataset drops the per-dataset database, but the metadata reset it triggers builds its cache key with the default vector_db_subprocess_enabled instead of the configured value — so with VECTOR_DB_SUBPROCESS_ENABLED=false (or from another worker) the reset hits a throwaway adapter and the poisoned PGVectorAdapter stays in the LRU.
2. That stale adapter's _metadata makes has_collection() return True for the freshly recreated (empty) database, create_collection is skipped, the INSERT raises asyncpg UndefinedTableError → SQLAlchemy ProgrammingError → tenacity retries into the same stale cache → the exact RetryError[ProgrammingError] from my report, forever.
3. pipeline_runs rows have no FK to datasets and aren't cleaned by full delete (memory_only resets them), so a recreated dataset inherits a completed pipeline status.

I have a patch working locally: evict the vector engine with the runtime cache key (mirroring how the graph handlers call evict_graph_engine), self-heal _metadata on ProgrammingError inside create_data_points so the existing retry recovers in the cross-worker case eviction can't reach, and delete the dataset's pipeline_runs in the same transaction as the pipeline-status cleanup. My regression test reproduces the identical error signature on clean dev and passes with the patch; neighboring vector/forget suites stay green.

Can't verify Cloud's exact deployment flags from outside, but both plausible triggers (key mismatch vs multi-worker) are covered by the same patch. Ready to open the PR against dev, signed off with test screenshots, as soon as it's assigned.