← Back to library

▶ quest log

Use a Structured Data Warehouse as Agent Memory

Data◆ Intermediate◆ Advanced@angus.sewellsaved MAY 19

◇ Could this help me?

Yes. The central claim is right and under-argued elsewhere: most questions people want an assistant to answer about their own work are structured queries wearing a memory costume, and a database answers those exactly and repeatedly where a similarity search answers them approximately and inconsistently. The advice to start with plain files when working alone is also correct and refreshingly unmonetized. General practice, with two honest corrections: the dismissal of vector search is overstated, since unstructured text that cannot be queried by field is precisely where it earns its place, and consolidating every source system into one warehouse concentrates access in a way that deserves a deliberate decision about what is copied in rather than connecting everything because it is easy.

✦ Walkthrough+10 xp / step
0 / 7 steps · adopt for +40
$ suggested_prompt+40xp

Help me decide whether what I want is actually structured retrieval rather than memory. Take the questions I most often want answered, described below, and sort them into ones that are really queries against records that already exist somewhere, and ones that genuinely require searching unstructured text by meaning. For the first group, design the tables they would need and tell me which of my existing systems would supply each field. For the second group, say so plainly rather than forcing them into a schema. Then tell me what I should deliberately not copy into a consolidated store, and why.

More from Data

Enter world →