mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
fix(@mastra/core, @mastra/memory, @mastra/pg): semantic recall perf + memory bundle size (#3419)
## Semantic recall perf + memory bundle size Before this it would take around 2.5 to 3 seconds before we sent a req to the LLM while we were processing semantic recall (embedding messages, querying db, etc). After this change it takes between 800ms and 1.1s. This is when using pgvector. This should also have a perf improvement for other vector stores but probably not as big of a change. Changes: 1. removed `@mastra/rag` from `@mastra/memory`, now it uses `tiktoken-js` to do chunking (this should also fix the bundle size issue https://github.com/mastra-ai/mastra/issues/3309 ) 2. uses measured dimensions instead of hardcoded dimensions and parallelizes embeddings as much as possible (most likely fixes https://github.com/mastra-ai/mastra/issues/3236 ) 3. combines 2 memory.saveMessages calls into one 4. combined another 2 memory.getThreadById() calls into one 5. adds a chunk/embedding in memory cache for Memory 6. swapped out pg vector advisory locks for in memory mutexes (saves almost a second of time on each request) 7. combined multiple pg queries into single queries, removed extra checks for if a table/index exists before the query to create it (since the query that creates them uses `IF NOT EXISTS` anyway) 8. adds an in memory (on the class instance) cache for creating indexes in pg and for getting index info, saves around 700ms per request Semantic recall is between 2 to 3 times faster with these changes
This commit is contained in:
parent
bd07c85663
commit
93875edd12
11 changed files with 360 additions and 305 deletions
1
.gitignore
vendored
1
.gitignore
vendored
68
pnpm-lock.yaml
generated
68
pnpm-lock.yaml
generated
Loading…
Add table
Add a link
Reference in a new issue