0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00

fix(@mastra/core, @mastra/memory, @mastra/pg): semantic recall perf + memory bundle size (#3419)

## Semantic recall perf + memory bundle size
Before this it would take around 2.5 to 3 seconds before we sent a req
to the LLM while we were processing semantic recall (embedding messages,
querying db, etc). After this change it takes between 800ms and 1.1s.
This is when using pgvector. This should also have a perf improvement
for other vector stores but probably not as big of a change.

Changes:
1. removed `@mastra/rag` from `@mastra/memory`, now it uses
`tiktoken-js` to do chunking (this should also fix the bundle size issue
https://github.com/mastra-ai/mastra/issues/3309 )
2. uses measured dimensions instead of hardcoded dimensions and
parallelizes embeddings as much as possible (most likely fixes
https://github.com/mastra-ai/mastra/issues/3236 )
3. combines 2 memory.saveMessages calls into one
4. combined another 2 memory.getThreadById() calls into one
5. adds a chunk/embedding in memory cache for Memory
6. swapped out pg vector advisory locks for in memory mutexes (saves
almost a second of time on each request)
7. combined multiple pg queries into single queries, removed extra
checks for if a table/index exists before the query to create it (since
the query that creates them uses `IF NOT EXISTS` anyway)
8. adds an in memory (on the class instance) cache for creating indexes
in pg and for getting index info, saves around 700ms per request

Semantic recall is between 2 to 3 times faster with these changes
This commit is contained in:
Tyler Barnes 2025-04-03 12:28:33 -07:00 • committed by GitHub
commit 93875edd12
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 360 additions and 305 deletions