Memorilla: Latent Semantic Memory for LLMs

arxiv.org

113 points by handfuloflight a day ago


jkingsman - a day ago

So if I'm reading this right, it's basically a specialized embedding of a RAG, and then a model in the middle takes the agent's queries and returns responses in a form that is injecting into the main model's input token stream directly i.e. skipping actual text. And the injection format is INCREDIBLY token efficient. Sort of injecting a vibe rather than quotable text. Their results of "good on concepts, less good on quotes/exact text retrieval" seem to intuitively line up with that.

bob1029 - 19 hours ago

I am finding that semantic ~= lexical when you have something like a frontier model driving. Using full text search engines is still the most compelling approach for handling large document sets.

I really struggle with the semantic / embedding concerns when we have a generative model that can reverse engineer a Mario 64 rom dump to run on my toaster oven. These models know how to compose Lucene & other FTS query strings in such a way that we are effectively getting semantic search over purely lexical systems.

airesearcher2 - 15 hours ago

memory as a modality is such a cool idea!

apishare_api - 18 hours ago

[flagged]

fiveapi_dev - a day ago

[flagged]