Architecture for many Separate Collections #21341
Replies: 4 comments 12 replies
-
If your cpu memory is sufficient, create a collection for each user would be a best way. Keep all the collections loaded in memory. |
Beta Was this translation helpful? Give feedback.
-
We will support collection LRU loaded but it is still under design and it might also be super slow(Load data from S3 is slow). |
Beta Was this translation helpful? Give feedback.
-
@gvilums You did write:
Do you know how long it takes? I've never tried it. But if it's not too long, then you can play with the timing when to load the collection. E.g. initiate the collection loading on page load or when the input field is in focus. The user needs time to write the query anyway, so maybe the collection is ready by then. |
Beta Was this translation helpful? Give feedback.
-
Hi @gvilums |
Beta Was this translation helpful? Give feedback.
-
Hello everyone 👋 ,
We're planning to use Milvus as a vector similarity search backend for a consumer service. In our model, each user only ever needs to search through the data provided by them, and will (and should) never find results related to other users.
I was wondering what the best approach for modeling something like this is in Milvus. My first idea was to create a separate
Collection
for each user. Then, when a query arrives, this collection can be loaded and the query can be answered, and finally the collection can be unloaded again.However, this approach has the problem that loading a collection takes multiple seconds, whereas we would like to keep total query latency under 200ms.
Alternatively, I was thinking that it might be possible to simply let a user's collection remain loaded, so that subsequent queries are answered faster. However, I'm not sure if Milvus implements any garbage collection for collections which are loaded but inactive, and fear that this approach would simply lead to eventual memory exhaustion.
Another approach would be to create a single collection for all users, and then filter based on some
user-id
attribute. However, I imagine that this would be quite inefficient, as the entire collection would have to be loaded all of the time, even for users which have been inactive for a long time.Finally, I was also looking at using a DISKANN index (but was unable to get it to work). Does this index type also require the collection to be loaded?
Any suggestions about the best way to approach this would be appreciated 😄
Beta Was this translation helpful? Give feedback.
All reactions