Replies: 1 comment
-
🤖 Hey @sbyim, back for more LlamaIndex adventures, I see! Good to have you around. Yes, you are correct. The LlamaIndex Milvus module does not store text from the nodes by default. The As for storing the text as metadata, it depends on your specific use case. If you need to frequently access the text associated with each node and want to avoid the overhead of retrieving it from the Milvus collection each time, storing it as metadata could be beneficial. However, this would also increase the memory footprint of your application. If you decide to store the text as metadata, you would need to modify the def node_to_metadata_dict(node):
entry = {
MILVUS_ID_FIELD: node.node_id,
self.embedding_field: node.embedding,
'text': node.text # Add this line to include the node's text
}
return entry Then, in the for node in nodes:
entry = node_to_metadata_dict(node)
insert_list.append(entry) This would store the node's text in Milvus along with its ID and embedding. Please note that this would increase the amount of data stored in Milvus, which could impact performance and memory usage. I hope this helps! If you have any further questions, please don't hesitate to ask. SourcesThis response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
Beta Was this translation helpful? Give feedback.
-
We are using LLamaIndex + Milvus, where LlamaIndex is used for metadata extraction + ingestion.
It seems Llamaindex Milvus module does not store text from the nodes as default, am i correct?
If so, is storing texts as metadata is the right way to do?
We don't want to use extra document store to store nodes..
def add(self, nodes: List[BaseNode], **add_kwargs: Any) -> List[str]: """Add the embeddings and their nodes into Milvus.
Beta Was this translation helpful? Give feedback.
All reactions