SQLRetriever output unrelated information #14084
Replies: 1 comment 4 replies
-
Hey @JonOnEarth! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together! The Here is how you can ensure that only relevant data is passed to the next module:
Example of a custom node postprocessor: from llama_index.legacy.schema import NodeWithScore, QueryBundle
from typing import List
class CustomNodePostprocessor:
def postprocess_nodes(self, nodes: List[NodeWithScore], query_bundle: QueryBundle) -> List[NodeWithScore]:
# Implement your filtering logic here
filtered_nodes = [node for node in nodes if self.is_relevant(node)]
return filtered_nodes
def is_relevant(self, node: NodeWithScore) -> bool:
# Define your criteria for relevance
return "relevant_keyword" in node.node.text Adding the custom postprocessor to the custom_postprocessor = CustomNodePostprocessor()
class CustomNLSQLRetriever(NLSQLRetriever):
def retrieve_with_metadata(self, str_or_query_bundle: QueryType) -> Tuple[List[NodeWithScore], Dict]:
retrieved_nodes, metadata = super().retrieve_with_metadata(str_or_query_bundle)
filtered_nodes = custom_postprocessor.postprocess_nodes(retrieved_nodes, str_or_query_bundle)
return filtered_nodes, metadata By using a custom node postprocessor, you can filter out unrelated information and ensure that only relevant data is passed to the next module [1][2][3]. |
Beta Was this translation helpful? Give feedback.
-
Hi, I used SQLRetriever to retriever the Database, it returned something like:
It will be the output to my next module in the query pipeline with synthesis response, and the answer will use unrelated information such as the node information.
such that:
Sometimes it even outputs the node id...
I checked the original code of SQLRetriever, it outputs the format like the example above, may I ask the reason, is there someone who met this issue?
Beta Was this translation helpful? Give feedback.
All reactions