-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question]: Joint QA Summary Query Engine no openAI #12749
Comments
The error you're encountering suggests that the LlamaIndex framework is attempting to use an OpenAI model by default, which requires an API key. Since you're not using OpenAI's models, you need to explicitly specify the model you're using or disable the LLM to avoid this error. To resolve this, ensure you're specifying your model correctly when initializing or calling functions related to the llm = resolve_llm("local:/path/to/your/model") Make sure the path is correct and that you have the necessary dependencies installed for your model. Alternatively, if you want to disable the LLM entirely (for example, for testing purposes), you can set llm = resolve_llm(None) This approach uses a For more detailed guidance, refer to the documentation or source code of the LlamaIndex framework, particularly around the Sources
|
NameError: name 'resolve_llm' is not defined |
Question Validation
Question
Code-:
llm = HuggingFaceLLM(
model_name=model_id,
tokenizer_name=model_id,
query_wrapper_prompt=PromptTemplate(PROMPT),
context_window=3900,
max_new_tokens=256,
model_kwargs={"torch_dtype": torch.float16},
# tokenizer_kwargs={},
generate_kwargs={"temperature": 0.3, "top_p": 0.90},
device_map="auto",
)
Settings.llm = llm
Settings.embed_model = embeddings
Settings.context_window=4096,
Settings.num_output=1500,
Settings.chunk_overlap_ratio=0.3,
vsi = VectorStoreIndex.from_vector_store(
vector_store = vector_store
)
query_engine_builder = QASummaryQueryEngineBuilder(
llm=llm,
)
Error-:
Traceback (most recent call last):
File "/home/seceon/opensearch_lm_index/model2.py", line 123, in
query_engine_builder = QASummaryQueryEngineBuilder(
File "/usr/local/lib/python3.10/site-packages/llama_index/core/composability/joint_qa_summary.py", line 77, in init
self.service_context = service_context or ServiceContext.from_defaults()
File "/usr/local/lib/python3.10/site-packages/deprecated/classic.py", line 285, in wrapper_function
return wrapped(*args_, **kwargs_)
File "/usr/local/lib/python3.10/site-packages/llama_index/core/service_context.py", line 187, in from_defaults
llm_predictor = llm_predictor or LLMPredictor(
File "/usr/local/lib/python3.10/site-packages/llama_index/core/service_context_elements/llm_predictor.py", line 108, in init
self._llm = resolve_llm(llm, callback_manager=callback_manager)
File "/usr/local/lib/python3.10/site-packages/llama_index/core/llms/utils.py", line 48, in resolve_llm
raise ValueError(
ValueError:
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys
To disable the LLM entirely, set llm=None.
This time I have defined QASummaryQueryEngineBuilder after LLM, still I am getting this issue.
The text was updated successfully, but these errors were encountered: