Currently at Meta, previously worked at American Express, Cadence, NEC, Cognizant, Infosys & HCL.
I don't think the challenge is of making a change in how we integrate an app with LLMs. I think, the key challenge is the capability of open-source LLMs, as they are still far from where OpenAI's GPT-4 is. When you switch your app from OpenAI model to any open source LLM, the quality of responses degrade drastically, and that normally breaks things in the app. I have been playing with #autogen framework from Microsoft and some opensource LLMs like #mistral7b, #Mistral8x7b models from Together AI and fine tuned #codellama model from Phind but none of them worked as expected. Even OpenAI #gpt4 didn't work as expected in my case. Anyway switching the URLs and keys is the least of the challenges I see. With that said, it doesn't mean that opensource models are of no use. They can still be used in many use-cases where response accuracy or format is not so critical for an app to work.
🚀 It's now easier than ever to switch from OpenAI to open LLMs Hugging Face's TGI now supports an OpenAI compatible Chat Completion API This means you can transition code that uses OpenAI client libraries to run open models by changing just two lines of code 🤗 Check out the full post ➡ https://lnkd.in/eABNVGZS TGI documentation ➡ https://lnkd.in/e-NFt-4D