Sam Joel’s Post

🚀 Launching GenAI Stack: Our Commitment to Open-Source Generative AI 🚀 While the AI landscape advances with Large Language Models (LLMs), various organizations are still finding it difficult to adopt. Concerns range from data privacy to the intricacies of transitioning from raw domain data to searchable embeddings, from these embeddings to building/using LLMs and then building applications on top of the LLMs. In addition, LLMs often "hallucinate", or generate less accurate content, especially when they aren't trained in the nuances of business language or context. 🚀Introducing GenAI Stack 🚀 We are launching GenAI Stack to navigate these challenges and make AI accessible for everyone - a vision close to our hearts. It is an end-to-end framework for the integration of LLMs into any application. It can be deployed on your own infrastructure, ensuring data privacy. Today, we're thrilled to announce it is open-sourced for everyone. 🌟Features of GenAI Stack: 🔹 ETL Simplified: Navigate data processing complexities effortlessly. 🔹 Hallucination-Free Inference: Trustworthy AI-generated content for decision-making and research. 🔹 Seamless Integration: Easy adoption, whether you're a pro or just starting. 🔹 Customization and Control: Tailor processes to your project's needs. 🌟 Applications of GenAI Stack: 🔹Enhance search engines 🔹 Quick and dynamic knowledge base Q&A 🔹 Real-time sentiment analysis 🔹 Efficient customer support chatbots 🔹Streamlined information retrieval Contributions are welcomed: https://lnkd.in/gKWPpDWQ #ai #opensource #opensourcecommunity #machinelearning #largelanguagemodels #llms #datascience

I made a new thing! I made a recent, cutting-edge vision AI model from #Microsoft available over an API for local deployment: find links to the pre-built #Docker image and detailed documentation for deploying & using this API and even instructions for building this yourself from the ground-up, or deploying the model locally, in my repository linked below: https://lnkd.in/gcEKgcZW In this repository, you'll also read about the the significance of #OCR in todays #LLM landscape, a refresher on #RAG, and where Multimodal-LLMs (#MLLMs), and specifically the #Kosmos-2.5 model, come into the picture (pun intended!). Before you go there though, read on below for the complete backstory: Given how rapidly evolving the #GenAI space is, new models with esoteric requirements are releasing at an incredible pace. Some of these are potentially very useful, but very tough to actually get working. Containerizing such models along with a #PyFlask API script can make them so much more accessible. While not a standard method of releasing such models, my experience here demonstrates how useful such containers can be to deploy, migrate and utilize such LLMs! Microsoft's Kosmos-2.5: A few weeks ago, a post on Reddit brought to my attention a new & exciting OCR-centric local LLM by MS. This caught my eye big-time as it's especially relevant to my usecase as the developer of #LARS, an open-source, citation-centric RAG application (now at 360 stars on Github & listed on the official llama.cpp page!). I got about trying to deploy it, and quickly realized that while Kosmos-2.5 is an incredibly useful model, and especially precious as an open-source MLLM that excels at OCR, it is also incredibly difficult to deploy & get working locally. Worse, it's even more difficult to deploy it in a useful way - one wherein it can be made available usefully to other applications & for development tasks. This is due to a very stringent and specific set of hardware and software requirements that make this model extremely temperamental to deploy and use: Popular backends such as llama.cpp don't support it and a very specific, non-standard and customized version of the transformers library (v4.32.0.dev0) is required to correctly infer it. The 'triton' dependency necessitate Linux, while the use of FlashAttention2 necessitates very specific generations of Nvidia GPUs. Worse, its dependence on a very specific version of the 'omegaconf' Python library wasn't made clear until a recent GitHub issue which led to an update of the requirements.txt. There are nested dependencies that broke big time before this was clarified! Even now, Python 3.10.x is not explicitly stated as a requirement, though it very much is as the custom fairseq lib breaks on v3.11.x. (continued in the comments due to charecter limit)

Denser Retriever An enterprise-grade #AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy. Denser Retriever combines multiple search technologies into a single platform. https://buff.ly/3XMNAjp #database #MLops #RAG

AI Scraping for product data is now available in Zyte API, and it's a game changer for web scraping developers. We need to protect our developers' most precious resource: time. This release does just that—at scale. ⏳ 🕣 🚀 🕷 Setting up and launching new spiders takes minutes, not days 🤖 Website unblocking happens automatically — launch your crawl and go grab a coffee 🛠 Customize spider templates in Scrapy as needed and stay in control of your code, while also leveraging the power of our industry leading patented, supervised ML model One place to crawl, unblock and extract data from websites three times faster than legacy web scraping tools and proxy APIs. Try it free: https://lnkd.in/gkCYHABY Read our CEO's take on this exciting release. 👇

Finally, a commercial grade, self service product for “real world” AI powered scraping. I’ve been in the scraping game for over a decade, and this is the first time I’m confident developers can easily use AI to collect public data on the kind of speed and scale only AI can deliver. The impressive thing is actually more subtle: AI is NOT a silver bullet. It’s not perfect. It will occasionally go wrong. Zytes solution acknowledges that and is designed to work with real world developers and data teams to embrace the continued need for strong developer oversight, involvement and control of the data gathering process. When AI doesn’t work it’s important that it’s easy for developers to quickly and easily override, customise and add to scraping solutions. Today, in its first iteration it’s focused on the popular e-commerce and product data extraction, other common data types are comming soon, along with other advanced AI features all to make scraping even quicker, easier, and more flexible at scale.

Service is part of Anaconda’s DNA. 🧬 Since our founding, we’ve helped hundreds of companies make the most of their data while embracing open-source tools whenever possible. Our Professional Services team, led by James Bednar, is a collection of data scientists, machine learning and AI experts, and multidisciplinary technologists with over 100 years of industry experience. As we move into a future dominated by AI, Anaconda, Inc. is committed to providing an operating system (OS) for AI that makes the technology easy, accessible, and performant - just like we do now for Python. If you’re looking to jumpstart AI projects at your organization, here’s how our Professional Services team can help: 🥾 Kickstart Services - 100-hour engagements designed to uncover large-scale changes to help your organization scale to new heights with specific deliverables. 🤖 Custom LLM and ML Annotation - Our experts will build on top of large language models (LLMs), such as ChatGPT, Llama, Bard, using your own data to deploy AI at scale. 🐍 Spark or SAS Migration to Python - Eliminate tech debt by converting legacy code to repeatable AI-ready models and packages, opening further enhancements from the Python and ML/AI ecosystem. ❄️ Python to Snowflake Snowpark Integration and Migration - Move your models into the world’s most popular database, leveraging Anaconda’s partnership with Snowflake and the help of our experts. As we continue to expand our offerings, we look forward to tackling new problems with new and existing customers and are always open to taking on novel problems and use cases. Learn more about our Professional Services here 🔗 👇 https://bit.ly/48HvJwS If any of these projects or services are a fit for your needs, please reach out to us via the link in the comments! 🔗

Introducing Structured Outputs: Enhancing Reliability in the OpenAI API. In a significant advancement for developers leveraging the OpenAI API, the company has introduced Structured Outputs, a groundbreaking feature designed to ensure reliable and consistent model outputs. This innovation addresses a critical need in the AI development landscape, where generating structured data from unstructured inputs is a core use case. Ensuring Adherence to Developer-Supplied Schemas: Structured Outputs enable developers to supply JSON Schemas that OpenAI models must adhere to when generating outputs. This feature is available in two forms: function calling via `tools` and a new `response_format` parameter option. By constraining models to match developer-specified schemas, Structured Outputs eliminates the need for workarounds and ensures seamless integration with existing systems. Improved Model Performance and Reliability: OpenAI's latest model, `gpt-4o-2024-08-06`, achieves a perfect 100% score on complex JSON schema following evaluations when using Structured Outputs, compared to less than 40% for the previous `gpt-4-0613` model. This significant improvement in reliability allows developers to build applications with greater confidence, knowing that model outputs will precisely match their requirements. Prioritising Safety and Usability : Structured Outputs maintains OpenAI's commitment to safety, with models still able to refuse unsafe requests. To simplify development, a new `refusal` string value is included in API responses, enabling developers to programmatically detect when a model has generated a refusal instead of output matching the schema. Native SDK Support for Seamless Integration : OpenAI's Python and Node SDKs have been updated with native support for Structured Outputs, making it easy for developers to supply schemas and automatically deserialise JSON responses into typed data structures. This integration ensures a smooth development experience and accelerates building reliable AI-powered applications. Structured Outputs represent a significant advancement in the OpenAI API, empowering developers to build more reliable and efficient applications. By ensuring consistent adherence to developer-supplied schemas, this feature addresses a critical need in the AI development landscape and sets a new standard for model output reliability. As the demand for structured data generation continues to grow, Structured Outputs position OpenAI as a leader in enabling developers to harness the power of AI with confidence. For more information, visit : https://lnkd.in/eFTSCC4X. #StructuredOutputs #OpenAIAPI #AIReliability #DeveloperTools #JSONSchema

Avenue Code Announces AI Studio to Ground Large Language Models on Proprietary Data Software consultancy Avenue Code, part of AI/R, announced today the enterprise availability of AI Studio, its solution to help companies put generative AI in production faster by using their own data to fine-tune the answers from large language models. Today, companies working to unlock GenAI's transformative potential need to address the complex challenge of creating consistent and reliable software agents that avoid the sometime invented answers or 'hallucinations' of models trained on third-party data. AI Studio enables the rapid development and deployment of intelligent software solutions that deliver tangible results with a series of premade open-source integrations to tap proprietary data in ten different Google Cloud databases including MySQL, Alloy DB, Spanner, Firestore, Oracle, and others.  https://lnkd.in/g-7zSdHY

DataStax updates tools for building gen AI applications: DataStax is updating its tools for building generative AI-based applications in an effort to ease and accelerate application development for enterprises, databases, and service providers. One of these tools is Langflow, which DataStax acquired in April. It is an open source, web-based no-code graphical user interface (GUI) that allows developers to visually prototype LangChain flows and iterate them to develop applications faster. To read this article in full, please click here #MachineLearning #ArtificialIntelligence #DrivenByData #BigData #DataAnalytics #ITDirector #AI #DataDriven

OpenAI's latest API feature promises reliable JSON outputs matching developer-defined schemas. But is this really innovative, or are they simply implementing what open-source solutions have offered for ages? Some thoughts: 1. The 50% price cut is suspiciously timed. Are they struggling to compete with open models? 2. Claims of 100% reliability seem dubious. LLMs are inherently probabilistic - can we truly trust "guaranteed" outputs? 3. This feels like a step backwards from AGI ambitions. Are we witnessing OpenAI's transformation into just another SaaS company? 4. The acknowledgement of open-source inspiration is nice, but where's the reciprocation? "Open" AI indeed... 5. Is constraining outputs detrimental to model creativity and reasoning? We may be sacrificing intelligence for conformity. Structured outputs could be a game-changer for developers, but at what cost to the broader AI landscape? https://lnkd.in/dcn787eX