Gradient’s Post

What is the Mixture of experts (MoE) approach? The MoE approach is a machine learning technique that combines the strengths of multiple expert models to solve a problem. In contrast to ensemble techniques, which combine results from all models, in MoE, only one or a few experts will be used to make predictions. Core Components This approach has two main components:  • Router: decides which expert to trust for a given input and how to weigh each expert’s results for the specific input. • Experts: individual models that specialize in different aspects of the problem. In Mixtral’s case, there are eight experts, of which two are selected. How does the MoE approach work?  - For a new input, the router decides which experts should handle that input. In Mistral’s case, the selection of experts appears to be based on the syntax rather than the domain - The selected experts then make their predictions based on their expert knowledge. - The final prediction is a combination of the outputs from these experts, weighted by how much the router trusts each expert for that particular input. Unfortunately, the lack of information on the training data in the paper leaves some open questions. Paper: https://lnkd.in/eGDsiE_j

We're now up to 15 #AI (LLM) agents within our core solutions, we will be around 50 before 2023 closes. Some cool things they are doing are standardizing spreadsheet data then auto loading. Auto loaders in classical programming are clunky and require pre-process standardization to not break. Also you can run most tabular data types to check for anomalies i.e. Routine Core Analysis with grain size of 274 when all others are in the range of 2.5-3. This is just one of many examples of how we can clean and store your data. Schedule a demo! http://www.nthds.com #processautomation #datamanagement

RAG Foundry is a library designed to improve LLMs ability to use external information by fine-tuning models on specially created RAG-augmented datasets. The library helps create the data for training, given a RAG technique, helps easily train models using parameter-efficient finetuning (PEFT), and finally can help users measure the improved performance using various, RAG-specific metrics. The library is modular, workflows are customizable using configuration files. https://lnkd.in/dCfY3wa6

Demo: applying retrieval-augmented generation with open tools🛠️ Large models excel in various areas thanks to their training on extensive data across diverse topics, but they face challenges such as being resource-intensive and not always current. They lack real-time updates and access to private data, limiting their use in dynamic or confidential contexts. To address this, retrieval-augmented generation (the ever-so-popular RAG) technique is being adopted to enhance LLMs for enterprise applications, enabling them to provide accurate, timely information by integrating up-to-date and proprietary data, especially crucial in regulated industries. Head straight to our blog to examine a quick example of applying RAG in a real-world context: https://lnkd.in/dbdZre9p #RAG #LLMs #opensource #PoC #Flowise

Today I earned my "Use prebuilt Document intelligence models" badge! I’m so proud to be celebrating this achievement and hope this inspires you to start your own @MicrosoftLearn journey! There are many document types that are common to most business and Azure AI Document Intelligence includes prebuilt models to handle them. If you have a collection of such forms that you want to analyze, you can extract data by using these prebuilt models and you don't have to train your own models. You can get up and running very quickly by submitting photos and scans to the most appropriate prebuilt model. Now that you've completed this module, you can: Identify business problems that you can solve by using prebuilt models in Azure AI Document Intelligence. Analyze forms by using the General Document, Read, and Layout models. Analyze forms by using financial, ID, and tax prebuilt models.

One of the best parts of launching something new is getting to hear from friends and supporters. Thank you so much to everyone who shared words of encouragement or took the time to check out Superpipe 🙏 The core thesis behind Superpipe is: 1. LLM-powered software needs evals 2. Evals need to be performed on YOUR data and YOUR use case. Benchmarks on public datasets don't translate directly to your data. 3. It's impossible to fine-tune effectively without an eval system Read more on our blog: https://lnkd.in/dHTHHGEJ

🚀 LLMs Are Democratizing High-End ML Tools A prime example: https://lnkd.in/dJew7bcJ 📊 RAGFlow: An open-source engine implementing the RAG methodology for deep document understanding. Key Features: 🔍 Deep Document Understanding • Extracts knowledge from complex, unstructured data • Ensures high-quality input leads to high-quality output 🎯 Advanced Data Retrieval • Locates specific information within vast data sets • Handles virtually unlimited tokens 🌐 Broad Compatibility • Works with diverse data sources, including: Word documents, PowerPoint slides, Excel sheets ,Text files and images,Scanned documents ,Structured data and web pages 💡 Implications: These features, which once required significant ML R&D effort and data engineering during pre LLM days, are now accessible to a much wider audience. While this democratization is beneficial for many users, it may have significant implications for the Intelligent Document Processing (IDP) industry, as previously unique offerings become widely available. 🤔 Thoughts on how this might reshape the ML and document processing landscape? #MachineLearning #LLM

Absolutely agree with the insights! Your proposed solutions are truly insightful and demonstrate a deep understanding of handling a major issue.

🚀 Building in Public: Week 2 Progress on TAM Lab's Multi-Agent LLM Excel Plugin 📊 Mission: Create an Excel plug-in that uses AI Agents to automate the generation of Total Addressable Market (TAM) reports. 📊✨ Before I dive in, don't miss out on being first in line when we launch - join our waitlist now at www.tamlab.ai 🌟 Now, let's get into what I've accomplished: 🔍 Implemented advanced web crawling across multiple organizational data sources, tackling the challenge of diverse website structures head-on. 🧠 Leveraged LLM capabilities to handle complex use cases: - Estimating data when primary sources fall short - Navigating scenarios with missing quantity or price data - Adapting to mismatches between input and clustered data sources 🏗️ Transformed our ambitious project architecture from concept to code, laying a robust foundation for TAM-Automation. 🏃♂️Conquered variability challenges across 7-8 use cases, ensuring a rock-solid data architecture despite diverse website structures. 🎥 Bonus: I've attached a brief demo showcasing our form input process and data scraping feature in action! Looking ahead, I'm diving into data cleaning, implementing CAGR calculations, and building out our forecasting model. Question for my network: What challenges do you foresee in integrating AI-driven tools with traditional Excel workflows? Your insights could shape our development roadmap! #AIInnovation #MarketAnalysis #BuildingInPublic #ExcelAutomation #LLM #AIAgents

Check out our latest post on Clappy.ai, "Leverage Layout-Aware RAG for More Accurate Retrieval of Knowledge from Documents." Discover how to harness the power of Retrieval-Augmented Generation (RAG) for documents in PDF, Doc, and Markdown formats. Dive into the specifics of implementation and explore our reference repository to get started with advanced RAG for your knowledge base. Don't miss out on this essential read! Continue reading here for all the details: The Power of Hierarchical Indexing with RAG. (https://lnkd.in/gbiVHfcH)