๐๐จ๐จ๐ค๐ข๐ง๐ ๐ญ๐จ ๐ซ๐๐๐ฎ๐๐ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ง๐ ๐ข๐ง๐๐๐ซ๐๐ง๐๐ ๐๐จ๐ฌ๐ญ๐ฌ ๐จ๐ ๐๐๐๐ฌ, ๐ก๐๐ซ๐ ๐๐ซ๐ ๐ญ๐จ๐ฉ 5 ๐ฌ๐ญ๐ซ๐๐ญ๐๐ ๐ข๐๐ฌ ๐๐จ๐ซ ๐ฒ๐จ๐ฎ ๐ญ๐จ ๐๐จ๐ง๐ฌ๐ข๐๐๐ซ 1. ๐๐ซ๐ฎ๐ง๐ข๐ง๐ ๐๐ง๐ ๐๐ฎ๐๐ง๐ญ๐ข๐ณ๐๐ญ๐ข๐จ๐ง: Convert the weights of a BERT-based model from float32 to int8, reducing precision. This minimizes the model size and accelerates both training and inference. 2. ๐๐๐ญ๐๐ก๐๐ ๐๐ง๐๐๐ซ๐๐ง๐๐: Process a batch of image classification requests simultaneously. In a computer vision application, batching allows efficient GPU utilization, speeding up inference. 3. ๐๐ง๐จ๐ฐ๐ฅ๐๐๐ ๐ ๐๐ข๐ฌ๐ญ๐ข๐ฅ๐ฅ๐๐ญ๐ข๐จ๐ง: Distill knowledge from a complex language model like GPT-4 into a smaller, faster model like TinyGPT. This decreases the computational requirements for both training and inference. 4. ๐๐๐๐ข๐๐ข๐๐ง๐ญ ๐๐๐ญ๐ ๐๐จ๐๐๐ข๐ง๐ & ๐๐ข๐ฉ๐๐ฅ๐ข๐ง๐ข๐ง๐ : Use TensorFlow Data Pipeline to optimize loading large datasets. This minimizes I/O bottlenecks during training, enhancing GPU usage efficiency. 5. ๐๐จ๐๐๐ฅ ๐๐ ๐ง๐จ๐ฌ๐ญ๐ข๐ ๐๐๐ญ๐-๐๐๐๐ซ๐ง๐ข๐ง๐ (๐๐๐๐): Train a meta-model on few-shot learning tasks, allowing quick adaptation to new domains. This reduces per-inference costs and enhances the versatility of the model across various applications.
Vaibhav Goyalโs Post
More Relevant Posts
-
This is the way. I think that current RAG patterns are very limited but there aren't better solutions yet, even though this is a promising one in the direction that I find the most interesting, which is a differentiable "soft" memory. โ Currently, a retriever extracts contexts (texts), which are sent to the LLM along with the user query. For a very good pipeline you need to have an embedding model, lexical search, query decomposition, hierarchical information chunks and lots of other things... that may not be enough for answering all kinds of queries. โ Have a sort of differentiable memory, which writes information into the LLM activations, that this model could request using cross-attention or some approach like this. Collect a dataset, research this memory and just let the neural net learn to do the task.
Can we pre-train LLMs with Retrieval Augmentation? ๐คย RETRO was a research by Google DeepMind, which included retrieval into the pre-trainng process. Now NVIDIA continues this research by scaling RETRO to 48B, where they continued pretraining a 43B GPT model on an additional 100 billion tokens using the Retrieval augmentation method by retrieving from 1.2 trillion tokens. ๐คฏ ๐๐บ๐ฝ๐น๐ฒ๐บ๐ฒ๐ป๐๐ฎ๐๐ถ๐ผ๐ป: 1๏ธโฃย Pretrain an LLM using the next token prediction on 1.1T token 2๏ธโฃย Continue pretraining of the LLM with retrieval augmentation (Retro-fitting) on an additional 100 billion tokens while retrieving from whole pretraining dataset. 3๏ธโฃย Instruction tune the model, with retrieval augmentation. Only update the weights of the decoder. ๐๐ป๐๐ถ๐ด๐ต๐๐: ๐ย Continued pretraining of LLMs with retrieval yields better decoders for QA. ๐ย Up to 10% improvements doing retrieval augmented pretraining ๐ย Pretraining with retrieval improves the incorporating of context ๐งฎย Retrieval database had 19 billion chunks, with each chunk containing 64 tokens ๐ย Retrieval pretraining is slower and more complex due index and retrieval step Check out the full paper: https://lnkd.in/d2R4zsQ5 Remember that these are just my personal findings. Make sure always to conduct your own research and analysis. ๐ค
To view or add a comment, sign in
-
-
Great work Thomas Dybdahl Ahle and Phoebe Klett! Being able to generate citations and attribution is essential for most real-world GenAI use cases. For the most part roughly the same RAG approaches have been leveraged across the board between low-stakes consumer applications and higher-stakes enterprise applications. What happens if we are willing to take full advantage of an LLM's attention mechanism, meaning we ultimately make more queries to the VectorDB? We see that are able to achieve state-of-the-art improvements in reliability (eliminating fact-based hallucinations) and retrieval quality. "Very smart and simple approach that simplifies a lot the RAG architecture. Just let the LLM work!" Thanks for the shout Diego Pelรกez Paquico. In general, we believe at Normal that Enterprise AI architectures need to be built with fundamentally different assumptions in mind compared to the lower-stakes environments where we've designed today's consensus GenAI stack. Daniel Kahneman might say that enterprise needs the System 2 of AI -- typically more intensive, but significantly more reliable. Our latest release casts further light on specific approaches we are using that fill this category. Read more about the work and Hugging Face release here https://lnkd.in/dkq2e3Z8.
Can we pre-train LLMs with Retrieval Augmentation? ๐คย RETRO was a research by Google DeepMind, which included retrieval into the pre-trainng process. Now NVIDIA continues this research by scaling RETRO to 48B, where they continued pretraining a 43B GPT model on an additional 100 billion tokens using the Retrieval augmentation method by retrieving from 1.2 trillion tokens. ๐คฏ ๐๐บ๐ฝ๐น๐ฒ๐บ๐ฒ๐ป๐๐ฎ๐๐ถ๐ผ๐ป: 1๏ธโฃย Pretrain an LLM using the next token prediction on 1.1T token 2๏ธโฃย Continue pretraining of the LLM with retrieval augmentation (Retro-fitting) on an additional 100 billion tokens while retrieving from whole pretraining dataset. 3๏ธโฃย Instruction tune the model, with retrieval augmentation. Only update the weights of the decoder. ๐๐ป๐๐ถ๐ด๐ต๐๐: ๐ย Continued pretraining of LLMs with retrieval yields better decoders for QA. ๐ย Up to 10% improvements doing retrieval augmented pretraining ๐ย Pretraining with retrieval improves the incorporating of context ๐งฎย Retrieval database had 19 billion chunks, with each chunk containing 64 tokens ๐ย Retrieval pretraining is slower and more complex due index and retrieval step Check out the full paper: https://lnkd.in/d2R4zsQ5 Remember that these are just my personal findings. Make sure always to conduct your own research and analysis. ๐ค
To view or add a comment, sign in
-
-
So Normal Computing just released a first version of what I believe is the way. To be able to generate text based on facts (memories) they combine the attention operations with a vector database, loaded with these external memories KV cache vectors. Then, at generation time, instead of computing attention against all the memories, calculate the usual attention queries, query the vector DB with them and continue the attention operations with the values retrieved! Very smart and simple approach that simplifies a lot the RAG architecture. Just let the LLM work! I see that the current challenges are to adapt this technique to other positional encodings and I suppose the generations speed slows significantly (didn't test this), because querying a vector DB at each transformer layer must add significant latency, but this is very promising. Thanks Phoebe Klett and Thomas Dybdahl Ahle ! https://lnkd.in/dHQ9QxHX
Can we pre-train LLMs with Retrieval Augmentation? ๐คย RETRO was a research by Google DeepMind, which included retrieval into the pre-trainng process. Now NVIDIA continues this research by scaling RETRO to 48B, where they continued pretraining a 43B GPT model on an additional 100 billion tokens using the Retrieval augmentation method by retrieving from 1.2 trillion tokens. ๐คฏ ๐๐บ๐ฝ๐น๐ฒ๐บ๐ฒ๐ป๐๐ฎ๐๐ถ๐ผ๐ป: 1๏ธโฃย Pretrain an LLM using the next token prediction on 1.1T token 2๏ธโฃย Continue pretraining of the LLM with retrieval augmentation (Retro-fitting) on an additional 100 billion tokens while retrieving from whole pretraining dataset. 3๏ธโฃย Instruction tune the model, with retrieval augmentation. Only update the weights of the decoder. ๐๐ป๐๐ถ๐ด๐ต๐๐: ๐ย Continued pretraining of LLMs with retrieval yields better decoders for QA. ๐ย Up to 10% improvements doing retrieval augmented pretraining ๐ย Pretraining with retrieval improves the incorporating of context ๐งฎย Retrieval database had 19 billion chunks, with each chunk containing 64 tokens ๐ย Retrieval pretraining is slower and more complex due index and retrieval step Check out the full paper: https://lnkd.in/d2R4zsQ5 Remember that these are just my personal findings. Make sure always to conduct your own research and analysis. ๐ค
To view or add a comment, sign in
-
-
โก๏ธ What is Machine Learning? ๐ฐ Machine Learning is a subset of artificial intelligence that focuses on developing algorithms and models that enable computers to learn from and make predictions or decisions based on data, without being explicitly programmed. It involves the use of statistical techniques to enable systems to improve their performance on a specific task through experience. Follow for more #interviewquestions #machineleaning
To view or add a comment, sign in
-
-
Building Business Solutions with AI | Generative AI Developer @Startino | Software Developer | Author @Medium @Hashnode
๐ Transform Your Deep Learning Workflows with TensorFlow's `tf.data.Dataset` API! ๐ Are you looking to optimize your data pipelines and enhance the efficiency of your machine learning models? TensorFlow's tf.data.Dataset API is here to optimize the way you handle data. Providing features like parallel computing, batch processing, caching, prefetching, etc. You can create custom pipelines for greater flexibility, easier debugging, advanced CPU and GPU utilization, and real time support Read the entire blog for a deeper understanding. I have also provided the link of my google colab notebook for practical implementation using a Convolutional Neural Network. Blog: https://lnkd.in/dstEc_ck Code: https://lnkd.in/duuNWAQa #MachineLearning #Deeplearning #DataScience #TensorFlow #ArtificalIntelligence #DataPipelines #Efficiency #Scalability #BigData #ParallelProcessing #NeuralNetworks #MemoryOptimization
To view or add a comment, sign in
-
Optimization and machine learning are deeply intertwined, with optimization serving as a fundamental tool in various aspects of machine learning. In machine learning, optimization algorithms are used to minimize or maximize an objective function, such as a loss function in training models or an evaluation metric in hyperparameter tuning. Optimization techniques like gradient descent, stochastic gradient descent, and their variants are central to optimizing model parameters and improving model performance. Additionally, optimization methods are employed in feature selection, constraint handling, ensemble learning, and other areas to enhance the overall effectiveness and efficiency of machine learning algorithms and models. Thus, optimization forms a critical foundation for the success and advancement of machine learning techniques and applications.
To view or add a comment, sign in
-
Co-Founder, Chief AI & Analytics Advisor @ InstaDataHelp | Innovator and Patent-Holder in Gen AI and LLM | Data Science Thought Leader and Blogger | FRSS(UK) FSASS FRIOASD | 16+ Years of Excellence
Theano: Bridging the Gap Between Theory and Practice in Machine Learning Theano: Bridging the Gap Between Theory and Practice in Machine Learning Introduction: Machine learning has revolutionized various industries by enabling computers to learn from data and make predictions or decisions without being explicitly programmed. This field has seen significant advancements in recent years, thanks to the development of powerful frameworks and libraries. One such framework [โฆ] https://lnkd.in/dSpj4SM4
Theano: Bridging the Gap Between Theory and Practice in Machine Learning
https://instadatahelp.com
To view or add a comment, sign in
-
๐๐จ๐๐๐ฒ'๐ฌ ๐ญ๐จ๐ฉ๐ข๐ ๐ฆ๐๐ค๐๐ฌ ๐ฒ๐จ๐ฎ ๐ญ๐ก๐ข๐ง๐ค! ... Why does training Large Language Models (LLMs) take a significant amount of time? Let me guess some factors to consider: ๐. ๐๐๐ญ๐ ๐๐จ๐ฅ๐ฎ๐ฆ๐: LLMs require massive datasets for training. ๐. ๐๐จ๐ฆ๐ฉ๐ฅ๐๐ฑ ๐๐จ๐๐๐ฅ: Models like GPT-3 and GPT-4 have billions to trillions of parameters. ๐. ๐๐๐ซ๐๐ฐ๐๐ซ๐ ๐๐ข๐ฆ๐ข๐ญ๐๐ญ๐ข๐จ๐ง: Training on GPUs or TPUs can still be time-consuming. NVIDIA become so powerful because of these offerings. ๐. ๐๐ญ๐๐ซ๐๐ญ๐ข๐ฏ๐ ๐๐๐๐ข๐ง๐๐ฆ๐๐ง๐ญ: Training involves multiple iterations for fine-tuning. and many other factors. But I believe the primary reason is quite fundamental. It usually comes down to thinking from First Principles. Let's not exaggerate too much; instead, let's understand where this issue begins. Matrix multiplication is a significant challenge and a current bottleneck. The complexity of performing matrix multiplications is substantial, typically being O(n^3). Although efforts have managed to reduce it to approximately O(n^2.34), it remains a substantial obstacle. ๐๐ก๐ฒ ๐๐๐ญ๐ซ๐ข๐ฑ ๐๐ฎ๐ฅ๐ญ๐ข๐ฉ๐ฅ๐ข๐๐๐ญ๐ข๐จ๐ง ๐๐๐ญ๐ญ๐๐ซ๐ฌ? ๐๐๐ฉ๐ซ๐๐ฌ๐๐ง๐ญ๐๐ญ๐ข๐จ๐ง: In LLMs, words, sentences, and code are represented as numerical vectors or matrices. Understanding the relationships between words and predicting the next word often involves linear algebra operations, primarily matrix multiplications. ๐๐๐๐ฅ๐: The matrices involved in LLMs are massive due to large vocabularies and long input sequences. Multiplying these repeatedly becomes incredibly computationally expensive. ๐๐ซ๐๐ง๐ฌ๐๐จ๐ซ๐ฆ๐๐ซ๐ฌ: Modern LLM architectures, like Transformers, heavily rely on attention mechanisms. Attention calculations essentially involve numerous matrix multiplications for each word in a sequence. In conclusion, the significant challenge in training large language models often stems from the fundamental bottleneck of matrix multiplication complexity. must read DeepMind publication: https://lnkd.in/geykHYVm Happy Sunday, Keep Learning Follow my group: https://lnkd.in/gD4E-jAU #dataanalytics #data #datascientist #machinelearningengineer #generatieveai #deeplearning #llm #llmops
To view or add a comment, sign in
-
Theano: Bridging the Gap Between Theory and Practice in Machine Learning Theano: Bridging the Gap Between Theory and Practice in Machine Learning Introduction: Machine learning has revolutionized various industries by enabling computers to learn from data and make predictions or decisions without being explicitly programmed. This field has seen significant advancements in recent years, thanks to the development of powerful frameworks and libraries. One such framework [โฆ] https://lnkd.in/dYGRAprn
Theano: Bridging the Gap Between Theory and Practice in Machine Learning
https://instadatahelp.com
To view or add a comment, sign in
-
Theano: Bridging the Gap Between Theory and Practice in Machine Learning Theano: Bridging the Gap Between Theory and Practice in Machine Learning Introduction: Machine learning has revolutionized various industries by enabling computers to learn from data and make predictions or decisions without being explicitly programmed. This field has seen significant advancements in recent years, thanks to the development of powerful frameworks and libraries. One such framework [โฆ] https://lnkd.in/dYGRAprn
Theano: Bridging the Gap Between Theory and Practice in Machine Learning
https://instadatahelp.com
To view or add a comment, sign in
Founder | Chief Investment Officer| Advisory Board AIF CAT II | Smallcase Fund Manager | Startup | Investor | ex TCS, Aditya Birla | AI enthusiast
8mo"Great insights on reducing precision and enhancing efficiency in model training and inference! Your expertise in Generative AI is truly inspiring. Keep sharing this valuable knowledge!"