Diego Correa Tristain’s Post

Want to know more about how to store automated synthesis data in an HDF5 file structure? Here's an example from our lab: https://lnkd.in/eNDVZpY7

If benchmarks are your thing, and you're curious about how Extism can push over 1GiB/s across the #WebAssembly host/guest boundary, then this is the post for you! https://lnkd.in/gJsCkgee 👏 Thanks to Chris Dickinson for the deep dive!

📢 Check out the new release of PiML, developed by our team of data scientists! This update includes inherently interpretable machine learning and model validation, complete with tests for weakness detection, robustness, reliability, resilience, and fairness. Install or update now to experience it for yourself. #PiML #MachineLearning #ModelValidation #ModelRiskManagement

Pretty cool stuff by Lamini for enterprise use cases where 95% LLM accuracy and no hallucination could be achieved. Better than SOTA LLM + Prompt + RAG? - Memory Tuning tunes a massive mixture of memory experts on any open-source LLM. - Each memory expert acts like a LoRA adapter that functionally operates as memory for the model. - Together, the memory experts specialize in a million different ways to ensure faithful and factual accuracy to the data that it was tuned on. https://lnkd.in/gnMqaacR

BPE__BytePairEncoding https://lnkd.in/dwa_P8Xx bigram = min(pairs, key = lambda pair: self.bpe_ranks.get(pair, float('inf'))) https://lnkd.in/dfFFJsyA The only literal byte pair left occurs only once, and the encoding might stop here. Alternatively, the process could continue with recursive byte pair encoding, replacing "ZY" with "X": XdXac X=ZY Y=ab Z=aa

RDF Dataset Canonicalization becomes a W3C recommendation. It provides a standard RDF Dataset canonicalization algorithm #rdf #graph #linkeddata #webstandards #knowledgegraphs #graph #database https://lnkd.in/e2WYdt43

If you want to know more about certifying local explanations, follow this work at ICML next week!

What is blocking LLMs from using long context length inputs? 🚨Introducing KVQuant which allows serving LLaMA-7B with 1M tokens on a single A100! 🔥 Current largest model is Claude-2.1 which is limited to 200K tokens. What is the challenge for increasing this? Two key problems: (i) Memory wall: the need to store Key and Value activations for each token, leads to a major issue for long context lengths: we quickly go out of memory even on 8 GPU systems. You can’t run LLaMA-70B with >32K on an A100 because of this. (ii) Catastrophic forgetting: LLMs do not pay attention to long sequences and either only focus on the beginning or the end. So far, training on long documents, and better positional encoding has enabled up to 200K context length, but further scaling to longer context lengths without accuracy drop is an open problem. KVQuant addresses the first challenge, and enables large context length inference by quantizing the cached Key/Value activations to ultra low precision without accuracy degradation by considering several consistent patterns observed in cached KV values across different LLMs. Why does this matter? The ability to consume such large context sizes could help increase the accuracy of LLMs by allowing one to provide different in context examples or include large files/contents in a general LLM and make it competitive with a fine-tuned model that is trained on such data. For instance, consider coding co-pilot. 10M context length provides enough space for the LLM to consume ~1M lines of code, which could help unlock new insights. Please see https://lnkd.in/gmtrsK3P for a quick TLDR of our method. Paper: https://lnkd.in/g6y7Yw3x  Code: https://lnkd.in/gPmnNAET Joint work with: Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael Mahoney, Sophia Shao, Kurt Keutzer

Leveraging phi-3 for an Enhanced Semantic Cache in RAG Applications. #RAG applications utilize a semantic cache layer and phi-3, a #SLM from Microsoft, to address the challenge of repeat queries while maintaining contextually accurate and diverse responses https://zurl.co/vHg3

Leveraging phi-3 for an Enhanced Semantic Cache in RAG Applications. #RAG applications utilize a semantic cache layer and phi-3, a #SLM from Microsoft, to address the challenge of repeat queries while maintaining contextually accurate and diverse responses https://zurl.co/vHg3