Artjom Shestajev’s Post

Learning Regular Languages with L* Algorithm https://lnkd.in/d8hmjVcr

Prompt writing is an art, but evaluating how my prompt does is all science. One of the first tasks I asked an LLM to do was categorize a meeting just by using a part of the transcript from the meeting. And guess what?! I nailed it on the three examples I was testing. And they were GOOD examples, one was an interview, one was a sales call, and one was an engineering stand-up. It was about a day after we put the model into production that I started to hear about all the places where the model was failing. It turns out that the way we speak to people in our meetings is A LOT more complicated than I set up initially.   I thought the LLM could figure it out with just a few context clues, but that was not the case.  Here’s how I test scalability now: 1) I collect 30-100 examples (depending on the task) and create a dataset for experimentation. 2) I clean the data and write a first draft of the prompt.   -I do this in Python, because it’s easy for me to quickly write loops and changes in a language I’m comfortable in. 3) I evaluate the prompt across the entire dataset. Sometimes it’s a simple yes/no, and sometimes I bring in a person to help give extensive feedback on each result.   -I have quality expectations going into the experiment. If my prompt isn’t at least 50% acceptable on my entire dataset, then I go back to the drawing board (new data, new prompt, or BOTH). Every prompt I write requires extensive iteration in order to be scalable. 

All models are wrong but some are useful. My Python professor used this quote by George Box while teaching predictive modeling last sem, and although I didn't fully grasp it at the time, I now realise how applicable it is to everyday life. It ties in to what mathematician Alfred Korzybski said - *the map is not the territory*. In other words, the models we use to make sense of reality aren't reality. The description of something, no matter how detailed, is not the thing itself. Reality is endlessly complex, and so, our mind creates maps or abstractions of it in order to deal with it better. And these abstractions are valuable. Think about actual maps - if they were made to scale, they would be more accurate, but we wouldn't be able to use them. The problem is, we usually don't understand the limits of our maps. Take someone who believes in outlandish conspiracy theories. They selectively gather and interpret information that supports their belief while dismissing or ignoring contradictory evidence. They don't entertain the possibility of their maps being wrong. According to Korzybski, even the best and most useful maps suffer from limitations like: ❌ Factual inaccuracies 🗺 Maps are based on reductions and abstractions, a process in which you're forced to ignore crucial information 👨🔧 People interpret maps wrong all the time To conclude, mental maps and models are incredibility useful for us to make sense of the world, analyse different situations, and plan for the future. However, it's important to be aware of their limiations, and realise that they are not the same as reality. #mentalmodels #maps #predictivemodeling #limitations

Score: GPT4 1, DH 0. I've written literally 000s of finance PQs, so I thought I'd try to write a good PQ that might stump the LLM. (My thesis is that LLMs disrupt domain *expertise* and therefore render obsolete many traditional assessments, but enable better PQs and meta-PQs). Here's my first try. However, both GPT4 and Claude3-O *aced* my question. Yikes, I need to do better. My thinking was that a human analyst can deduce or "see" the answer without any calculations, while the LLM might get tripped up in the calcs. But my hypothesis was incorrect (note #1). * GPT4: https://lnkd.in/eATgxGgy * Claude: https://lnkd.in/emfz7RuR #1: Well, I was mostly wrong and a little bit right. The above link is my "clean, fair test" of GPT4, but in yesterday's test GPT4 gave a wrong answer with correct python code. In that conversation, it impressively wrote both correct R and correct python code, but somehow didn't execute the python code correctly. For that conversation, see https://lnkd.in/eh_AMbxq (I did clean memory between tests, btw)

I recently discovered an 𝗶𝗻𝗰𝗿𝗲𝗱𝗶𝗯𝗹𝘆 𝘂𝘀𝗲𝗳𝘂𝗹 𝗟𝗟𝗠 𝗰𝗼𝗺𝗽𝗮𝗿𝗶𝘀𝗼𝗻 𝘄𝗲𝗯𝘀𝗶𝘁𝗲 that can help anyone find the best LLM for their use case! Thanks to Vellum, you can get the most updated information on the market's LLMs: - 📊 Compare LLMs based on reasoning skills (MMLU), Math, Python Coding, future capabilities… - 👨💻 LLMs Coding Leaderboard - 💰 Cost and context window comparisons - 📑 Access to technical reports of major models: GPT, Gemini, Claude, Mistral, and more. Instead of searching each model's website, you can get a comprehensive overview and 𝘁𝗶𝗺𝗲-𝘀𝗮𝘃𝗶𝗻𝗴 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀 into the LLM ecosystem with this website. 𝗗𝗼 𝘆𝗼𝘂 𝗸𝗻𝗼𝘄 𝗼𝗳 𝗮𝗻𝘆 𝗼𝘁𝗵𝗲𝗿 𝘄𝗲𝗯𝘀𝗶𝘁𝗲𝘀 𝘁𝗵𝗮𝘁 𝗰𝗼𝗺𝗽𝗮𝗿𝗲 𝗟𝗟𝗠𝘀 𝗼𝗿 𝗼𝘁𝗵𝗲𝗿 𝗶𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗶𝗻𝗴 𝗔𝗜 𝗺𝗼𝗱𝗲𝗹𝘀? Let me know in the comments! You can access Vellum LLMs Leaderboard here: https://lnkd.in/d8QeDK_r … 🔄 Share this post and Follow Agustin Fernandez for more similar content. #LLM #LargeLanguageModels #AIInsights #GenAI

🦜 𝗟𝗟𝗠-𝗣𝗼𝘄𝗲𝗿𝗲𝗱 𝗔𝗽𝗽 𝘄𝗶𝘁𝗵 𝗟𝗮𝗻𝗴𝗖𝗵𝗮𝗶𝗻 𝗮𝗻𝗱 𝗦𝘁𝗿𝗲𝗮𝗺𝗹𝗶𝘁 𝗶𝗻 𝗝𝘂𝘀𝘁 𝟭𝟴 𝗟𝗶𝗻𝗲𝘀 𝗼𝗳 𝗖𝗼𝗱𝗲 LangChain is a powerful framework designed to harness the capabilities of Large Language Models (LLMs) for building versatile applications. LangChain’s strength lies in its six core modules: • 𝗠𝗼𝗱𝗲𝗹 𝗜/𝗢: Seamlessly integrates model inputs (prompts) with the LLM, producing outputs through advanced parsing techniques. • 𝗗𝗮𝘁𝗮 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻: Handles loading, transforming, storing, and querying of user data with tools like document loaders, transformers, embedding models, and vector stores. • 𝗠𝗲𝗺𝗼𝗿𝘆: Equips chains or agents with both short-term and long-term memory, enabling them to recall previous interactions. • 𝗖𝗵𝗮𝗶𝗻𝘀: Allows you to combine multiple components or chains into a single, cohesive pipeline. • 𝗔𝗴𝗲𝗻𝘁𝘀: Makes intelligent decisions on what actions to take based on the input, utilizing available tools and data. • 𝗖𝗮𝗹𝗹𝗯𝗮𝗰𝗸𝘀: Executes specific functions at key points during an LLM run. Streamlit is a leading Python library for building web apps in machine learning and data science. Now, you can create a basic LLM-powered app using LangChain and Streamlit in just 18 lines of code. Check out the code link in the comments! #LLM #LangChain #Streamlit #Python #MachineLearning #DataScience #AI #WebApp #Coding #TechInnovation #OpenSource #ArtificialIntelligence #Programming #ML #DataEngineering

Chain of Thought can improve your LLM’s performance, but it’s slower + more expensive. Instead, you can use Constrained CoT for prompt engineering. Here’s how: You know that Chain of Thought (CoT) prompts give detailed reasoning steps in LLM responses for better accuracy. But Constrained Chain of Thought (CCoT) prompts also add length constraints to these reasoning steps, as an instruction to the prompt. So it’s able to maintain both accuracy AND conciseness. You can use CCoT in just 3 lines of code with the QualityPrompts library, by simply calling the .constrained_chain_of_thought_prompting() method. In the below example I tried, a typical CoT prompt generates a lengthy response. While a CCoT prompt, with a specified word limit, produces a succinct answer while still being accurate. Link to the Python library: https://lnkd.in/gN5eHby5 Paper for CCoT: https://lnkd.in/gjnRuPUZ #LLMs #AI

Today's language, SNOBOL, is both related to RegEx and obscure, but was taught in Brooklyn College in the 1980s (it was big from the 1960s through 1980s in universities, and interestingly in Humanities). It was pattern matching intended for string manipulation and an early but advanced OOP (object oriented programming language). There were four iterations (I only ever used the last one, SNOBOL4 as well as SPITBOL, the compiled version). There was no IF or looping structures; it was all (gasp) GOTOs! Not only that, but unlike most languages supporting GOTOs, SNOBOL allowed the GOTO destination to be the value of a dynamic variable (known as an 'indirect goto'); it also supported tables and arrays.   Here's the Sieve of Eratosthenes (ancient algorithm for finding prime numbers, credit to Peter M. Maurer for the code snippet), note how looping is an artificial construct:   *         define the sieve data structure *         everything is potentially prime until proven otherwise           CANDIDATES = ARRAY('0:1000', 1) *         Neither 1 nor 0 is prime, so flag them off           CANDIDATES<0> = 0           CANDIDATES<1> = 0 *         start the sieve with the integer 0           I = 0 LOOP1     LT(I,1000)                                :F(ELOOP1) *                  advance to the next un-crossed out number. *                  this number must be a prime           EQ(CANDIDATES<I>,0)                       :F(ELOOP2)                   I = I + 1                                 :(LOOP1) ELOOP2          J = 2           K = I * J LOOP3     LT(K,1000)                                :F(ELOOP3)           CANDIDATES<K> = 0           J = J + 1           K = I * J                                 :(LOOP3) *                  advance to the next candidate ELOOP3    I = I + 1                                 :(LOOP1) ELOOP1          I = 0 *         all uncrossed-out numbers are prime (and only those numbers) *         print all primes LOOP4     LT(I,1000)                                :F(ELOOP4)           NE(CANDIDATES<I>,0)                       :F(SKIP1)                   OUTPUT = I " is prime" SKIP1     I = I + 1                                 :(LOOP4) ELOOP4    I = 0 END

Recursion: Leave it or Deal with the Negative Consequences This is my new paper-like post that is available at the following url: https://lnkd.in/dVA_bvzS If you like it, share it as you wish. May the force be with you! Abstract "The moment we acquire knowledge of computer programming—mainly at a professional level—recursion is a technique we definitely avoid to use while either designing or implementing solutions for real-world problems. Although recursion has a great link with Math and a tight relation with heuristics related to Artificial Intelligence (AI), helping defining and understanding a problem in an easy way, the negative consequences of its use may become a nightmare during the execution of an algorithm written with any programming language. In order to avoid the use of recursion, even if we rely on compiler-tools optimisations such as Tail Call Elimination, we should always design an iterative solution from scratch if performance is an intended goal. In this paper-like post, we discuss what recursion is, visiting its drawbacks to understand why we must choose an iterative solution every time we can, being prepared to deal with the negative consequences of recursion if no other option is yet available. We present the example of the Fibonacci’s sequence, where our iterative solution outperforms the recursive one."

What is Group Relative Policy Optimization (GRPO)? Deepseek Coder v2 is the best open Code LLM rivaling GPT-4 on coding tasks. As part of the technical report, GRPO is mentioned as RLHF method, but what is it? 🤔 GRPO was introduced in the DeepSeekMath Paper earlier this year and is method in designed to improve improve mathematical reasoning capabilities with less memory consumption. Implementation 1️⃣ Generate multiple outputs for each input question using the current Policy 2️⃣ Score these outputs using a reward model 3️⃣ Average the rewards and use it as a baseline to compute the advantages 4️⃣ Update the Policy to maximize the GRPO objective, which includes the advantages and a KL term Insights 💡 GRPO doesn't need value function model, reducing memory and complexity 🔗 GPRO adds the KL term directly to the loss rather than in the reward 📈 GPRO improved GSM8K and MATH ~5% 👉 GPRO looks similar to RLOO method (available in TRL) 🔁 Used Iterative Approach to train new Reward Models 📊 RL data consisted of 144k CoT prompts from SFT dataset 🧠 Reward Model was trained using “Math-Shepherd” process RL is “boosting the correct response from TopK rather than the enhancement of fundamental capabilities.” DeepSeekMath: https://lnkd.in/eGAk_vbG Math-Shepherd: https://lnkd.in/etUVyBgm