AI Product Engineer

🦆 QuackChat: Your AI News Digest 🚀 Exciting developments in the AI world today: 🎨 Image Generation Battle: Flux vs Ideogram (ex-Google Imagen team) hits 1 billion images generated! Are they the new kings of AI art? 🧠 Function Calling Face-off GPT-4 leads the pack, but open-source alternatives like Functionary Llama are making waves. What's your go-to for function calling? 🐘 Microsoft's Phi-3.5 Models New models with 128K token context. It's like giving AI an elephant's memory! How could this change your AI implementations? 🤖 Aider v0.51.0 by Paul Gauthier: AI Writing AI This tool wrote 56% of its own code in the latest release. Are we entering an era of self-improving AI? 🚗 Waymo's Self-Driving Success $130M revenue run rate and 100,000+ trips weekly. Is your industry ready for the autonomous revolution? Dive deeper into these topics in our full QuackChat episode. Link in comments! #AINews #TechInnovation #MachineLearning #FutureOfWork #QuackChat

We are launching Evidently AI on Product Hunt and would love your support! 🎉 After putting a lot of effort into LLM evaluation and observability, we are rolling out some cool open-source updates, including new templates for LLM judges. They make it easy to configure evals for your custom criteria. • 🏆 Define criteria in plain English.  • 💬 Let Evidently create the prompt with formatting and reasoning. • ▶️ Run evals. Run over datasets or traces. • 📊 Get results. Get reports, run tests, or set up a live dashboard. With this latest update, Evidently has all the open-source infrastructure to build and manage LLM evaluation workflow: from the metric backend and testing interface to a self-hostable dashboard to track results. This is a fully open-source launch! Help us spread the word, and let us know what you think about the latest features. Link in the comments 👇

We're buzzing with excitement after last night's incredible AI dinner hosted by TitanML in Toronto! 🍽️💡As the AI Product Engineer community - empowering developers to build real-world AI applications - we were thrilled to be part of this event. Despite the pouring rain, over 20 AI enthusiasts gathered, creating an electric atmosphere of innovation and collaboration. 🌧️ Discussions were passionate and aligned perfectly with our mission to democratize AI development: * Cutting-edge trends in Generative AI * Practical challenges in AI implementation * Innovative solutions for AI scalability and security These topics resonate deeply with our community's focus on hands-on guides and techniques for integrating AI into apps and websites. A massive shoutout to TitanML for organizing this event and bringing together such a diverse group of AI leaders, researchers, and practitioners. It's events like these that fuel our commitment to filling the knowledge gap for experts seeking to leverage AI in their products. We loved having discussions with Shashank Shekhar, Patrick Tammer, Michelle Choi, Pahul H., Razy Shafiee, Akbar Nurlybayev, Joshua C., Tural Gulmammadov, David Hostler, Jason Carayanniotis, Rod Rivera and all the attendees. These interactions reinforce our goal of providing accessible explanations and code examples for production-ready AI. Interested in building the AI products of the future? Join our community at AI Product Engineer! #AINetworking #TechTO #GenerativeAI #AICommunity #AIProductDevelopment #TMLS2024

We will be watching the Euro Cup final with other AI Product Engineers and the TitanML team in Toronto. Join us! RSVP: https://lu.ma/pv8xduzp #TMLS2024

This GenAI Observability quadrant raises more questions than it answers. The categorization lumps together tools that serve vastly different purposes. Take Splunk and Datadog - these are standard IT infrastructure tools. How do they suddenly become GenAI observability leaders? Then we have Evidently AI , LangChain, and Weights & Biases in the same space. These tools couldn't be more different: * Evidently is for evaluation of AI products * LangChain is a framework for developing LLM applications * Weights & Biases is primarily for experiment tracking in ML Calling all of these 'GenAI Observability' tools is like calling a hammer, a paintbrush, and a camera all 'home improvement tools'. It's technically true, but misses crucial distinctions. And what about Elastic and New Relic? Are we now saying every data / infra tool is a GenAI tool? This overly broad categorization does a disservice to both buyers and vendors. For buyers, it muddies the waters. How can they make informed decisions when tools for infrastructure monitoring, ML experiment tracking, and LLM development are all presented as equivalent? For vendors, it creates a 'me too' environment where everyone claims to do GenAI, diluting the meaning of actual GenAI-specific tools. We need more nuanced, use-case specific classifications. Otherwise, we risk turning 'GenAI' into a meaningless buzzword that encompasses everything and nothing. This quadrant, while well-intentioned, falls short of providing truly useful guidance in its current form.

AI models don't understand text like we do. They use "tokens" - small pieces of words or characters - which can lead to unexpected results and biases. This tokenization is crucial for identifying patterns in sequences, whether it's language or time series data. Key points to consider: * Tokenization can cause issues with math, capitalization, and non-English languages. * Some languages need up to 10 times more tokens than English for the same meaning. * While tokenization has limitations, it's essential for pattern recognition in sequences. * Alternative approaches like full sequence embeddings also involve summarization. How might these tokenization challenges affect AI applications in your field? Can we develop new architectures that overcome these limitations while preserving the benefits of sequence segmentation?

CEO Dario Amodei of Anthropic disclosed on the "In Good Company" podcast in late June that compute accounts for over 80% of their expenses. Does that mean running Anthropic costs $1B/year?👇 With approximately 600 employees and estimated average compensation of $250,000 (based on Glassdoor data), Anthropic's annual salary costs are around $150 million. Assuming 70% of compensation is cash, that's about $100 million in yearly cash salary expenses. If we estimate that salaries represent only 15% of total expenses (with 5% for other overhead), it suggests Anthropic's annual operating costs could be approaching $1 billion. This staggering figure contextualizes Anthropic's recent $7.3 billion fundraising efforts. It underscores the immense capital requirements for companies at the forefront of AI research and development. Key questions for our industry: 1) Is this level of spending sustainable in the long term? 2) How might this impact the competitive landscape in AI? 3) What strategies could reduce costs without compromising innovation? Link to the podcast (in the comments)

This week, in #mlopszoomcamp by DataTalksClub, after the deployment module it was time for the monitoring module, focused on Evidently AI and Grafana Labs. ➡ Evidently AI makes it easier to monitor and understand the performance of ML models in real-world conditions. We used several tools, including: ▪ Concept Drift Detection, to detect concept drift, which occurs when the statistical properties of the target variable change. This part is critical for maintaining the model's accuracy and relevance. ▪ Dashboards, for monitoring dashboards to track model performance and data quality. This helps identify performance drops or data issues early. ▪ Reports, to analyze the model's behavior over time, it is also important to generate detailed reports, which provide insights into potential areas of improvement. ➡ Grafana Labs is an open-source monitoring and observability platform that provides advanced real-time data visualizations. We used: ▪ Configuration-based Provisioning, to recreate dashboards using configuration-based resource provisioning, ensuring consistency and ease of deployment. ▪ Integration with Various Data Sources, Grafana allows integration with many data sources, making data visualization flexible and comprehensive. ▪ Plot Time Series Data, creating dashboards that track time series data, helping study changes in performance metrics over time. The combination of these two tools allows for extremely comprehensive monitoring and ensures that our ML models are robust and reliable over time. A big thank you to Alexey Grigorev for this course. Let's dive into the last module, about best practices 🚀

This post brings back memories of a conversation I had with Vladimir Vapnik, one of the inventors of Support Vector Machines, back in 2015. Vapnik, with his characteristic wit, claimed that deep learning was "the work of the devil" – not because of its intelligence, but its persistence. It's a perspective that's stuck with me over the years. My own journey with deep learning has been a winding one. I started out enamored with the elegance of Gaussian processes, appreciating their additivity, explainability, and ability to incorporate business assumptions. I also explored other approaches like Topological Data Analysis and Causal methods, drawn to their mathematical beauty. However, as the field evolved, it became clear that for many practical applications, these methods couldn't match the performance of deep learning techniques like CNNs. What finally won me over to deep learning was the incredible power of unsupervised learning using vector embeddings. The ability to cluster data without labels – essentially FAISS before FAISS existed – was a game-changer for me. While I agree that some early predictions about deep learning were overoptimistic (like Hinton's famous radiologist quote), and we're still working on challenges like fully autonomous vehicles, the impact of deep learning can't be overstated. It's transformed machine learning into an accessible engineering discipline, opening doors for millions of computer scientists and enthusiasts alike. Remember, before deep learning, even tackling a simple classification problem required a deep understanding of Hilbert spaces! The field was largely limited to those with PhDs and strong mathematical backgrounds. Deep learning democratized AI, allowing a diverse range of people to build upon it, many without formal computer science education. So while deep learning might not be perfect, it's currently the most powerful and accessible tool we have. It's exciting to see how far we've come and to imagine where we might go next. I'm curious about your own journey with deep learning. Were you an early adopter, jumping in around the time of AlexNet? Or did you come to the field more recently?

Perfect opportunity to learn first-hand best practices for agents for any AI Product Engineers based in Miami! 👇