Helping 7,000+ learn Data Science for Business | Marketing Analytics | Time Series Forecasting | Quantitative Finance || @mdancho84 on Twitter
Bayes' Theorem is a fundamental concept in data science. But it took me 2 years to understand its importance. In 2 minutes, I'll share my best findings over the last 2 years exploring Bayesian Statistics. Let's go. 1. Background: "An Essay towards solving a Problem in the Doctrine of Chances," was published in 1763, two years after Bayes' death. In this essay, Bayes addressed the problem of inverse probability, which is the basis of what is now known as Bayesian probability. 2. Bayes' Theorem: Bayes' Theorem provides a mathematical formula to update the probability for a hypothesis as more evidence or information becomes available. It essentially describes how to revise existing predictions or theories in light of new evidence, a process known as Bayesian inference. 3. Bayesian Statistics: Bayesian Statistics is an approach to statistics that interprets probability as a measure of belief or certainty rather than just a frequency. This belief may be based on prior knowledge of the conditions that might be related to the event or experiment in question. This allows for making probabilistic statements about unknown parameters. For instance, instead of estimating a single value for a parameter, Bayesian statistics provides a distribution of possible values, reflecting the uncertainty. 4. Bayesian vs Frequentist: Bayesian inference is fundamentally about updating beliefs or probabilities as new data is observed, which can be very intuitive and aligns with how we often think about the world. Frequentist statistics interpret probability as the long-run frequency of events. The problem I have with frequentist approaches is that pre-determined distributions are used (e.g. Normal Gaussian), which does not always make sense. 5. Bayesian Machine Learning: Any time true confidence and probabilistic decision making is needed, Bayesian is the answer. Here are a couple of examples. Uncertainty Modeling: Unlike traditional machine learning methods that often provide point estimates, Bayesian methods focus on estimating distributions. Time-Series Analysis: Bayesian methods are particularly useful in time-series analysis, where uncertainty in the future is crucial. 6. Business Context: Businesses can use Bayes' Theorem to assess and quantify various risks, such as market risks, credit risks, or operational risks. By continuously updating the probability of risks as new information emerges, businesses can make more informed decisions. === Ready to learn Data Science for Business? I put together a free on-demand workshop that covers the 10 skills that helped me make the transition to Data Scientist: https://lnkd.in/gbEBVf5f And if you'd like to speed it up, I have a live workshop where I'll share how to use ChatGPT for Data Science: https://lnkd.in/gCvh6UAy If you like this post, please reshare ♻️ it so others can get value.
Fantastic post! (Bookmark worthy) What an intellectual boost to kick off the new year! Thank you, Matt! 💎⭐️ Happy new year! 😊🙏
Thanks for posting
Great post, Matt. Idea: why not add Dall-e to your course to show how much fun this can become:
"The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from" provided me the needed clarity on why, what and how of Bayes in a way no other book did..
Frequentist approach is a bit out of intuition they will say certain events have certain predetermined distributions and then work on central limit theorem using averages, but the mathematical foundations of those are based on assumptions that are rarely maintained unless under extremely controlled environments. Bayesian control systems are good for ML but for quality control and assurance ANOVA and t distributions and other frequentists approach are largely used.
Bayes theorem is indeed very important to making forecasts that are more intuitive with the way we think in everyday life, when we update our view on a topic with every new information on that topic.
Bayes Theorem has been very interesting topic right from my college days and the interest kept growing with the real-world applications of this concept in compliance, and risk management. The fundamental aspect of incorporating new information to modify the probability of an event is interesting unlike the frequency methods.
Bayes' Theorem is the probability of near certainty.
Data Science Manager @ Meta
7moon 4) I believe pre-determined distributions are widely used within a bayesian context. Using a Beta prior to a Binomial is one example. I understand parametric/non-parametric is in a different category than bayesian/frequentist.