Forecasting Time Series Data with Prophet: Build, improve, and optimize time series forecasting models using Meta's advanced forecasting tool

BIRMINGHAM—MUMBAI

Forecasting Time Series Data with Prophet

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Publishing Product Manager: Aditya Datar

Content Development Editor: Shreya Moharir

Technical Editor: Sweety Pagaria

Copy Editor: Safis Editing

Project Coordinator: Farheen Fathima

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Production Designer: Joshua Misquitta

Marketing Coordinators: Shifa Ansari, Vinishka Kalra

First published: March 2021

Second published: April 2023

Production reference: 1230323

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83763-041-7

www.packtpub.com

Contributors

About the author

Greg Rafferty is a data scientist at Google in San Francisco, California. With over a decade of experience, he has worked with many of the top firms in tech, including Facebook (Meta) and IBM. Greg has been an instructor in business analytics on Coursera and has led face-to-face workshops with industry professionals in data science and analytics. With both an MBA and a degree in engineering, he is able to work across the spectrum of data science and communicate with both technical experts and non-technical consumers of data alike.

About the reviewers

Jose Angel Sanchez, born and raised in Oaxaca, Mexico, is a software developer at Pinterest. Previously, Jose worked at Bayer, Credijusto, and Connus International, and throughout his career has had the opportunity to work with different technologies and solve problems in different disciplines. A math lover and a crypto-believer, he knows that only through science and skeptical thinking will the human race achieve its true potential. Jose lives happily with his wife, Mariana, and their dog, Koly, in St Louis, Missouri.

Shuvojyoti Acharjee is a data engineer enthusiast holding a master’s degree in big data and technology from the University of Westminster, London. Shuvojyoti has worked in multiple industries including customer service, research analysis, and hospitality and currently works as a manager at Travelex. 

I would love to thank my friend, Rakhi Mondal, for helping me complete this book. She has been a great motivation for me and has always been there for me.

Table of Contents

Preface

Part 1: Getting Started with Prophet

1

The History and Development of Time Series Forecasting

Understanding time series forecasting

The problem with dependent data

Moving averages and exponential smoothing

ARIMA

ARCH/GARCH

Neural networks

Prophet

Recent developments

NeuralProphet

Google’s robust time series forecasting at scale

LinkedIn’s Silverkite/Greykite

Uber’s Orbit

Summary

2

Getting Started with Prophet

Technical requirements

Installing Prophet

Installation on macOS

Installation on Windows

Installation on Linux

Building a simple model in Prophet

Interpreting the forecast DataFrame

Understanding components plots

Summary

3

How Prophet Works

Technical requirements

Facebook’s motivation for building Prophet

Analyst-in-the-loop forecasting

The math behind Prophet

Linear growth

Logistic growth

Seasonality

Holidays

Summary

Part 2: Seasonality, Tuning, and Advanced Features

4

Handling Non-Daily Data

Technical requirements

Using monthly data

Using sub-daily data

Using data with regular gaps

Summary

5

Working with Seasonality

Technical requirements

Understanding additive versus multiplicative seasonality

Controlling seasonality with the Fourier order

Adding custom seasonalities

Adding conditional seasonalities

Regularizing seasonality

Global seasonality regularization

Local seasonality regularization

Summary

6

Forecasting Holiday Effects

Technical requirements

Adding default country holidays

Adding default state/province holidays

Creating custom holidays

Creating multi-day holidays

Regularizing holidays

Global holiday regularization

Individual holiday regularization

Summary

7

Controlling Growth Modes

Technical requirements

Applying linear growth

Understanding the logistic function

Saturating forecasts

Increasing logistic growth

Non-constant cap

Decreasing logistic growth

Applying flat growth

Creating a custom trend

Summary

8

Influencing Trend Changepoints

Technical requirements

Automatic trend changepoint detection

Default changepoint detection

Regularizing changepoints

Specifying custom changepoint locations

Summary

9

Including Additional Regressors

Technical requirements

Adding binary regressors

Adding continuous regressors

Interpreting the regressor coefficients

Summary

10

Accounting for Outliers and Special Events

Technical requirements

Correcting outliers that cause seasonality swings

Correcting outliers that cause wide uncertainty intervals

Detecting outliers automatically

Winsorizing

Standard deviation

The moving average

Error standard deviation

Modeling outliers as special events

Modeling shocks such as COVID-19 lockdowns

Summary

11

Managing Uncertainty Intervals

Technical requirements

Modeling uncertainty in trends

Modeling uncertainty in seasonality

Summary

Part 3: Diagnostics and Evaluation

12

Performing Cross-Validation

Technical requirements

Performing k-fold cross-validation

Performing forward-chaining cross-validation

Creating the Prophet cross-validation DataFrame

Parallelizing cross-validation

Summary

13

Evaluating Performance Metrics

Technical requirements

Understanding Prophet’s metrics

Mean squared error

Root mean squared error

Mean absolute error

Mean absolute percent error

Median absolute percent error

Symmetric mean absolute percent error

Coverage

Choosing the best metric

Creating a Prophet performance metrics DataFrame

Handling irregular cut-offs

Tuning hyperparameters with grid search

Summary

14

Productionalizing Prophet

Technical requirements

Saving a model

Updating a fitted model

Making interactive plots with Plotly

Plotly forecast plot

Plotly components plot

Plotly single component plot

Plotly seasonality plot

Summary

Index

Other Books You May Enjoy

Preface

In 2017, Facebook (now Meta) released its Prophet software as open source. This powerful tool was developed by Facebook engineers because its analysts were overwhelmed with the number of business forecasts demanded by managers. The developers of Prophet wanted to simultaneously solve two problems: 1) completely automatic forecasting techniques are too brittle and inflexible to handle additional knowledge, and 2) analysts who are consistently able to produce high-quality forecasts are rare and require extensive expertise. Prophet successfully solved both of these problems.

Prophet was designed so that forecasts produced with no parameter tuning or other optimizations are usually very high-quality. Nevertheless, with just a little bit of training, anyone can intuitively tweak the model and increase performance dramatically.

Starting from the most basic model and advancing to the deepest technical dives into Prophet’s inner workings, this book will teach you everything there is to know about Prophet. Many advanced features not even covered by the official documentation are discussed here, with complete working examples for every topic covered. This book is not intended to provide you with the ability to build a Prophet clone from scratch, but it will teach you how to use Prophet just as well as, if not better than, Meta’s own highly trained engineers.

In the years since the first edition of this book was published, the world has changed dramatically. The global COVID pandemic disrupted every forecaster’s predictions, and we all are still struggling to learn how to forecast in this new world. The second edition of this book includes an update on our understanding of how to forecast during these types of unexpected events.

Additionally, since the first edition, Prophet has seen many updates, including graduating from beta status and releasing an official version 1! We have updated every section and code block in this second edition with all of the new features and changes in Prophet since the publication of the first edition.

The data science teams of several other companies have also open sourced their own forecasting packages in recent years, and we have included a new discussion of NeuralProphet, LinkedIn’s Greykite, and Uber’s Orbit and their strengths and weaknesses compared to Prophet. Because of the encouraging feedback from so many readers of the first edition, we have written an entirely new chapter all about the math behind Prophet. This new chapter will give you a richer understanding of how to build the very best forecasts for your domain and provide the knowledge to explain to stakeholders exactly how your forecasts have been developed.

This second edition is a big update over the first and I can’t wait to hear from all of you about your forecasting work!

Who this book is for

This book is for business managers, data scientists, data analysts, machine learning engineers, and software engineers who want to build time series forecasts in Python or R. To get the most out of this book, you should have a basic understanding of time series data and be able to differentiate it from other types of data. Basic knowledge of forecasting techniques is a plus.

What this book covers

Chapter 1, The History and Development of Time Series Forecasting, will teach you about the earliest efforts to understand time series data and the main algorithmic developments up to the present day.

Chapter 2, Getting Started with Prophet, will walk you through the process of getting Prophet running on your machine, and then will test your installation by building your first model.

Chapter 3, How Prophet Works, will discuss why Facebook (now Meta) decided to build its own forecasting package and how the philosophy of analyst-in-the-loop forecasting applies to Prophet. This chapter will also present the mathematical equations underpinning the forecasting algorithms within Prophet.

Chapter 4, Handling Non-Daily Data, will cover how to modify the approach taken in Chapter 2, Getting Started with Prophet, in order to handle data that is recorded on a scale other than daily, so that you will be set up to work through all of the examples in later chapters.

Chapter 5, Working with Seasonality, will discuss all of the ways to control seasonality in Prophet. Seasonality is one of the building blocks of Prophet models and contains the most control parameters, so this chapter is the longest but also one of the most important.

Chapter 6, Forecasting Holiday Effects, will teach you how to add the effect of holidays to your forecast. You will learn how to include a basic set of default holidays, how to change that set for different regions, how to add your own custom holidays, and how to control the strength of the effect.

Chapter 7, Controlling Growth Modes, will describe the three growth modes a trend line in Prophet can follow: linear, logistic, and flat. You will learn which scenarios to apply these modes to and observe what effect they have on your future forecasts.

Chapter 8, Influencing Trend Changepoints, will talk about how to control the rigidity of your final model. You will learn how to make a flexible model that can change direction often or a rigid model that follows a constant line, why you may choose one or the other, and about the effect this has on the uncertainty of your model being used on future data.

Chapter 9, Including Additional Regressors, will teach you how to include additional columns of data in your model. Similar to multi-variate regression, Prophet is able to combine multiple input vectors in a predictive forecast.

Chapter 10, Accounting for Outliers and Special Events, will show you the two types of problems that outliers can cause in a Prophet model and will teach you several automated techniques for identifying outliers and how to handle them with Prophet.

Chapter 11, Managing Uncertainty Intervals, will cover how to quantify the uncertainty in your model using different statistical methods, what the benefits and drawbacks of each method are, and how to visualize the amount of risk in your model.

Chapter 12, Performing Cross-Validation, will teach you how to perform cross-validation in Prophet. You may already be familiar with cross-validation techniques in machine learning, but with time-series data, a different approach is needed. This chapter will teach you that approach and how to implement it in Prophet.

Chapter 13, Evaluating Performance Metrics, will build upon the previous chapter and introduce the performance metrics Prophet features. You will learn how to combine cross-validation with your chosen performance metric to carry out a grid search and optimize your model to gain the highest predictive accuracy.

Chapter 14, Productionalizing Prophet, is the final chapter and will teach you some additional techniques that will come in handy when using Prophet in a production environment. You will learn how to save your models for later use, how to update models as new data comes in, and how to use Prophet’s Plotly plot functions to build highly interactive charts suitable for sharing on a web-based dashboard.

To get the most out of this book

To run the code examples in this book, you will need Python 3.x installed. All examples in this book were made using Prophet version 1.1 in Jupyter notebooks. macOS, Windows, and Linux are all supported. Although all examples in this book will be written in Python, everything is also fully compatible with R and you may use that language if you prefer, although this book will not cover R syntax. Please refer to the official Prophet documentation for R syntax (https://facebook.github.io/prophet/).

Chapter 2, Getting Started with Prophet, will walk you through installing Prophet, and installing either Anaconda or Miniconda is strongly recommended in order to correctly install all of Prophet’s dependencies. It is possible to install Prophet without using Anaconda, but it can be very difficult depending upon the specific configuration of your machine, and this book assumes that Anaconda will be used.

In order to follow the examples, you must at least be familiar with the pandas library for data processing and Matplotlib for making plots. In a few cases, the numpy library will be used to simulate random data but following the examples will not require that you know the NumPy syntax. All of these libraries will be installed automatically as Prophet dependencies, if not already installed. All datasets used in this book are hosted and can be downloaded from this book’s GitHub repository here: https://github.com/PacktPublishing/Forecasting-Time-Series-Data-with-Prophet-Second-Edition.

Prophet supports parallelization with Dask but while setting Prophet up to run on a Dask cluster will be covered, installing and using Dask is beyond the scope of this book. If interested, I encourage you to refer to the Dask documentation at https://docs.dask.org/en/stable/. Similarly, this book will cover how to build interactive Prophet visualizations in Plotly but putting those together into a Dash dashboard will be left up to the reader to learn elsewhere. The Dash documentation at https://dash.plotly.com/ is a great place to start.

If you are using the digital version of this book, we advise you to type the code yourself or access the code via the GitHub repository (link available in the next section). Doing so will help you avoid any potential errors related to copying/pasting code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Forecasting-Time-Series-Data-with-Prophet-Second-Edition. In case there’s an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: To control Prophet’s automatic changepoint detection, you can modify both of these values with the n_changepoints and changepoint_range arguments during model instantiation.

A block of code is set as follows:

model = Prophet(seasonality_mode='multiplicative',

                yearly_seasonality=4,

                n_changepoints=5)

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

model = Prophet()

model.fit(df)

future = model.make_future_dataframe(

periods=60, freq='MS'

)

forecast = model.predict(future)

fig = model.plot(forecast)

plt.show()

Any command-line input or output is written as follows:

pip install pystan

pip install prophet

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: In the following seasonality plot, I’ve used the Toggle Spike Lines and Compare Data buttons from this toolbar to add further information to the hover tooltip.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read Forecasting Time Series Data with Prophet, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow