![U.S. flag](https://www.dol.gov/themes/opa_theme/img/flag-favicon-57.png)
An official website of the United States government
In January 2015, the U.S. Bureau of Labor Statistics (BLS) implemented two improvements to the Chained Consumer Price Index for All Urban Consumers (C-CPI-U). First, the revision schedule of the index series was changed from an annual revision to a quarterly revision. As a result of the more frequent revisions, the lag between the initial and final publication of each monthly index was reduced by an average of 11 months.1 Second, and the focus of this article, the index formula used to calculate the initial version of the index was changed from an adjusted Geometric Young formula to a constant elasticity of substitution (CES) formula. The new formula was implemented to more closely approximate the final indexes than the previous Geometric Young formula.
The objective of the BLS Consumer Price Index (CPI) is to approximate a cost-of-living index (COLI). This is accomplished by addressing the following question: What is the cost, at the current month’s market prices, of achieving the standard of living attained in the base period?2 Within the context of consumer utility-maximization behavior, this cost represents the change in spending needed to attain a constant level of consumer satisfaction when faced with prices in future periods that differ from those in the base period. The headline Consumer Price Index for All Urban Consumers (CPI-U) assumes that consumers respond to price change by purchasing the same quantities as in the past and does not reflect consumer substitution in response to relative price change for a defined biennial period. The CPI-U, therefore, is an overstatement of a COLI if, within a biennial period, consumers change their consumption patterns in response to changing relative prices, like buying fewer goods at relatively higher prices and substituting with comparable goods at relatively lower prices.
In contrast to the CPI-U, the C-CPI-U, first introduced with the release of July 2002 CPI data, employs a formula that reflects the effect of substitution that consumers make across item categories in response to changes in relative prices.3 The final C-CPI-U is a superlative index4 and is a second-order approximation of a COLI.5 The final C-CPI-U better approximates a COLI for several reasons, not the least of which is due to the weights. The weights used are derived from the current period and base period, the periods over which price change is measured. A first-order approximation like the CPI-U uses weights from only one of the periods. The monthly weights for the C-CPI-U formula directly reflect substitutions that consumers make when facing relative price change.
Prior to the release of the January 2015 CPI-U, three upper-level aggregation formulas were calculated for BLS CPI products: Lowe (used for the CPI-U and the Consumer Price Index for Urban Wage Earners and Clerical Workers), Tornqvist (used for the final C-CPI-U), and Geometric Young (used for preliminary versions of the C-CPI-U).6 Starting with the release of the January 2015 CPI-U, the Lloyd–Moulton CES formula (referred to as CES for the remainder of this article) replaced the Geometric Young formula for preliminary versions of the C-CPI-U.7 Each of these formulas use the same measure of elementary item-area price change as an input: (IXt/IXt-1). However, the expenditure weight input varies on the basis of assumption of the overall consumers’ substitution elasticity (σ). The four formulas are summarized in table 1.
The CPI-U is calculated with a Lowe fixed quantity formula, in which quantities are represented by price-updated expenditures.8 This formula assumes that substitution elasticity is zero within a biennial period, meaning consumers do not substitute across items in response to relative price change. The difference between price change estimated using fixed quantity weights and the actual quantities purchased is referred to as upper-level substitution bias in the CPI-U.
A Tornqvist formula, which includes consumer substitution using the current and previous month weights, is used to calculate final C-CPI-U. BLS is currently unable to produce the final C-CPI-U in real time because the monthly expenditure weights lags the current index period by about four quarters. This time lag is necessary to conduct and process the Consumer Expenditure Survey household data, the source of the monthly weight estimates. BLS publishes preliminary C-CPI-U indexes, designed to approximate the final C-CPI-U index, to address the time gap.
Up to the January 2015 CPI-U release, preliminary C-CPI-U indexes were estimated with a Geometric Young formula that assumes substitution elasticity to be unitary within a biennial period. The underlying assumption of this estimator implied consumers always substitute to items whose prices are falling relative to items whose prices are increasing. Since the release of the January 2015 CPI-U, the preliminary versions of the C-CPI-U have been estimated with a CES index formula, which assumes consumer substitution is constant within a biennial time period and the level of substitution is estimated by prior period behavior. The remainder of this article describes the process behind making this formula change to the preliminary C-CPI-U, and the resulting improvement. Table 1 shows the price index formula relationships among different CPI products. Figure 1 displays a comparison of index levels from these formulas with December 1999 = 100.
When the C-CPI-U was introduced in 2002, the Geometric Young formula was selected over the CES formula as a, “plausible, simpler approximate of the Tornqvist in real time.”9 At that time, CES was characterized as a possible alternative. However, many issues related to the substitution elasticity estimate could not be resolved, including a variable estimate of substitution elasticity, an unstable value across months, and a lagged estimate not representative of the current index period.
A variable substitution elasticity estimate would, in principal, be linked to the CPI elementary level of indexes to construct an entire demand system. The CPI elementary structure consists of more than 200 consumption items and 38 geographic areas. This structure would theoretically be based on over 20,000 substitution pairs of items ((211 * 210)/2) in each of the 38 areas. Creating a representative measure of substitution across the requisite elementary cells is not feasible for index estimation; therefore, research shifted to establishing a single “constant” estimate of substitution elasticity.
Robert Cage, John Greenlees, and Patrick Jackman attempted to determine which substitution parameter most consistently produced a CES index that came closest to the final Tornqvist index, based on data from 1987 to 2000. To do this, they calculated monthly CES indexes using the substitution parameter values 0.1, 0.5, and 0.9. Then, the optimal parameter value was selected in which the resulting CES index was closest to the final Tornqvist index. For the original analysis of a constant estimate of substitution elasticity, the stability of the estimator was cited as a major weakness of CES and research showed monthly substitution values that ranged from 0.06 to 2.78.10
Research by John Greenlees in 2010 demonstrated that many of the earlier cited weaknesses could be effectively addressed.11 First, as indicated above, a constant estimate of substitution elasticity could replace a variable estimator. Second, using a model to pool data over longer periods created a more stable estimator by minimizing the impact of monthly price changes. Finally, fixed biennial expenditure shares could be replaced by weights that are updated to an index month comparable to the CPI-U aggregation weights.
Furthermore, Greenlees showed the CES formula could outperform the Geometric Young formula using empirical CPI data, resulting in smaller C-CPI-U revisions and smaller estimates of prediction error. Subsequent research focused on evaluating which preliminary C-CPI-U formula is the best estimate of the final C-CPI-U. The following sections explain why CES formula performed better than the Geometric Young formula for estimation of the preliminary C-CPI-U.
Sigma
Sigma is the measure of elasticity of substitution in the CES formula. To estimate sigma, Greenlees used a regression model developed by Robert Feenstra and Marshall Reinsdorf.12 The model estimates consumer elasticity of substitution as a function of the relationship between the relative change in expenditure shares and the relative change in prices. Greenlees calculated sigma values that ranged from 0.56 to 0.64 and effectively addressed the weakness of an unstable sigma by using the Feenstra–Reinsdorf model and pooling data over annual periods.
The limitations of using a constant elasticity-of-substitution measure for all items and areas remain. The level of consumer substitution between two items such as gasoline and public transportation may be much different from the level of substitution between a different pair, such as coffee and tea. The Feenstra–Reinsdorf pooled regression model, however, measures the average level of consumer substitution across all elementary items and areas to produce a constant sigma value. So the assumption that consumer substitution across all pairs of elementary items is constant remains.
The Feenstra–Reinsdorf model, shown in equation (1), is summarized as the change in expenditures because of a change in price. Sigma is measured as 1 minus the resulting beta (β) of the Feenstra–Reinsdorf pooled regression. The annual expenditure shares are calculated using elementary level annual expenditure weights. The annual average index relatives are calculated using elementary level 12-month average price index relatives.
The Feenstra–Reinsdorf pooled regression over annual periods is expressed as follows:
,(1)
where
is the natural log annual expenditure weight share at the item area stratum level,
is the natural log annual index relative at the item area stratum level,
is the annual reference period of expenditure weights and indexes, and
is the previous annual reference year of expenditure weights and indexes.
The Feenstra–Reinsdorf formula, equation (1), is weighted as follows:
,(2)
where AEWS is the annual expenditure weight share at the item area stratum level.
Greenlees used annual expenditure weight and price index data from 1999 to 2008, and recommended excluding items with large decreasing index relatives. He calculated a relatively stable sigma value that ranged from 0.58 to 0.64, pooled across annual periods.
The results from the Greenlees article are based on an analysis of annual data; however, estimating an optimal sigma using annualized data from biennial expenditure reference periods would procedurally coincide with the weight updates of the CPI-U as demonstrated in equation (3).
The Feenstra–Reinsdorf pooled regression over biennial periods is expressed as follows:
,(3)
where
is the natural log biennial expenditure weight share at the item area stratum level,
is the natural log biennial index relative at the item area stratum level,
is the biennial reference period of expenditure weights and indexes, and
is the previous annual reference year of expenditure weights and indexes.
The above Feenstra–Reinsdorf formula, equation (3), is weighted as follows:
,(4)
where BEWS is the biennial expenditure weight share at the item area stratum level.
Sigma results based on a biennial time frame are summarized in table 2. For the biennial relative years 2003–04:2001–02, the biennial expenditure weight share is equal to the natural log expenditure share and the 24-month average price indexes are based on natural log average price indexes from 2003–04 relative to 2001–02 time frame. The pooled results summarize the Beta values across biennial time periods, and then the sigma pooled is equal to 1 minus beta pooled.
Biennial period | Relative period | Beta | Pooled biennial periods from 2003-04 | Beta pooled | Sigma pooled | Constant elasticity of substitution index period |
---|---|---|---|---|---|---|
2003–04 | 2001–02 | 0.398 | to 2003–04 | 0.398 | 0.602 | ― |
2004–05 | 2003–04 | 0.479 | to 2005–06 | 0.438 | 0.562 | ― |
2007–08 | 2005–06 | 0.324 | to 2007–08 | 0.403 | 0.597 | ― |
2009–10 | 2007–08 | 0.493 | to 2009–10 | 0.417 | 0.583 | ― |
2011–12 | 2009–10 | 0.522 | to 2011–12 | 0.438 | 0.562 | 2014–15 |
2013–14 | 2011–12 | 0.229 | to 2013–14 | 0.421 | 0.579 | 2016–17 |
Source: U.S. Bureau of Labor Statistics. |
Beta values range from 0.229 to 0.522 over the 6 biennial periods. Beta averaged across the 6 biennial periods is equal to 0.408, comparable to the 2003–04 to 2013–14 beta pooled result of 0.421. Sigma pooled values range from 0.562 to 0.602 across the pooled biennial periods. The CPI has elected to use a sigma value of 0.6, which equals the rounded sigma pooled value for the current period, as well as the previous 5 pooled biennial periods.13
Revisions to the value of sigma will be considered in conjunction with biennial weight revisions, which occur in January of even years. The criteria for making a revision to the sigma value will be based on evaluating the size of changes to the pooled estimate of sigma, and evaluating potential index direction bias. Small changes to sigma have a limited impact on the resulting CES index. Therefore, sigma will only be updated if its optimal value changes by greater than 0.1.
Updated CES weights
The Geometric Young formula uses biennial expenditure weights from 12 to 24 months before the index month. For instance, the initial indexes in 2012 used weights that reflect expenditures made by consumers in 2009 and 2010. The CES formula uses biennial expenditure weights that are more current because the price portion is updated to the current index pivot month by using index relatives, as shown in equation (5). For example, 2009–10 biennial weights are updated by index relatives to represent expenditures in December 2011. Equation (5) displays how the CES expenditure weights are updated to the pivot period. This weight update process is comparable to the last step of aggregation weight calculation. There, elementary level expenditure weights are adjusted by the corresponding elementary level index in order to calculate an expenditure weight value with an implicit price in the pivot month and an implicit quantity of the weight reference period.14
The CES pivoted expenditure weight for biennial reference periods is expressed as follows:
,(5)
where
is the elementary level annualized expenditure weight from the biennial reference period,
is the elementary level 24 month average index from the biennial reference period,
is the sigma for the biennial reference weight period, and
is the elementary level annualized expenditure weight from a biennial reference period in which prices are from a pivot period adjusted for consumer substitution.
CES index relatives
The CPI preliminary estimate of 1-month price change, calculated using the CES formula, is derived from the ratio of CES current and previous month indexes, relative to the pivot month, as shown in equation (6). The pivot-month index is updated every 24 months in conjunction with the biennial weight revisions. The CES functional form also contains the substitution parameters as exponents applied at the elementary-level index relatives, and then at the aggregate level. To create a constant level of substitution across elementary level cells, the same sigma must be applied to the elementary level index relative and the resulting aggregate level index relative.
The CES index for relative biennial periods is expressed as follows:
(6)
Distributional analysis of optimal sigma
The following analysis extends the distributional analysis of sigma prepared by Cage, Greenlees, and Jackman by evaluating biennial, annual, and monthly weights to create a real time CES that uses previous period weights. The optimal sigma for each period is the value yielding the closest CES estimate of the final C-CPI-U index. Sigmas ranging from 0 to 1, incremented by 0.01, were tested, as shown in equations (7) and (8). Analysis of the optimal sigma was capped at 1 because in theory, sigma of 1 will result in unitary substitution preferences and therefore will be equal to the Geometric Young formula. CES index relatives are trimmed values in which elementary-level (item/area) relatives greater than 10 are set equal to a value of 10 and relatives less than a value of 0.1 are set equal to a value of 0.1. A histogram of the optimal values is shown in figure 1.
,(7)
(8)
where
is the weight periodicity (biennial, annual, monthly) from the previous period;
is the sigma for component index period, and aggregate index period ranging from .01 to .99, incremented by .01; and
is the final aggregate index relative.
Optimal sigmas are relatively similar across weight periodicity as shown in figure 2, which displays a count of months with optimal sigmas from 2000–12. The largest difference of absolute values across weight periodicities was 7 months for bins 0.1, 0.3, 0.7, and 0.9. The overall average across weight periods is approximately 0.7, which indicates that the weight periodicity potentially has a minimal impact when optimal sigma is evaluated for this type of analysis.
Index analysis of optimal sigma
The optimal sigma producing the best CES estimate relative to the final C-CPI-U index varies across index months. The index months can roughly be divided into three optimal sigma periods, as displayed by the periods to the left, center, and right of the gray highlight in figure 3. From January 2000 to December 2003, the optimal sigma is 0.999, indicating unitary substitution preferences. From January 2004 to December 2007, the optimal sigma is 0.8, and from January 2008 to December 2012, the optimal sigma is 0.7.
The following analysis compares revision sizes of the final C-CPI-U less the CES and Geometric Young formulas to evaluate how predictive these preliminary formulas are of the final C-CPI-U. This analysis excludes revision chaining to isolate the impact of the preliminary formula.15
For a 1-month change, the revision size of CES is smaller than Geometric Young for 101 out of 156 months (65 percent), as displayed in figure 4. The size of revision associated with each formula is summarized in absolute terms in table 3. For a sum of absolute revision size, CES is 77 percent of Geometric Young. For a mean of absolute errors, CES is 78 percent of Geometric Young. For the root mean squared error revision size, CES is 66 percent of Geometric Young. Each of these measures of revision size of a 1-month change from the CES formula is lower than the Geometric Young formula, indicating the CES formula more closely predicted the final C-CPI-U than the Geometric Young formula.16
Values | Geometric Young formula | Constant elasticity of substitution (CES) formula | CES/Geometric Young |
---|---|---|---|
Sum absolute | 8.46 | 6.55 | 77 |
Mean absolute | 0.055 | 0.043 | 78 |
Root mean square error absolute | 0.050 | 0.033 | 66 |
Source: U.S. Bureau of Labor Statistics. |
For 12-month change, the revision size of CES is smaller than Geometric Young for 83 out of 132 months (63 percent) as displayed in figure 5. The size of revision is summarized in absolute terms in table 4. For a sum of absolute revision size, CES is 65 percent of Geometric Young. For a mean of absolute revision size, CES is 65 percent of Geometric Young. For the root mean square error of revision size, CES is 57 percent of Geometric Young. Each of these measures of revision size of 12-month change is lower for CES than the Geometric Young formula, indicating the CES formula more closely predicts the final C-CPI-U than the Geometric Young formula.
Values | Geometric Young formula | Constant elasticity of substitution (CES) formula | CES/Geometric Young |
---|---|---|---|
Sum absolute | 26.43 | 17.09 | 65 |
Mean absolute | 0.200 | 0.129 | 65 |
Root mean square error absolute | 0.168 | 0.095 | 57 |
Source: U.S. Bureau of Labor Statistics. |
For the 2002 introduction of the C-CPI-U, the CPI opted to implement the Geometric Young formula for the preliminary version. This formula was, in large part, chosen over the CES formula because of problems with the CES formula identified by Cage et al., such as the instability of a monthly sigma. In 2010, Greenlees effectively resolved the instability of sigma with a Feenstra–Reinsdorf regression model that pooled data across biennial periods. The CPI has also identified another improvement in the CES formula: the incorporation of weights updated to a more current pivot period, comparable to the aggregation weights of the CPI-U. This implementation of the CES formula produces a better estimate of the final C-CPI-U than the Geometric Young formula because, for most months, the revisions are smaller and the corresponding absolute revision sizes and root mean square errors are smaller. The updated CES formula allows for a defined level substitution within a biennial time frame that is more flexible than the unitary substitution preferences of the Geometric Young formula, in which substitution across items occurs in response to price change, and more flexible than the substitution preferences of the Lowe formula, in which substitution across items does not occur in response to price change. The CPI implemented the CES formula with the release of the January 2015 CPI-U for the preliminary version of the C-CPI-U. This analysis supports the Greenlees 2010 analysis that the CES formula with a sigma equal to 0.6 approximates the final C-CPI-U better than the Geometric Young formula.
Joshua Klick, "Improving initial estimates of the Chained Consumer Price Index," Monthly Labor Review, U.S. Bureau of Labor Statistics, February 2018, https://doi.org/10.21916/mlr.2018.6
1 See Consumer Price Index: October 2014, USDL-14-2106 (U.S. Bureau of Labor Statistics, November 20, 2014), https://www.bls.gov/news.release/archives/cpi_11202014.pdf.
2 BLS handbook of methods, chapter 17, “The Consumer Price Index” (U.S. Bureau of Labor Statistics, June 2015), p. 2, https://www.bls.gov/opub/hom/pdf/homch17.pdf.
4 Robert Cage, John Greenlees, and Patrick Jackman, “Introducing the chained Consumer Price Index” (paper presented at the Seventh Meeting of the International Working Group on Price Indices, Paris, France, 2003), p. 213, http://www.ottawagroup.org/Ottawa/ottawagroup.nsf/home/Meeting+7/$file/2003+7th+Meeting+-+Cage+&+Greenlees+&+Jackman+-+Introducing+the+Chained+Consumer+Price+Index.pdf.
5 W. E. Diewert, “Exact and superlative index numbers,” Journal of Econometrics 4, 1976, p. 117; Practical guide to producing consumer price indices (International Labour Organization, 2009), p. 3, http://www.ilo.org/public/english/bureau/stat/download/cpi/guide1.pdf.
6 Upper-level estimation uses these price index formulas to aggregate elementary indexes and elementary weights into a final product such as the all-items U.S. city average, the default aggregation level used throughout this article.
7 See P.J. Lloyd, “Substitution effects and bias in nontrue price indexes,” American Economic Review, 1975, vol. 65, issue 3, pp. 301–13; Brent Moulton, “Constant elasticity cost of living index in share relative form,” unpublished manuscript (U.S. Bureau of Labor Statistics, 1996).
8 Consumer price index manual: theory and practice (International Labour Organization, 2004), pg. 6, par. 1.48, http://www.ilo.org/wcmsp5/groups/public/---dgreports/---stat/documents/presentation/wcms_331153.pdf.
9 Cage, et al., “Introducing the chained Consumer Price Index,” p. 228.
10 Ibid.
11 John Greenlees, “Improving preliminary values of the chained CPI-U,” Business and Economic Statistics Section—JSM, 2010, https://www.bls.gov/osmr/research-papers/2010/pdf/st100060.pdf.
12 Robert C. Feenstra and Marshall B. Reinsdorf, “Should exact index numbers have standard errors? Theory and application to Asian growth” (paper presented at the CRIW Conference, September 2003), http://cid.econ.ucdavis.edu/Papers/pdf/Feenstra-Reinsdorf.pdf.
13 Note that this model is sensitive to outlier values, particularly large decreases, because of natural logged index relatives. To mitigate the impact of outliers for the sigma analysis below, index relatives less than 0.25 were set at 0.25. Six cells were trimmed over the 2001–02 to 2013–14 biennial periods: two cells during 2001–02, one cell during 2005–06, and three cells during 2009–10.
14 BLS handbook of methods, chapter 17, p. 33.
15 The published initial C-CPI-U CES formula is revised for 3 quarters in succession by chaining to the terminal month final C-CPI-U. In contrast, the discontinued published C-CPI-U Geometric Young formula is revised annually by chaining to the terminal month final C-CPI-U. For more information, see BLS handbook of methods, chapter 17, p. 34 https://www.bls.gov/opub/hom/pdf/homch17.pdf; BLS News Release, Consumer Price Index- January 2015, p. 5, https://www.bls.gov/news.release/archives/cpi_02262015.pdf.
16 Results support Statistical Canada CPI analysis of price index formulas to reduce substitution bias that results from an asymmetric formula such as Lowe. For more information, see Ning Huang, Waruna Wimalaratne, and Brent Pollard, “Choice of index number formula and the upper-level substitution bias in the Canadian CPI” (paper presented at the 14th Ottawa Group Meeting, May 2015), http://www.stat.go.jp/english/info/meetings/og2015/pdf/t1s4p13_pap.pdf.