Building Central Bank Credibility: The Role of Forecast Performance

We use a large online experiment to study the causal relationship between a central bank’s forecasting history and its ability to control inflation expectations with subsequent inflation forecasts, a vital aspect of monetary policy. Participants respond to forecast precision in a theory-consistent way but under-reward a history of highly precise forecasts. Recent forecasting performance heavily influences perceptions of forecast credibility, especially when recent performance was poor. Contextual communication can significantly bolster a bank’s credibility. For policymakers, these findings stress the importance of swiftly addressing forecasting errors, given the difficulty in restoring lost credibility, and underscore the significant role of clear communication during periods of credibility crises. Central banks cannot rest on their laurels when exogenous shocks precipitate a rapid decline in forecast precision.

Introduction:

Controlling inflation expectations is considered to be crucial for central banks operating an inflation-targeting framework because inflation expectations are believed to play a significant role in the determination of inflation (Clarida, 1999; Woodford, 2005; Galí, 2008; King et al., 2008). As communication is one tool that central banks can use to influence inflation expectations, most of central banks nowadays devote considerable resources to operating a multi-faceted communication strategy. These strategies comprise some, or all, of monetary policy statements following policy decisions, minutes of the policy meetings, releasing numerical economic forecasts, contextualization of those forecasts via reports, as well as press interviews, political testimony and speeches. As Bernanke (2015) says: “monetary policy is 98 percent talk and only two percent action”; so-called “open-mouth operations” have become central to monetary policy.

In idealized monetary models with fully informed rational agents, there is no need for the central bank to communicate about inflation as both parties form coincident and optimal inflation expectations. In reality, information is far from perfect, and central banks’ communication may help to shape inflation expectations. The challenge then is to ensure this communication effectively controls these expectations (Blinder, 2000). The effectiveness of inflation forecasts and contextualizing communication hinges critically on the central bank’s forecast credibility. A highly credible central bank may be successful in controlling and anchoring inflation expectations, enabling it to focus on short-run stabilization policies that further enhance its credibility. However, a negative cycle could emerge; a central bank that has low credibility in its inflation forecast finds it difficult to manage inflation expectations, leading to fluctuations in inflation that further undermine its credibility.

Despite this first-order importance of central bank forecast credibility, and perhaps because theoretical assumptions often nullify concerns about credibility, we have precious little evidence regarding the determinants and dynamics of forecast credibility. In McMahon and Rholes (2023), we use large online experiments to provide experimental evidence on the causal relationship between key features of a central bank’s forecasting history and its ability to influence inflation expectations via inflation forecasting. Specifically, we examine how the central bank’s historical forecast performance, and the time profile of forecast errors, influence the bank’s ability to control inflation expectations with its inflation forecasts. Additionally, we study whether contextualizing communication can bolster central bank credibility — that is, whether a central bank can talk its way out of a low-credibility position.

Our Methodology:

In our individual-choice forecasting experiment, U.S.-based participants took part in three independent decision periods. We began each decision period by showing participants the three most recent years of forecast performance by the central bank (the bank’s forecasts alongside the subsequent inflation outturn). Participants provided two sets of one-period-ahead point and range forecasts of inflation (e.g. `Initial Forecasts’ and `Updated Forecasts’). To ensure participants take the forecasting exercise seriously, we incentivize both the point and range forecasts (Rholes and Petersen 2021). Subjects provide Initial Forecasts (priors) for the next period based on historical data. We then reveal the central bank’s own inflation forecast and allow subjects to update their own density projection (i.e. Updated Forecasts, or posterior). Figure 1 shows the timeline. Using the initial and updated forecasts, we can precisely quantify the degree to which a participant incorporates the central bank’s signal into their inflation outlook using a Bayesian signal processing framework and relate this causally to historical economic information we reveal at the start of each decision period.

Figure 1: Decision Period

We conducted numerous histories to show to the participants arranged around two broad issues.1 One set of histories were used to study how historical forecast precision impacts forecast credibility (Forecast Performance) and another set studied how the time profile of forecast errors impacts forecast credibility (Timing). We also study the effect of Contextual Communication.

Forecast Performance Histories

To isolate the causal relationship between historical forecast precision and forecast credibility, we produce five histories in which the central bank exhibits a consistent average annual forecast performance with no discernible pattern to forecast errors. The different histories preserve the time profile of forecast errors but vary the central bank’s historical forecast precision. Precision in these alternative versions of Consistent vary from Consistent-Great performance, through Consistent-Good, Consistent-Moderate, Consistent-Bad, and Consistent-Terrible. We depict these histories in Figure 2a. For our Forecast Performance treatments, we randomly allocate subjects to one of these five versions of Consistent, which subjects always face as the third decision period (N = 528).

Figure 2a: Forecast Performance Histories

Timing Histories

In Early, the central bank commits significant forecast errors in the first third of the forecasting history, moderate errors in the second third, and minimal errors in the last third. We calibrate our simulation so that Early qualitatively matches the Bank of England’s (BoE) forecast performance for the three-year period beginning in the first quarter of 2010 and ending in the final quarter of 2012. In Late, we exactly reverse this pattern of errors. Consistent-Bad, the same as the history used in the forecast performance analysis, is the comparator as it exhibits the same overall forecast error performance, but the errors are consistent across the years of the history. A Bayesian agent who equally weighs all historical information would find the central bank’s forecast equally credible after viewing each of the Early, Late and Consistent-Bad histories. By comparing forecast credibility across these histories, we can study how the timing of forecast errors influences forecast credibility. Additionally, we can assess whether this time dependency varies based on the direction in which forecast performance changes. We depict these histories in Figure 2b.

Figure 2b: Timing Histories

For Timing, we conduct a set of six treatments comprising all permutations of these three histories. We allocate participants randomly to one of these six treatments (N=516).

Contextual Communication Histories

Finally, to study the effect of Contextual Communication, we use a single ordering of histories — Early, Consistent, and Late. Early and Consistent are standard. However, in the Late history we include a written statement alongside the central bank’s numerical forecast in the Late history. We test six different written statements (N=679). One is a control statement explaining the purpose of a central bank. Another includes a written version of the central bank’s outlook in addition to the control statement. The remaining four statements additionally explain the shock to forecast performance as being either exogenous or endogenous and specify whether the central bank under or over-performed relative to peer forecasting institutions.

Key Results

Forecast Performance:

Central bank forecast credibility is causally related to its historical forecast performance in that subjects respond to decreases in historical forecast precision qualitatively like Bayesians. However, the empirical relationship we observe between forecast credibility and forecast precision is flatter than predicted by theory. We show this in Figure 3, where we compare our estimates of forecast credibility (blue dots, U*) to a Bayesian benchmark (connected red triangles). All estimates are surrounded by 99% confidence intervals. In fact, our estimates suggest there is no significant loss in credibility if a central bank’s forecast deteriorates from Good to Terrible. In some sense, this is good news because the forecast credibility cost of large errors may not be quite as high as the theory predicts. However, we observe much stronger evidence that subjects underuse signals from highly precise central banks.

Figure 3: Forecast Credibility in Forecast Performance

Timing:

The primary result from our Timing treatments is that our participants exhibited considerable recency bias when evaluating historical economic information to form a perception of the central bank’s forecast credibility. We show this in Figure 4, where forecast credibility deviates positively in Early from the Bayesian benchmark and negatively in Late. This pattern of deviations is consistent with participants basing their perceptions of forecast credibility primarily on the central bank’s most recent forecast performance. Interestingly, the magnitude of this deviation is about five times larger in Late than in Early, indicating a nonlinearity in recency bias. Our results reveal that it isn’t just the change in forecast precision that drives recency bias. Instead, it also matters in which direction precision changes; poor forecast performance is more salient for participants whenever forming a perception of forecast credibility.

We conduct additional treatments to show this result is robust to the direction of forecast errors (N=518) and if we re-configure our experiment so that we are collecting three-year-ahead inflation expectations rather than short-term inflation expectations (N=504).

Figure 4: Forecast Credibility in Timing

Contextual Communication:

Contextualizing communication, as one might find in the Bank of England’s Monetary Policy Report or a typical FOMC statement, can significantly increase the central bank’s forecast credibility. Figure 5 presents the improvement in the update relative to the Late treatment in the timing experiment. This is true even when the contextualizing communication only reinforces the central bank’s inflation outlook without providing any new information to our participants (Control + Outlook). This could be because participants are better at extracting qualitative or narrative information from text, if the process of reading text yields a better synthesis of information, or if simply seeing the information again but in text form somehow reinforces learning. Additionally, the mere act of attempting to communicate in a helpful manner could boost the central bank’s reputation, as suggested by Haldane and McMahon (2018).

Figure 5: Forecast Credibility in Contextual Communication

Policy Implications:

Our experimental findings underscore a crucial point: forecast credibility relates endogenously to forecasting and, as with other policy instruments, the central bank is not necessarily in full control of its forecast credibility. Such high-frequency, endogenous variation in credibility does not arise in standard models and so warrants further investigation. We show in McMahon and Rholes (2023) that accounting for endogenous credibility and recency bias in an otherwise standard New Keynesian model produces persistent inflation dynamics, thereby better aligning model dynamics with empirical data without assuming highly correlated or persistent shocks.

For policymakers, the message is clear that the central bank cannot rest on its laurels when unanticipated shocks precipitate sharp decreases in forecast performance. This is because such changes in forecast performance undermine the bank’s forecast credibility precisely when the bank most wishes to guide inflation expectations. This is true even when the bank has a track record of sound predictions; public attention will inevitably gravitate towards the recent subpar performance, thus overshadowing prior successes. Additionally, our results reveal that while a rebound in forecast performance following a negative shock to forecast precision can help
in restoring credibility, the pace of recovery is invariably slower than the rate at which credibility initially eroded. It is paramount that central banks guard against and react swiftly to negative forecasting episodes to prevent credibility loss because the cost of rebuilding credibility may significantly outweigh the cost required to maintain it.

Finally, our results suggest a strong role for contextualizing communication during periods of low forecast credibility. Communication during crisis periods is paramount and should both rationalize past forecast performance while reinforcing the central bank’s economic outlook.