REFinBlog

April 15, 2016

Consumer Survey Surveillance

As I had noted previously, The Consumer Financial Protection Bureau had issued a notice and request for comment on the Financial Well-Being National Survey.  My submitted comments on it are below:

The Consumer Financial Protection Bureau has solicited comment on whether this collection of information is necessary for the proper performance of the functions of the Bureau, including whether the information will have practical utility.  While the Bureau has increased the rigor it has brought to its financial education mission over the last few years, it is unclear what the Financial Well-Being National Survey is meant to measure and it is unclear what it, in fact, will measure.  Specifically, one of the goals of the Survey is to measure the level of financial well-being of American adults, but the survey relies too heavily on the subjective responses of participants to achieve that goal.  We have reason to believe that subjective assessments of financial literacy are suspect.

 Much of the academic literature around financial literacy and financial education is very depressing as it reveals that Americans are not very financially literate at all. Lusardi, Annamaria, Financial Literacy: Do People Know the ABCs of Finance? (November 30, 2014). Global Financial Literacy Excellence Center Working Paper No. 2014-9. Available at https://ssrn.com/abstract=2585246.

Not only is financial literacy in bad shape, but efforts to improve it have not proven to be very effective.  Lauren Willis has provided a sobering, even depressing, overview of what we know about the efficacy of financial education.  Willis, Lauren E., Financial Education: Lessons Not Learned & Lessons Learned (January 31, 2013). Life-Cycle Investing: Financial Education and Consumer Protection 125 (Zvi Bodie et al., eds. 2012); Loyola-LA Legal Studies Paper No. 2013-4. Available at SSRN: https://ssrn.com/abstract=1869313 or https://dx.doi.org/10.2139/ssrn.1869313.

Willis asks, “Does financial education work as hoped?” (125) She answers her own question:  “Empirical evidence does not support the theory. Some (but not all) studies show a positive correlation between financial education and financial knowledge or between financial knowledge and financial outcomes. But no strong empirical evidence validates the theory that financial education leads to household well-being through the pathway of increasing literacy leading to improved behavior.” (125)

Even worse, Willis finds that some people who would have reason to think they are more financially literate because of their participation in financial education initiatives, do even worse than those who did not participate:  “the only statistically significant effect of mandatory personal financial training on soldiers was that they adopted worse household budgeting behaviors after the training than before it.” (126) Some of Willis’ other important conclusions (based on a thorough review of the literature) include

• “Youth who took a personal finance course in high school do not report better financial behavior several years later than youth who did not take the course.  Adults who attended public schools where they were required to take personal financial courses were found to have no better financial outcomes than adults who were not required to take such courses.” (126, citations omitted)
• One “reason financial education is unlikely to produce household financial well-being is that consumers’ knowledge, comprehension, skills, and willpower are far too low in comparison with what our society demands.” (128)

Willis’ conclusions about the efficacy of financial education initiatives are bolstered by a meta-analysis of the literature on financial education that was conducted by researchers at the World Bank.  Their abstract reads,

This paper presents a systematic and comprehensive meta-analysis of the literature on financial education interventions.  The analysis focuses on financial education studies designed to strengthen the financial knowledge and behaviors of consumers. The analysis identifies188 papers and articles that present impact results of interventions designed to increase consumers’ financial knowledge (financial literacy) or skills, attitudes, and behaviors (financial capability). These papers are diverse across a number of dimensions, including objectives of the program intervention, expected outcomes, intensity and duration of the intervention, delivery channel used, and type of population targeted. However, there are a few key outcome indicators where a subset of papers are comparable, including those that address savings behavior, defaults on loans, and financial skills, such as record keeping. The results from the meta analysis indicate that financial literacy and capability interventions can have a positive impact in some areas (increasing savings and promoting financial skills such a record keeping) but not in others (credit default).

Miller, Margaret and Reichelstein, Julia and Salas, Christian and Zia, Bilal, Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature (January 1, 2014). World Bank Policy Research Working Paper No. 6745. Available at https://ssrn.com/abstract=2380391

My first instinct is that there is no harm in conducting the Financial Well-Being National Survey. It asks reasonable questions, such as “How would you assess your overall financial knowledge?” and “How confident are you that the way you are managing money today is getting you to the results you want?” (5) There are also questions that ask concrete questions about the respondents’ financial situation, but they rely on self-reporting.

The key question that remains, then, is will the answers to such questions actually help shape consumer protection policy in a productive way? The Bureau should be sure that the answer to that question is yes before proceeding with the Survey.

I do not suggest that the Bureau jettison this survey, but I do suggest that the Bureau clarify what the Survey is meant to measure and that it ensures that it does measure those things.  To do so, the Survey should be supplemented with studies that attempt to determine how accurate the subjective assessments contained in the Survey are.  For instance, if respondents report that they are confident that they are managing their money effectively, targeted follow-up studies could determine whether that confidence is warranted.  If the respondent reports that his or her home is valued at a certain level (and homeowners are wont to overestimate the value of their homes), follow-up studies can determine whether that valuation was accurate.  Indeed, a well-designed follow-up study could determine the extent to which people overestimate their financial literacy and their financial situation.

It is of great importance that the Bureau gets its financial education initiatives right from the start.  It is worth investing heavily at the outset to ensure that it does.