There's no question that 2020 was a whirlwind of a year. Humanity faced a pandemic and social unrest of historic precedence. How did these events influence our optimism toward the future? To address this question, a team of IU researchers, including faculty from the Luddy School of Informatics, Computing, and Engineering and IU's Cognitive Science Program turned to Twitter.
In "Quantifying changes in societal optimism from online sentiment," published in Behavior Research Methods, researchers Calvin Isch (B.S. in Cognitive Science ‘20), Research Fellow in the Center for Social and Biomedical Complexity Marijn ten Thij, Provost Professor of Cognitive Science and Psychological and Brain Sciences Peter M. Todd, and Professor of Informatics and Cognitive Science Johan Bollen, developed a method to measure societal optimism from Twitter data by looking at the sentiment with which people talk about different points in the future, ranging from two days to 30 years from now. They analyzed the sentiment of more than 3.5 million tweets, statistically comparing how people discuss different time points in the future, similar to how treasury bond yield curve compares yields at different times to maturity.
The left plot shows a typical (black line) and inverted (yellow) treasury bond yield curve with yield rate on the y-axis and distance in the future on the x-axis. The right plot depicts a typical (blue) and inverted (red) "social optimism curve" which contains the same x-axis but has the average sentiment of tweets (higher is more positive) on the y-axis.
"When the bond yield curve inverts, generally it signals a forthcoming recession because investors see higher economic risk in the near future than in the far future," Bollen said. "Likewise, we find a so-called 'social optimism curve' in our data, which also recently inverted during the COVID-19 pandemic."
Using this information at the scale of entire societies requires an interdisciplinary approach that combines expertise and methods spanning cognitive science and computational science. The collaboration began as the pandemic started, when Isch, working with Todd on how people think about the future, reached out to Bollen and ten Thij to explore ways to leverage their long-standing work on gauging public sentiment from large-scale social media content using AI and Natural Language Processing techniques. The idea emerged to model not only how Twitter sentiment can gauge how hundreds of millions of people feel about the present (i.e., "I am so angry right now") but also how they feel about different time points in the future ("It makes me sad that in a year from now we will…").
The research found that prior to March 2020, users tended to talk about the future in a positive light, and they talked about the near-term future more positively than the distant future. At the start of the pandemic, the positivity of tweets declined drastically but still outpaced negative posts. The decline, however, existed only for tweets that discussed a future less than 12 months out. Those referencing the more distant future remained similarly positive to before the pandemic. After a few months, tweets that referenced the nearest future (less than one month away) returned closer to pre-pandemic levels, and tweets that referenced more than one year into the future remained similarly positive.
"Our analysis showed that the shock of the pandemic had a considerable effect on optimism toward the near and medium future," Bollen said. "Sentiment toward the farther future was left largely unchanged, still somewhat positive. This could indicate that people tend to see things getting back to normal over the long term, even if the near/medium term looks dark. This indicates a degree of societal resilience, even after a global shock."
Todd added, "We've seen this kind of resilience in other positive attitudes during the pandemic, including willingness to help others."
The group hopes to expand its research towards the assessment of behaviors or attitudes in people with varying political or generational backgrounds, and to discover insights into how people who are predisposed to look at the future more or less optimistically may differ in their concern about climate change, political polarization, and other pressing future challenges.
"As we learn about how people's attitudes toward the future influence their behavior," Isch said, "we might find ways to empower people to perform more positive actions, both for their individual goals and society at large."
"Twitter has proven to be a rich source of data that can be parsed in so many innovative ways," said Kay Connelly, associate dean for research at the Luddy School. "This kind of study uses AI and NLP techniques to provide insights that would otherwise have been impossible to discover, and it's the perfect example of how the Luddy School is using technology to gain a better grasp of the world around us."