Forecasting financial markets with signals from social media

Episode 1 September 03, 2025 00:12:03
Forecasting financial markets with signals from social media
FTSE Russell Convenes
Forecasting financial markets with signals from social media

Sep 03 2025 | 00:12:03

/

Show Notes

Media reports, social media posts, press releases, CEO conference calls and many other channels deliver unstructured data to the market. MarketPsych uses AI and its large language models to analyse emotions and sentiment and turn these into signals and forecasts. Listen to learn more about this exciting and powerful growing field. 

 

Chapters

View Full Transcript

Episode Transcript

Richard Peterson: So when markets become emotionally overreactive, whether it's irrational exuberance or excessive fear and panic, in both cases you tend to have a subsequent reversal in the price. So if people are too exuberant about something too excited, they tend to be disappointed later as markets reverse and vice versa. On the downside, there's capitulation and then prices rally and our data is used for that, for detecting that. David Sol: Welcome to Footsie Russell convenes. I'm David Sol and in this session we're going to talk about how moods and emotions affect markets. And for that I'm talking to Dr Richard Peterson CEO of MarketPsych. Welcome back Richard. Richard Peterson: Thank you David. It's great to be here. David Sol: Now let's take a step back. What is MarketPsych. What do you do. What does your company actually do. Richard Peterson: So MarketPsych does media analytics where we look at news social media transcripts and we analyze them for the tone of the language that's used. So sentiment is an easy example. How positive or negative is it. But also emotionality. Things like how much optimism does a CEO like Elon Musk express in an earnings call? Transcripts? Or how much disapproval do analysts express in an earnings call transcripts. You know, towards the CEO. So these nuances we capture and we turn them into data. So not only sentiments and emotions, but now we also do topics and nuances like events. So we're capturing everything that we believe could be impactful on financial markets in a structured format, whether data feeds or in textual structure. David Sol: So when you talk about social media, you're referring to X, you're referring to Reddit and LinkedIn. What type of sources are we talking about? Richard Peterson: You know, message boards globally, places where people discuss stocks and currencies and other events. David Sol: And you also look at things. What governments are publishing central banks is also important sources for your analysis. Richard Peterson: Exactly, exactly. So we look at websites for central banks and then also things like their X accounts. So when some central banks like to put out, you know, press releases essentially on X, so they're announcing their next policy statement on social media. David Sol: So how does MarketPsych work? You you grab the words from those websites and then they transform it into a signal. It's in a score. Or how does it actually work in practice. Richard Peterson: So we have a number of ways of dealing with that type of data. The big picture is we're converting unstructured data, which is text or information flow, into structured content that can be used in statistical modeling or forecasting. So the standard way of looking at this is taking sentiment scores from language. So as people discuss, say, a given central bank, the Swedish central bank, they'll describe that. Oh, we think they're going to raise interest rates or lower interest rates. And we have algorithms that look at that language and calculate the median expectation of what the Swedish central bank is likely to do. And then we publish out an interest rate forecast index or score for the Swedish krona. So that's one example. But we do that on every currency and every central bank in the world and every major commodity in the world, every major company. So we're publishing all sorts of indexes or scores around lots of different themes and topics, trying to create as much structured data or information that can be used, and statistical modeling as possible. David Sol: So who is using your data? Who are your clients? Richard Peterson: The biggest hedge funds in the world are our clients. We can't name most of our clients. One that we can say is, For example, Yahoo Finance uses our software engine, actually for named entity recognition, meaning to identify wouldn't an asset like gold or, stock futures or Apple stock is discussed in a news article. They use our software to identify that target and then link it to a stock price page. So we do that for that is also in the iPhone finance app is in part powered by our software. David Sol: And can you talk a bit about predictability. What is that you actually forecasting and how accurate is it. Richard Peterson: I mentioned the forecasting side because it is, I think, easy to understand. And we know there's a whole industry of quantitative people in finance whose goal is to predict markets with data. And we we do find that there is predictive power. We actually launched a predictive feed based on only media data on January 1st of 2020, and we co-branded it with the StarMine Group at LSEG. It's called the StarMine Markets like media sentiment model, and that's been doing well. It ranks every, US stock from 1 to 100 every day. With those closer to 100 more likely to outperform those ranked closer to one based on the media sentiment. And it has had a good performance, a Sharpe ratio of one between the the top 10% on the bottom 10%, which is how we measure the benchmark, the performance. So it has been doing well, you know, both in historical testing. But since it launched, as well as showing that there really does seem to be, significant predictive power from the media. David Sol: And we sometimes see a disconnect between the emotions of the market and kind of the emotions in words and then how stock markets perform. Richard Peterson: There's two really interesting findings from my perspective, because I'm a nerd about these things. But, one is called overreaction in the markets. So when markets become emotionally overreactive, whether it's irrational exuberance or excessive fear and panic, in both cases you tend to have a subsequent reversal in the price. So if people are too exuberant about something too excited, they tend to be disappointed later as markets reverse and vice versa. On the downside, there's capitulation and then prices rally. And our data is used for that, for detecting that. But on the other hand, there's another phenomenon called under reaction, which is exactly the opposite. And in this case, when there's say positive routine, somewhat boring information, but generally positive people don't react to it, it doesn't motivate them to buy right away. It sort of creates a trickle of buying, which creates a nice long stock price drift. And that's true both on the positive side and on the downside. And the key difference is that the emotional overreaction is when there is something dramatic, something like an award for a CEO causes a big spike in the stock price or, something on the downside, like a management scandal, suddenly the stock drops. But those often reverse in the other direction, but only if there's a lot of attention paid to it and a lot of people, again, emotionally reacting. David Sol: So of course, this year there was a lot of attention paid by the markets to tariffs, a lot of news around tariffs. So what did you see in kind of the signals in, around words that were being used in releases related to tariffs. Richard Peterson: That we've looked at a number of different sources of information. The one we like to talk about lately the most is earnings call transcripts. In the social media side, we had the leaders like Trump posting his latest plans for tariffs. So in that case, social media was the news, and driving trillions of dollars in value, both evaporating and adding value based on what he was tweeting or posting on Truth Social. On the other hand, though, companies themselves, in their earnings call transcripts were mentioning tariffs in different ways, some would say, don't worry about it, it's not a problem. Some didn't say anything at all. Others mentioned them frequently and often, those who mention them the most frequently. For example, in March we did a study companies that mentioned tariffs in their earnings call transcripts in March, more than 30 times had a stock price decline of about 6 or 7%, whereas those who didn't mention tariffs at all, which is another thousand or so companies, had an average stock price increase of about 7%. Richard Peterson: So you have a big gap between which companies are outperforming and underperforming, based on those who were talking about tariffs that might hit the bottom line and those who didn't. And this was predictive. So it was from March. We then looked at the share prices in April in May. And that's when we saw this big, almost 15% gap open up. David Sol: That's very interesting. And kind of you see how are you using those technologies to to gather insights. If we talk about AI specifically, of course, AI itself, the technologies developing large language models are developing very, very rapidly. How is that reflected in what you're doing? How are you using that? And how do you notice that these large language models have improved, compared to 1 or 2 years ago? Richard Peterson: Part of the way that we use them is named entity recognition. So a key aspect of what we do is just identifying what are people talking about. Are they talking about, this company or this commodity or this currency? And so we have to do a very good job of identifying those. But sometimes, you know, some the owner of a company might have a company named after themselves. So Henderson incorporated, it was owned by and I'm making this up maybe Mr.. You know John Doe Henderson. And so you have to understand when the article mentions Henderson, are they talking about the person Henderson or are they talking about the organization, the company? Henderson. So we the lamps now have actually become very good at that. Whereas historically we had to use a lot of rules and context, essentially human coded knowledge and how to understand that, the lens solve it for us. David Sol: And is that also the beginning of the text? How do they look when you have a scripts, an earnings calls? Because something can be announced at the beginning and maybe halfway things can change. How how is the technology dealing with different aspects, different emotions perhaps throughout the call, throughout the text? Richard Peterson: Well, what's amazing now is it can detect which speaker is which. I can even tell you based on the tone of voice. And if there's an argument between two people, you can pick out which person is saying which line. So the AI has really become quite impressive and discerning each speaker, which is great because it allows us to then tag the emotions of the CEO versus the emotions of, say, an analyst who's insulting them or asking challenging questions. And that emotionality is itself quite predictive. We've done research on CEO optimism, as I mentioned at the beginning, that is quite predictive of share price returns. Subsequently, and analysts, a disapproval of a company is also quite predictive. Obviously on the downside, later on. So we measure 13 different emotions. There's a number of nuances you can capture and I mentioning the highlights, but there are certainly really interesting findings coming out of this type of research. David Sol: So 13 different emotions okay. And one would ask me to name them okay okay. That's the lever that if you take a glimpse into the future, where are we in one year or two years from now? How will this technology have been involved, evolved, and how would you have applied it within your MarketPsych framework? Richard Peterson: Well, to go back to the predictive model that we created, that was done with Lasso regression after, a laborious process of feature generation and testing. So it took many, many months, maybe a year to come up with this model. Now where I think we're going, which is quite exciting, is the AI itself, the deep neural network these large neural networks can actually solve. They can they can handle a lot of the nuances that humans are doing and can create predictive models much more rapidly. So they're not yet at the stage where they can do the predictions. But, it's quite exciting. I think we're getting to that stage where they will be doing a lot of their own predicting, based on context. And it's going to be an interesting world when that starts to happen. David Sol: Is definitely going to be an interesting and exciting world. I'll keep a very close eye on MarketPsych. And what is so interesting, if you look at the founder of economics, the father of economics, Adam Smith, spoke about emotion, spoke about passions, and here we are now talking about passions. Thank you very much. A very interesting discussion. I'll keep an eye on markets. Richard Peterson: Thank you David. Yeah, great to be here.

Other Episodes

Episode

September 19, 2023 00:23:58
Episode Cover

Creating the best playlist for your portfolio | Andrew Ang, BlackRock

Today, investors have endless options to choose from when tailoring their portfolios, particularly when targeting specific investment factors. But how can you create the...

Listen

Episode

July 10, 2024 00:13:35
Episode Cover

The macro and the micro of market structure | Michael Brandt, Fuqua School of Business

In our first episode of FTSE Russell Convenes, Season 4, we welcome Michael Brandt, Professor of Finance, Duke University at The Fuqua School of...

Listen

Episode

July 17, 2024 00:09:18
Episode Cover

Vanguard’s Approach to the >$11 trillion US ETF Market | Daniel Reyes, Vanguard

In our second episode of FTSE Russell Convenes, we are joined by Daniel Reyes, Global Head of Product at Vanguard, whose career with the...

Listen