PODCAST: Just how good is AI at predicting the future?

James Cook: Making a forecast is an imperfect science, the weather forecast calls for sunny skies, but you get rain instead. Whatever it is you're predicting, you know to take the output of a prediction model with a grain of salt. Bring that umbrella with you just in case. But Recent strides in technology have helped to make many prediction models a little bit less imperfect. It turns out. Computers are really good at making big calculations.

Ryan Severino: It's just amazing to me what the computer can do that I simply can't. I just don't have enough time in infinite lifetimes to be able to do the massive amount of computing power that we can do by utilizing the most advanced technology that literally exists in the world today.

James Cook: That's Ryan Severino.

Ryan is using AI to forecast commercial property rents at BGO where he is chief economist. On this episode of trends and insights, Ryan is going to join our chief economist here at JLL David Rea and our global forecasting director, Alberto Lopez. To discuss the possibilities and challenges that come with using machine learning to forecast rents.

James Cook: This is Trends & Insights: The Future of Commercial Real Estate. My name is James Cook, and I am a researcher for JLL.

James Cook: So, Ryan, you're joining us from the New York City area. David, you're in London, and Alberto, you're in Madrid, and I'm here in the Midwest in the Indianapolis, Indiana area, so we're covering a good portion of the world today. This is awesome. We're talking about forecasting and both traditional methods of forecasting and the newer cutting-edge stuff. I really want to start with the basics and do a very simple question, and Ryan, you're our guest, so I'm going to start with you. What's the traditional way that people forecast rents and remember we're going to need that in plain English, please.

Ryan Severino: I will tell you as somebody who has done this longer than I usually admit in polite company, traditionally it was always done in a very sort of textbook econometric sense. That basically the fundamental idea was as an economist, as an econometrician, my job was to identify the variables that were the most impactful for predicting a variable like rent.

And then build a model that would help us actually predict said variables. And a lot of the onus was really on people like myself and David and Alberto to go out and to really understand a market or an economy, a micro economy, and then use that knowledge to build a model that was a pretty good approximation of how we understand this economy to work, so that we could actually make some predictions about where we thought the world was going.

James Cook: David, let me ask you this, how do you explain the forecast to a client?

David Rea: Hello, I'm David Rea. I'm Global Director of Macro Research and Chief Economist EMEA for JLL.

David Rhea: Well, the first thing we usually caveat with is all forecasts are wrong, but some are useful. Then it's getting into saying, this is our perspective on the outlook and there are innumerable possibilities of where it might go. So really, it's not about the numbers. It's about the narrative. Can you tell a sensible, believable story based on what are the drivers of that model. And if we're saying, it's GDP growth, it's employment in office using industries, that intuitively makes sense to the client. If we're saying actually, it's the second derivative of GDP growth plus the second derivative of the change in unemployment and that gives the best model, you know what you're confused before you even get to the end of the sentence.

James Cook: Okay, so let's turn to technology. Let me back up a little bit. Alberto, I’ll ask you this. How have you seen the technology that you use to forecast change in the time that you've been doing it?

Alberto Lopez: Hello, everyone. My name is Alberto Lopez. I'm the Global Forecasting Director at JLL

I think it has changed more in terms of the methodology, how we actually approach data, how to improve the model generation, rather than using different techniques. Specifically, to forecast trends of rental growth, right? You will talk about other activities in the real estate sector, yes. But we haven't seen much change in the way that the forecast is done in the last 15, 20 years. I know everybody's talking about artificial intelligence and especially to forecast rents, machine learning. The issue with machine learning is that you need a huge data set, and you don't have that to forecast rents, right? We have a very limited history of rents.

James Cook: Ryan, I know that you're doing some work with AI and machine learning which sounds pretty cutting edge to me. Tell me a little bit more about what you're doing.

Ryan Severino: So, to Alberto's point, we are just using massive amounts of data. And potential models so large, it's really difficult to wrap your mind around. Our current iteration of the model uses roughly 32,000 variables. I’m going to get the number wrong, but somewhere in the hundreds of billions of data points. And the number of potential models that we're testing is so large that I could tell it to you, but I'd have to do it in exponential notation, and it would be a meaningless number. The one thing I can say with certainty is that the number of potential models we're testing is larger than there are miles in the observable universe. So, it is a serious mathematical undertaking that wasn't even possible just three, four or five years ago.

James Cook: Just so I understand, you're just testing, it's not like you're getting more data than a traditional forecaster would get. It's that you're doing more permutations.

Ryan Severino: Right? So, here's the way I think about it, we have a variable that we're really focused on, which is rents in this case. We have, call it 40 something years of data, probably on a quarterly time series. So, we're using a fairly limited data set for what we're trying to predict. But what we found is that if you actually use massive amounts of potential explanatory variables, then you can actually do a much better job of predicting rents than you can using traditional econometric methods. And I say that as somebody who, I'm not a data scientist. I don't pretend to be one.

But having been on the inside of this now for the last year, it's just amazing to me what the computer can do that, that I simply can't I just don't have enough time in infinite lifetimes to be able to do the massive amount of computing power that we can do by utilizing the most advanced technology that literally exists in the world today.

David Rea: Hello, I'm David Rea. I'm Global Director of Macro Research and Chief Economist EMEA for JLL.

Ryan, can I ask a follow up question? Is that still looking at any sort of linear regression, or is it more panel data based? And is that then dealing with the more qualitative variables that might be more specific to a building rather than a rental level within a city?

Ryan Severino: Right now, we're doing it at the metropolitan area level and the sub market level. The variables are everything from the traditional economic demographic variables that you might imagine. Just some cutting edge things like location data that we’ve gotten from providers to what I will call unlabeled unstructured data.

Because what we'll do is, we're actually running multiple models because models are designed to do something, and they do that thing very well, but they don't do other things well at all. And so, we're actually using, multiple versions of linear regression-based models will use different versions of models that do variable selection and reduction. And then we do completely unsupervised machine learning where we just give the computer all of these variables and data points and say, “Hey, see what you can figure out in a way that we haven't been able to do.” Similar to what happened when at Google and DeepMind, where they created AlphaGo. And instead of training AlphaGo based on historical games of Go, they just gave the computer the rules and said, “teach this to yourself,” and now it dominates people. We're doing something similar in that sense. We're just giving the computer a bunch of structured and unstructured data sets and saying, see what you can figure out with this, and it's been, it's been pretty remarkable to see.

James Cook: So, Ryan, my understanding with AI is that because of the way it's trained, you can't look under the hood and see and understand how it works. You know, it kind of like writes itself. Does that translate to what you're doing? Like, if you come up with a successful model, is it a black box? Can you know what is driving it?

Ryan Severino: No, what's great about it is we can pinpoint the specific variables that the model retains, we can group them into categories. Because one of the things that is a challenge with this to your point is explainability, right? Nobody wants to know that you're investing millions of their dollars or billions of their dollars based on a black box.

And so, we always have to be able to go back to them and explain, not so much with the discrete mathematics that's going on, but everybody wants to know, what is the model looking at? And so, we can basically distill it down into categories that people can understand. Economic variables, demographic variables, things like that, without getting into the weeds of we're doing multiple layers of matrix multiplication to come up with this equation at the end that tells us what are the most relevant variables?

James Cook: Alberto, I'm going to start this with you. So, with your non-AI forecasting, what is the human element of it? It's going through and assessing the different models, right?

Alberto Lopez: It is not that much, to be honest, because what we try to do is to make sense of the trends that come out of the models rather than the numbers, because as David said, you're never going to get the numbers right.

You have to know what is happening in the market. And we know that by our local teams, that they know the markets very well. We show them different models, which we believe they may represent what is actually happening in the market or what is going to happen in the market. And we discuss with them the drivers on the forecast that they produce. And we come up with, let's say the best possible model to explain what is going to happen in the next five years to 10 years, and that's actually what we do it. By using linear regression models is very transparent because you can analyze the drivers, but not only the drivers, but the impact that each driver have in the market, and you can create a scenarios for those drivers. You can create a scenario, let's say for a recession, if GDP growth is one of your drivers of employment and employment, so you can create different scenarios. So, it's not only about the number itself that you produce, but what you can do with the tool.

James Cook: Ryan, would you give a similar answer with your new AI forecasting?

Ryan Severino: The one wrinkle to that, I would say, is one of the things that I'm responsible for my team is responsible for, is really making sure that there's the there, there. Because one of the things that you always have to be careful of with models like this is that it's not just a spurious relationship in the data that it finds.

And I know this because whenever I'm teaching econometrics one of the things that I always do in the beginning of class, maybe the first lecture, is I'll show the students a relationship that looks incredibly sound, right? Won't tell them what the variables are, but I'll show them the relationship, I'll show them all the regression output and everything, and they'll say, “Oh, this looks like a sound, meaningful relationship.”

And then I spring the gotcha on them at the end, which is, they're basically telling me that they think they can predict Manhattan rents based on cumulative rainfall totals, and I don't know anybody that wants to invest in New York based on cumulative rainfall totals. And so, what we do is we will actually look at the output to make sure that there is some real-world economic phenomenon there. But the thing that is great about just using these massive amounts of data is that what we find with the models is that sometimes the models will identify variables that again, in infinite lifetimes, I would never select myself, right?

It seems like there's no meaningful real world economic relationship between this independent variable that the model is highlighted and the dependent variable that I'm trying to predict. But then once you start to think about it and you make connections, I apologize. I'm going to sound like an economist for a minute but effectively what you get is a variable that's effectively known as an instrumental variable, which means it's standing in for another variable. And what it's usually standing in for is something that we would like to observe but is just not being measured in the world. And what's great about that is it helps us reduce a phenomenon that's known as omission variable bias, which means you're leaving something out of the model.

And in traditional econometrics, we tend to leave variables out of the model because we just don't have them. We can't quantify them. But what happens is, once you start looking at Again, tens of thousands of variables and combinations of those variables, it starts to find proxies for those variables that are not actually being quantified.

And that's one of the things that helps us make better rent forecasts than we could with a more limited set independent variable.

James Cook: Alberto, when you look at variables, do you have sort of a standard group of inputs that you're looking at in every market or does it kind of vary, like tell me a little bit more about that. Because obviously it sounds to me like Ryan is taking the approach of, we're going to throw in everything in the kitchen sink because a machine is going to do all the heavy lifting. Whereas you're curating this a little bit more. Do you find that you have the same inputs for every forecast model, for every city?

Alberto Lopez: We actually, we throw everything also in the pot. Because we have this code in Python that is going to generate models, right? And we allow the code to let us know which models may be more relevant to forecast specific market. We would love to have one single structure that works with all markets, but as Ryan said, that's not the case, right?

But we find that there are certain variables, certain drivers that they show up in many markets, right? For instance, in the U.S. corporate profits is one of the drivers that it shows up in many models. But there are other drivers. There are more market specific. But, again, as Ryan does, is we just put everything in there. We don't assume that this driver is not going to be relevant in this market. We let the machine to decide for us, with certain filters.

And if I may say our approach using artificial intelligence in forecasting is slightly different to the one that Ryan mentioned, because it's not that we only use linear regression, but we have used machine learning also to forecast, but to do a more qualitative forecast for rental growth, which is using other approaches, which like the natural vacancy rate. Basically, with the natural vacancy rate, you're familiar with it, if for a specific market, the vacancy rate is above the natural vacancy rate, you expect negative rental growth, and the other way around, right? And traditionally, that has been calculated using linear regression. We have used artificial intelligence, in this case support vector machine, to identify what is the natural vacancy rate. Using linear regression, it doesn't work. You find that in many cases, it's like flipping a coin, right? By using machine learning we increase that percent the possibility of getting right if he's going to be positive or negative rental growth by 90%.

In the main European markets, it works very well. And we use support vector machine, which is a very simple artificial intelligence tool that we can communicate to clients. And that's what they think. That's why we try to use the simple approaches when we use artificial intelligence, because it's easy to communicate to clients.

James Cook: Ryan, when you're talking with your stakeholders about the efficacy of this new AI forecasting, how do you explain? Do you have metrics around improved accuracy?

Ryan Severino: One of the other things that we do, which is really impactful, is we will show what our models say versus consensus. And so, for consensus, we'll use the forecast from, say, third party data providers. Because what we're ideally looking for is we're looking for opportunities where we break from consensus.

James Cook: So your model might say, okay, we think rents are going to grow by X percentage in, I don't know, industrial in this market in Asia or something like that. And everybody else thinks they're not going to grow. So, now's the time to move.

Ryan Severino: Here's a perfect example of that. About four years ago, the model was really bullish on industrial in Las Vegas, and nobody was remotely excited about it then. And we had conviction in the model, and we were willing to invest in it. We knew it was a fairly finite window because we could see, the way I explain it when I'm talking to clients, it's almost buy, hold, sell recommendations for stocks.

You might have a really strong buy recommendation today, but once that thesis plays out, now it's not a buy recommendation anymore. It doesn't mean you were wrong making that investment. It just means the window of opportune investment closes. And then it becomes a whole rating or sell rating at some point.

So, we saw an opportunity to invest in Las Vegas and industrial. It is very much an off the radar type of market. And yet the model had strong conviction enough in it that we were willing to make a sort of a calculated bet in that case. I think that deal, the last time I checked, the rents are 45 percent underwriting. So that is a quintessential example of how we are using this technology.

James Cook: So, you're looking for the edge. So, David, let me ask you this. When you're communicating our JLL forecast to clients, are you looking to be contrarian or are you, it sounds to me like you want to sort of, Represent the best overall consensus of what expectations are.

David Rea: It is a challenge that we face. We are trying to manage, let's just say one part of the business that wants to predict that there's going to be really strong rental growth and another part of the business that really wants to predict that there's going to be very low rental growth. So, we're trying to take a middle ground that we feel is justifiable that we feel can be model driven, as well as incorporating the local market intelligence from both parts of the business.

Yeah, we are trying to walk that tight rope, which can be quite challenging and manage those different views. Ultimately, my take is if we can disappoint everybody equally, we're doing a good job.

James Cook: Yeah. As a researcher if anybody is just always really happy with what you're saying, you might have a problem,

David Rea: Definitely.

James Cook: So, I'm going to end, and Alberto, I'm going to start with you. We're going to get meta here. I'm going to ask you to forecast about forecasting, blowing your mind a little bit right now. How you forecast, how is that going to change in the future? Or is it going to change substantially?

Alberto Lopez: It's definitely going to change, substantially. I think the application of more and more artificial intelligence is going to penetrate the forecasting sphere where we are, definitely that is going to happen, but not only that, not only the technology that we use, but also the data available, how that data is used, that definitely is going to change.

I think it's going to be a combination, as Ryan said. That they seem to use a very sophisticated models, but they also use linear regression. So, it's combining all that. The challenge is that how do you use that? Because we know all these algorithms for artificial intelligence, but the question is not to know them, but how do you apply them?

The challenge is that you need somebody with data science knowledge and real estate knowledge. Somebody who has been in the industry, in the real estate industry for many years. very much. And those kind of profiles are really difficult to find in the market. But you need teams that they combine both, right?

And that's going, that is what is going to be one of the challenges in forecasting in the future in real estate.

James Cook: David, do you think the future expectation is going to be, because people are seeing such amazing things happening with these large language models and.

These, this generative AI that creates like human like things, are people going to expect because of how technology is even more accuracy from these models?

David Rea: I think that touches on a really interesting point. The more we rely on AI technology computing power, the more this requires an understanding of the limitations of that technology and ultimately decision making is always going to be done by humans. And so, humans need to know what their biases are in, do they believe the model? Do they just take it wholesale? Are they going to go, “you know what, we're going to take this black box and put all our trust in it,” or do they find a way of understanding that actually, you know, their decision making process needs to take in some qualitative variables. It needs to take in a much greater stakeholder approach than just what the model is kicking out in terms of financial variables.

So, I think technology, AI, all the things that will go into modeling will take it on leaps and bounds. Hopefully, it'll make it faster, more efficient, more accurate. But, without that understanding of what the limitations are, what the parameters of its capabilities, I don't, it will need that as another basis before it will help to make an improvement in human decision making.

James Cook: So, Ryan, we're going to end with you. What's your forecast for the future of forecasting?

Ryan Severino: No. So, it's a little unknowable of course but certainly, It's more data points, it's more variables, it's more potential models to check, interesting cutting edge techniques in ways that we can't even imagine today just because the computing power is increasing at such a nonlinear rate.

James Cook: Thank you all so much for joining me today. This has been an absolutely fascinating conversation.

David Rea: Thanks, James.

Alberto Lopez: Thank you very much.

Ryan Severino: Thanks guys.

James Cook: If you liked this podcast, do me a favor and go into the app that you're listening to right now and give us a rating. Even better. Give us a little review, just write a sentence about one thing that you liked about the show. Of course, you need to be subscribed to trends and insights, the future of commercial real estate in that same app to get a new episode.

Every time we publish. Or you can find us on the web anytime at jll.com/podcast. We'd love to hear from you. Send us a message, a note, an idea for a new episode, whatever. Email us@trendspodcastatjll.com.

This episode of trends and insights was produced by Bianca Montes.