Why Firefighters Train in VR Before Real Fires

What We Need to Grow, Episode 057: Your Brain Gets Tricked. A Founder Talk conversation with Evan Sitler-Bates, co-founder and CEO of XpertVR. Cleaned for readability; the words are the speakers' own.

Caleb Pedosiuk: All right, here we go, Evan. Let's do this. So you co-founded XpertVR roughly six years ago to build hyper-realistic VR training simulations for first responders. How wild is that? Firefighters and search and rescue teams train together in shared virtual environments for high-pressure fire ground scenarios, including a partnership with an Ontario college. The mission to get people learning from doing, not from lectures. Let's hear about it. What is going on? What are you building? Tell us all about it.

Evan Sitler-Bates: Caleb, thank you for having me on. You explained it well there. We create virtual reality simulations for firefighters, working with multiple Ontario colleges, with the Ontario Fire Marshal. So very heavy within Ontario here, but we're also spread out across North America, into South America a little bit, and starting to go into Europe.

Basically we put people into virtual reality headsets, as well as onto laptops if they don't want to go into VR, and in that experience we plop them down into basically any virtual scenario you can imagine. It looks super realistic. All the physics interact realistically, so they can practice how to handle all kinds of different fires, hazards, how to rescue people. It's not the physical side, obviously, but it's the strategic, the communication, the resource management side of things of how do you handle that scenario when it happens in the real world.

Caleb: So how did you get into this?

Evan: It's a long journey. Back in 2016 I put on a VR headset for the first time and basically instantly started a company around VR. Back then we were capturing 360 videos. We did a lot of marketing, we did some research studies, and leading up to 2019 we were doing quite a few research studies with different academic institutions across Ontario. Consumer behaviour research studies, or looking at different environments in the real world and how things might affect those environments, like pollution in the ocean and things along those lines.

And then COVID happened in 2020 and we were no longer able to run those research studies, because people had to come in person to meet with the researcher and go through it. But a lot of those academic institutions had students who were now trapped at home and couldn't come in to get that physical in-person training. So they asked, how can we take what we're doing in the research and bring it over to what those students need? That's when we started working with pre-service fire colleges, as well as police foundations and the trades, all kinds of different things, back in 2020. We built out all these simulations so that students could learn at home and still get that in-person type of training.

Caleb: So along that trajectory, parallel, is this revolution in technology with AI, specifically with generative AI. How realistic Will Smith eating that plate of spaghetti was a few years ago, compared to, if I was in VR having dinner with Will Smith and we both ordered spaghetti, what that would have looked like if I was truly training on teaching him some manners back then. All right, here, you're a manners etiquette instructor, here is someone in front of you. How real that would have felt compared to now would be night and day. So how are you able to leverage some of the Higgsfield revolution that is here upon us?

Evan: When it comes to Higgsfield and those sorts of things, we're not really able to utilize it too much on the video side, because what we're creating has to be interactable. With Higgsfield, currently we can't live-time create that environment around people. That said, there are people who are working on that. One of the main headsets we work with, Pico, I know they're up to some really cool experiments with AI. So there is stuff coming there.

Right now what we're focused on is, for the characters and for our developers, really using AI. For our developers, they're already incredible developers, but let's 5 or 10x their skills by putting AI into the work that they're doing, so they don't have to do the smaller tasks that are just very boring to do.

But then the exciting part that people see is the characters, and now what we're looking at is the environment. So instead of just walking up to a character and they're screaming at you and then you drag them out of a building and rescue them, you can go up to them and say, oh, do you have an injury? And they'll respond back to you if they do have an injury. Oh, is there other people in this house? Is your child in this house? And they will react with the emotions and with the response that you would expect them to have in the real world. So that's adding a lot.

And then on the cutting edge of things is the environment creation. Right now people can go around and use this technology called Gaussian splatting to capture basically any environment in the real world and bring it into the virtual world, but it's not optimized for what we do. It's kind of like, hey, you can look at it and it looks cool. So with AI we're building out pipelines where you can take that and bring it into our simulation, so that firefighters can train with any environment that they have available to them that they can capture.

Caleb: Where is this in two years?

Evan: Two years, it's going to be evolving pretty quickly. The VR side, what I explained about using AI for creating the environments, having AI for the conversational side of things, that is going to continue to grow and get better. But that's what we're going to see in the VR space. We might start to see generative video games where you really don't have anything when you enter the game, but then as you make decisions and as you walk down different paths, the world generates around you. I know a few people doing experiments around that, so really exciting there.

What I think is much more exciting, looking at the cutting edge of things, is mixed reality. Right now, if you put on a Meta Quest 3 or a Pico 4 Ultra or any of the top headsets out there, it has VR, but it also allows you to see the physical world around you and layer virtual objects on top of it. And Meta just two weeks ago released their Meta VR glasses, and that is basically a pair of glasses that you put on and it can do that same layering on top of the virtual world. Snapchat has, in my opinion, an even better pair of glasses. Pico is releasing some cool stuff. All the main people, Samsung, Google, Apple, are putting out things as well. So in two years we're going to have a pair of glasses that is at a consumer scale where you can really change the world around you.

Caleb: Okay, so let's put this out there for whoever wants to build a company. I'd love to build it with them. You ready? Here's what we should do, and we'll call it a game that we want to do. It works like this. All you've got to do to sign up for the game is grant permissions to be able to view your camera roll.

What's going to happen is it's going to create this version of the game and it's going to take all the characters from face recognition in it. Your uncle, your aunt, your siblings, your kids, your grandparents. It knows, because it's geolocated, it actually has a map of the world, the places you've been, places you've looked at, where your people live. And then we're going to go into this world and we're going to take some hero's journey, but along the way we get to bring actual family. Because we've got videos, we know how they talk, we know the mannerisms. You can go back in time and space. So I could go back and say, I've got to go talk to my son when he was three, he's thirteen right now, and it'll reconstruct, because it's got the voice and it can have the character doing the thing. So you could pick any of these themes, these quests, from a traditional game, but you're going to have actual real characters go do it. That's the first game.

Here's the second game. Let's say that my kids have access, or my grandkids have access, to my camera roll. So I'm grandpa, and they're like, man, let's go back in time to when grandpa was around. So the next game is, we're going to go back to grandpa's world and we're going to relive what it was like from the day your dad was born. And they travelled that Christmas, they went down to the States. So it can take you, and you can hop through at these parts where it has the geographic information and the actual likeness of people. You could be like, hey, they stopped at that gas station, I know that because he had to take a photo of a receipt or something. And we know from spatial, from Google Reviews, what that spot was like. So you could reconstruct it. You could have people.

I just think it would be so neat. I've wondered about this before with grandparents, great-grandparents. How could you actually step into it? You see some stuff right now that's trending on Instagram where people go back to, hey, what Christmas in the nineties was like, the nostalgia. A lot of people are like, oh man, VCRs and gift guides, that kind of thing. So anyways, if anyone out there is wanting to go build some side project, let me know. Hit me up.

Evan: Both of them are possible. It's just getting it to scale and making it very high quality. But a million different ideas are coming to mind of the different pieces that are already existing. It's just about bringing those pieces together. It's a very exciting time.

Caleb: But just keep the horror genre out of it, or the glorifying violence out of it. Because the flip side of that is, this is my opinion, I'm not saying this should be rule of law, because of how powerful this tech is.

I feel like your use case is so smart. I did some search and rescue training a couple of years back, and we travelled down to Tennessee for it, and it was so extensive. Five days, four nights, immersive, you're in the woods. And on the fourth night they kept us up all night with this big scenario, this big group, probably thirty of us, lots of cadre coming from first responders and special forces. All night we were up. They intentionally pushed you in your training.

Here's the scenario. You're given this brief: okay, you guys are going to be bussed out into this disaster area where tornadoes came through, there's flooding. Now you're going to show up as first responders on the scene. And they had crisis actors. So not only have we been camping for days and just stretched, we've been up all night. We're showing up and this is the final test.

They had crisis actors. You would show up and they're done up Halloween style, missing stuff. And each one you would have to talk to and say, hey, what's going on, what is wrong? You have to find out. Has it been a head injury? You can't always visibly see it. So I got thrown up on, from this gross mixture that they had made in a bottle. And I saw the lady going for it too. And then you find her and she's just acting. So I was covered, and it stunk. But VR could do it.

Sorry, I know I'm saying a lot. I want to mention one other thing. Have you guys integrated at all, while someone's in a VR set, actual physical stuff? Would you put heaters around them without them knowing and blast heat, or would you have water going? Would you have something where they're actually feeling aspects of that?

Evan: Back in our research days, back in 2019, 2020, we helped build a lab at Brock University in St. Catharines, the Niagara area. In that lab we had a full filtration system, so we could pump in different smells and then pump them out. We could change the heat of that room or cool it down. And we had some fans if you wanted wind. We also built out a specially hand-blown wine glass, because they were doing a lot of wine research in the Niagara region. With the wine glass you could drink while in virtual reality, because with a normal wine glass you're kind of hitting the headset.

So we had a lot of those sensations coming in there. Obviously it's evolved a lot. There are vests and suits you can get to change heat and other aspects. There are even a couple of suits where, if you're training police officers or soldiers, you can feel actual bullets hit you to a certain degree. My business partner was in one and had quite a bit of pain until I turned it off. So they're pretty realistic. It's crazy. There are also gloves, all sorts of different things.

We haven't really put it into our firefighting simulations, just because of the budgetary constraints. Every piece of hardware that you add, it costs more than to supply that hardware and the software to support it. So it's something we've avoided, so that it's a nice and very simple kit for fire departments to roll out. But it is heading that way. If you watch or read Ready Player One, it's all about those suits and the treadmills that you're on and different things, so that you feel fully immersed in the world.

Caleb: Wasn't it Disney that created a unique treadmill that you could walk 360? They had some tech that their lab had come up with.

Evan: They have the most advanced one. I can't remember the name of the technology, but it's built into the floor and basically it redirects you as you walk. Apparently once you get used to it, it does feel pretty natural walking on it.

There are quite a few consumer versions, though, that are more like an actual treadmill. It has a thing around your waist so you don't fall over, and basically it's like a very short bowl. So you're standing in it, and the shoes that you're wearing, as you walk they slip back down the bowl, so that you can run or walk or crawl within it. That technology has been advancing quite a bit as well.

Caleb: If someone wanted to experience something like this, where can they do that? I think the Vision Pro Apple Stores will let you put it on, but is there anything going on where someone could go and actually have an immersive experience?

Evan: Definitely, like you said, Apple. I think it's pretty cool going to the Apple Store and trying it out. It is a very expensive headset, so I wouldn't recommend it to most people. There are a lot of other great headsets out there.

The number one place I would say people can go is their local VR arcades. Most medium to large cities now, and even some smaller cities, will have a VR arcade. It's actually been exploding over the past year. A lot of people thought that it was a market that was going to crash as VR headsets became more accessible to consumers, but the best VR arcades, like you said about the physical side, they're actually kind of like an open warehouse, so you can full-on sprint in that warehouse. And they'll have different objects you have to hold, like if you're holding a gun to fight zombies, or a bat that you can swing if you're playing baseball, and different things that are tracked in that virtual world. So I definitely recommend it. That's where I had my first VR experience, and there are quite a few of them out there now that are very realistic, and you're wearing a vest and other accessories.

Caleb: That is so wild. The ability to learn through immersive experiences is just crazy. It feels like a hockey stick curve of opportunity, because so much of learning is visual. Well, sensory, as opposed to just ingestion through the mouth, reading or trying to imagine what that situation would be like.

Evan: Definitely. There have been a couple of memory retention studies, and a lot of people debate them because there haven't been too many studies out there. So don't quote me on the exact numbers, but from what I've heard people talking about, and from some of the studies that have come out, you remember about 20% of what you learned in a classroom two weeks after you learned it. So just watching videos and watching PowerPoints, versus in VR, because you're actually interacting, you're actually building a memory. Some of these studies have shown that you're remembering up to 80% of what you learned a year after you learned it. So just a much longer time horizon, and much more that you retain.

And I can see exactly why. Obviously I'm biased in the space, but when you're wrapped into that environment, your brain gets tricked into thinking, I am actually in this environment, I'm actually doing these actions. So you remember that memory.

Caleb: I don't know if this is too controversial to talk about, but I feel like I've heard, I think there's some fiction, sci-fi, built around it, but I think it's based on actual technology, where they've tried to expose certain people to a really traumatic situation. In one instance, I think it was for people that have perpetrated a certain kind of crime, to actually give them the experience from the view of the victim, to get a sense of what that was like. But it's made me almost wonder if there's a way to have proponents of war, that might be greenlighting things, actually experience what that might be like on the front lines for an extended period of time.

Evan: That's a great idea. I haven't seen the one you're talking about, but I have seen a ton of simulations around phobias, if you don't like heights or spiders or sharks or what have you, and getting exposed to those. There's a lot of research behind how that changes people's perceptions and how they overcome those phobias.

I also have seen quite a few police departments starting to use it, where they can capture a crime scene and put a judge or a jury into VR to show them, okay, this is the crime scene, this is where we found the evidence, and walk them through their investigation. So it's definitely something that would be possible if it hasn't been done before. But I could definitely see it already having been done as well. Very interesting idea.

Caleb: So what do you guys need most to grow right now? It seems like you have product-market fit, where you're working with different fire departments. You found this niche, it's in demand, the technology is changing, it's very exciting. What do you need most?

Evan: Right now it's about us growing into new markets, and being able to show those markets, geographically-wise, the benefits of this technology. It's been a long road getting in front of fire departments. We've been talking to fire departments now for six years. And like you said, we found product-market fit, we're definitely growing, but it's in individual regions. In Ontario we've covered the whole province, working with the Ontario Fire Marshal. So at this point we're asking, where do we pick and choose the right markets to get into? Right now California, Colorado, and spreading out of Ontario within Canada are the areas that we're in.

We're looking at how we get into other areas, how we make sure we have the support on our team to support those areas as we expand. It really worked well working with the Ontario government to get this out to the whole province. Obviously they saved a lot of money compared to every fire department buying it, and it allowed us to do a larger deployment, where it's much more scalable for us. So we're looking at how we can do that at a provincial and state level as well, to show the state lawmakers, the provincial lawmakers, the policymakers, that this technology is really beneficial and isn't only going to save costs, but can save lives, and make sure that firefighters are more prepared for the jobs that they have to do.

Caleb: That's so wild. So how can people who want to learn more, if someone is like, hey, I actually know a decision maker in an industry that could benefit from this, or they are that person, what's the best path to get in touch?

Evan: Either email or LinkedIn. My email's just evan@xpertvr.com, and there's no E on the front of the X, so xpertvr.com. Or I'm very active on LinkedIn, I've got over 12,000 connections, most of them firefighters there. So I'm always posting basically every day about different things we're up to, new updates that are coming.

I think a big piece of that too is, it's great to connect with me, and if you're interested and want to buy it, amazing. But even if you're just like, hey, I would love to see this type of training, or I saw this video and you could improve this, we're taking feedback on the daily. We're always building out new versions of our simulation and improving upon it. So any feedback that we get as we're continuing to develop is always appreciated as well.

Caleb: Very cool. Well, Evan, thank you so much. Appreciate it.

Evan: Yeah, Caleb, thank you for having me on.