Photogrammetry and Laser Scanning in Forza Motorsport 7 – Theater Presentation 1

Just another WordPress site

Photogrammetry and Laser Scanning in Forza Motorsport 7 – Theater Presentation 1

>> All right. We’re going to get started here So, thank you guys for coming. I appreciate it Got about half an hour here and I appreciate the chance to chat a little bit about some of the work that we did on Forza Motorsport 7 So, we’re going to talk about creating some of the realistic environments for Forza Motorsport and some of the unique ways that we approach Photogrammetry and Laser Scanning as a key part of our development process So, we’ll get going. So, my name is Travis Gosnell I’m the Environment Art Director for Turn 10 Studios Turn 10 Studios is a first party studio within Microsoft Game Studios We are in Redmond Town Center So, this is a kind of a quick presentation about our process and we’ll get going If anyone is not familiar with Forza Motorsport, we’re going to go and watch a quick video here and then we’re going to jump into it >> Play it on XBOX One >> So, Forza Motorsport 7 was released in October This was a XBOX One, PC title This was the first time we launched concurrently on both the XBOX and on Windows 10 So this was a big launch for us This was our first Windows 10 game as always Forza Motorsport is 60 FPS but this time we also incorporated 4K and HDR as being some of our key rendering features And from the environment side it’s a massive game The game has 700 cars We’ve got 32 tracks So, we’re talking somewhere in the range of 180,000 art assets that are used to make up Forza Motorsport and this has built up over the years And so we’re in kind of talk a little bit about how we go about our approach to capturing all of that information and converting it over to the game and hopefully get some ideas to take away about what are unique approaches So, some of the varied environments, we have everything from realistic race tracks that are kind of the marquee tracks of the racing world We’ve got one here that is Sebring International We do wide open landscapes and cities We have our prog racetracks Ones that are a little bit more close knit, a lot of high detail, a lot of high fidelity with the race tracks And then we’ve got this one which is our Dubai track Dubai was one of our marquee tracks for Forza Motorsport 7 Each year, we try and take one of our tracks that’s we know we’re going to have our IE track And this is going to be a combination of a track that can both showcase off the game, showcase off the new features and particularly this year as we are being a launch title for the XBOX One X being able to leverage some of the power of the new system really let us push some of our workflows and our technology in ways that we hadn’t before And so what do I care about with environments? The two big things are the accuracy and authenticity Forza Motorsport prides itself on being a very realistic racer We have a huge investment in creating real world tracks, creating real world physics, creating real world cars and so that’s a huge focus for us And so when we talk about accuracy, we really want to give the player the same experience that a race car driver is going to have visiting those locations So, this is a piece of reference photography from Long Beach and this is that same track in Forza Motorsport 7 So, we’re trying to recreate as close as possible down to the direction of the light, the position of the trees It may seem like there’s flexibility with that but when it’s a racing game, those are breaking points They’re markers for the drivers

that are keeping an eye out for them So we really double down on how to get that authenticity This is an older video but one that I think showcases very well says from one of our community members, some of the differences in the real world and what we get in the game So, as you will see as a kind of drives around here you’ll see that it’s very close to a one to one match We’re paying attention not just to the road but to the whole environment around it So, we are trying to get all of the flags in the right places, all of the buildings are at the right scale, the tree lines are exactly where they’re going to be And so doing this across 32 environments is a huge undertaking and a huge task for the team On the authenticity side, we’re not just trying to capture that range, we’re also trying to capture a lot of high fidelity detail We’ve won all of those details that give the character to the unique tracks whether they’re a brand new circuit or whether they’re a track that has 100 years of history We’re really looking for workflows that we can use to capture as much character and as much detail and storytelling in these environments as possible And so we do that by really doubling down on reference So, these real world tracks are average of four miles long, 32 tracks, there’s 120 plus miles of environments in Forza Motorsport And so really doubling down on our references i key The current gen hardware, the advances that we’ve made with Forza tech with X Box One X has really allowed us to start bringing over data from the real world into the game engine with a fidelity that we haven’t been able to before And so for us, it’s a huge important piece of our investment is really doubling down on that preproduction, doubling down on how we capture it And so we do a lot of the standard reference photography, capturing PBR textures but we’re not talking about a lot today is the laser scanning and the photogrammetry And so a quick kind of breakdown So, when we start one of these tracks, the Dubai track which was our marquee track for Forza Motorsport 7, we start with a lot of our research and so we’re going out we’re finding the areas that are the inspiration, they are the iconic areas to that race track So for us when we went to Dubai, it was a lot of the modern cities, the modern architecture that’s springing up all over the country It’s the sand dunes and the desert, the nature creeping back over onto reality and it’s areas that are iconic to the motor sport world So, in the bottom left there we have the Hafeet pass So, this was a really key area for us to capture because it’s on the list of greatest driving roads in the world And we want to be able to deliver those aspirational experiences to the player And so we start out by travelling to the locations We do reference trips out to all of the tracks that we create for the game Here, we spent about eight days in the UAE and so did a road trip across the entire country We went with the city supervisor, a technical artist, a concept artist and then a couple members of the track team So, we’re really trying to get representation out across the broad tune when we go to capture data We’re doing a lot of standard things We’re doing Reference Photography both from an in game type of perspective but we’re also doing a lot of high risk panoramas that we’re going to use to create map paintings We’re doing a lot of lighting reference And then we’re also doing a lot of our capture for our PBR system So a lot of Color Correct Reference, Color Correct Photography And you can see a lot of these images will have the Macbeth chart in the lower right hand corner and so we’re using that to be able to get the albedo values for all of the assets that we’re going to be building in the game And on the left side, we’re doing some captures using polarizing filters to separate off the specular combination So, a lot of the imagery is key to what we do but images aren’t enough We have found no matter how many of these that we do you can send a whole team and spend a week there get 10,000 images and there’s always going to be those sides that you don’t get coverage is always going to be a problem You’re going to get back and someone’s going to ask for the photo of the backside of the building or did anyone get a picture of what’s on the left side of the road here And so finding other ways of capturing that data is important to us Photos also miss a lot of key information, sense of scale was a problem You don’t always have a core checker or a human in every photo that you take so backwards engineering exactly what the scale of an object was That’s a real problem for us and it’s hard to get measurable data We can check exposure, we can get albedo values but getting measurable data out of photographs is just has its limits And so to address that problem we do a lot of other things In the past, we have done everything from wearing GPS backpacks So for the early days of Forza, this was one of the key ways of capturing the information for the tracks Here, we would put on a backpack with a GPS transponder We would walk the track left side, right side, inside, outside and so put about 12 miles in walking all around the race tracks for the day It’s very very accurate from a X and Y position but it’s not very accurate on the vertical And so we needed more ways of doing that We would go out with levels and actually

measure the angle of every turn on a race track We would get our tape measures and sit down and measured the width of every rumble strip, record all of this information and bring it back But we had a data set that was a lot of numbers on a sheet, a lot of splines that we could bring back into max and while it gave us the information that we needed that wasn’t the most useful So, starting on Forza Motorsport 5 we moved over to laser scanning and there’s a lot of small text on the screen here But the big takeaways are that we needed a better way of capturing that information We needed a better way of getting better data into our system to drive more accurate racetracks coming out the other side And so we want to do kind of an overview of our process here and how that informs the production So, the basic overview is that we do Scan Capture, Processing/Registration, a lot of View and Cleanup and then use that to feed into our production pipelines So, when we do our Scan Capture we’re going out on location we’ve got a third party group that works with us for most of the time We use two or three scanners These are like a Laser Scan systems We’re going out setting these up They’re going to spin around shootout millions of little laser beams that will bounce back And what you end up with at the end of that is something that looks sort of like this So, this is a point cloud that has been processed coming back out of that capture So, this area that we’re in right here is one of the metro stations in Dubai, and it’s one of the really nice pieces of modern architecture that was going to feature heavily in our race track So this is one of the areas that we’ve focused on As you can see, if you can see all of the little dots down here that is each individual points that are captured by the Laser Scanner, we’re capturing with about six millimeter accuracy, which is about a quarter of an inch give or take And so this is giving us very, very accurate measurements It’s real world data that we can go in afterwards, we can grab two points, measure the distance between them You don’t have to depend on the number someone wrote down from a tape measure and it’s just much, much more flexible Each scan that we do has to be registered together to create a cohesive environment So, on each one of these red dots is one of the locations that we set up our scanners So, more or less we get about 200 meters You can adjust for scale and time constraints as needed Well, we’ll spend about three or four hours at any given location like this and gather up about eight to 10 scans And then, these scans will be stitched together, referenced together to create a cohesive environment So, once all the scans go together, you get something that looks like this which is a large scale cohesive environment Once we have this data, we can do a whole lot with it This is a short video, let’s see if it’ll play in Microsoft ReCap or Autodesk ReCap So, this is a program from Autodesk for viewing and working with point clouds And so, unlike the photography which gives a single still image, this is really allowing us to come back and anyone on the team can run around the environment They can explore the environment in a really a way that we can otherwise As we have started using this data, we’ve started using it more and more across the team also Once you have data that is at this level you can also start passing it out to the other teams The cinematic team can start doing some of their previous for their camera shots We don’t have to wait for levels to be created, for grey boxes to be created You can immediately put it into the hands of the team Have concept artist start painting over it, have cinematics guys start working with it And then on the art team, we start using it as the basis for our production So, for the Dubai race track, you can kind of see all the different areas here that were our main points that we went in laser scan So, we had our city areas, our highway areas, the feed mountain road that was the driving road that we talked about before, the desert areas And so, from the very start we’re using this as a way of informing the production So, what we’re doing here is laying out the basics of the race track using the laser scan data before we’ve done any of the work in our grey box And so, this is giving us a way of pre-visiting what the environment is going to be at a very early stage That’s also extremely informative both for the production teams and for everybody else who will be working with it What you’re looking at here is a render from 3D Studio Max, so this is just a viewpoint render you can see the mesh that’s laid along the ground there is representative of where the race track is going to be That’s our very first collision that we’re laying down to race So, at this point we’re doing our very first work to get it playable in the game which you can already pre-visit It’s something that is both aspirational for where we’re headed and very representative of what steps need to happen to get us there We talk about the authenticity and letting players drive the locations A track like this is made up of multiple scans and each one of those may be aspirational for the area that we’re playing, but when you’re in any of those areas, it’s still very important to us that if you are driving the feed road portion of the track, you’re literally driving that road, right? Every turn it’s going to be laid out as that turn exists in real life

And the scan data, lets us take those areas that we’ve captured, look at them from a large scale and start placing them together in a way to recreate these environments The other thing that we can do from it is start breaking up that laser scan data in a way to inform the actual production So, instead of going through just 10,000 reference photos, and trying to pick out what are all of the unique assets, we can fly around the environments and actually start picking out the assets that really matter Which ones are breaking up silhouette which ones are close to the track Start creating our asset manifests in a way that will inform the production and really speed up the production for us At that point we can start isolating the scan data for each one of the assets and that will go off into its own Maxwell that will then go off to a 3D artist to build, but at that point, they have reference that is already captured with real world scale They’ve got the core correct photography to go along with it They’ve got all the physically based material reference and so, they’ve got a package that’s ready to go for the creation of an asset they can really streamline production in a big way That same Metro station that we were talking about, this is it with the scan data converted over into 3D Studio Max And you can see that it’s then used to model directly on top of So, a modular can go in and instead of trying to put two planes up look at where the images are and try and figure out the exact scale of this thing, they can fly around in 3D and use it as really a organic piece of reference to inform their work And then, from there it goes over into our game This is a quick image of fuel which is our proprietary engine And this is where we apply all the PBR materials to it So, it’s a very easy translation from the initial scan data across into our game engine From there we can take an overview of that scan and take over the different areas This is one of the large images for the Hafeet pass So, you can see again, just like we collected all the scans for the first area, this was a set of about 30 scans that were collected going up and down this road that have been stitched together They give us that 3D representation And so, when we are making objects like the metro station, it’s relatively easy to go through, sign it out to an artist’s we’re doing a lot of kind of meat, potatoes hard surface modeling When we get to all the organics around the terrain, that’s where it’s a new challenge for us and we can use a lot of the traditional terrain technologies, right? We can use our rolling machines, we can use our views, we can use our Houdini train engines, we’re also trying to be true to the authenticity and the accuracy of these locations And so, for us, really doubling down on the photogrammetry was a big piece of that And so, we will start with a simple gray box and that’s just blocked out by hand from the laser scan data And this is informing us where we’re going to start breaking down that photogrammetry And from there, we’re going to start replacing that with our final in-game assets that’s generated off of the photography that we took on site And so, if anyone is not familiar with photogrammetry the quick overview is that, photogrammetry is made up of taking a whole lots of photos of a real world object then using software We use a combination of AGIS photo scan and reality capture to calculate all of the trajectories off of those images and figure out how you would recreate a 3D mesh off of those So, while we were on site, we take an absolute ton of photos Everything from all of the rocks to the large scale mountains And when we come back to the studio and process that now we’re going to get a lot of the data that we’re going to use to repopulate the world When we’re breaking it out off of the environment, we sort of break it up into a couple of main areas For us, it is kind of broken off into large scale environments set pieces, so these are large scale elements of photogrammetry, we’re breaking it off into the surface materials so ground materials, ones that are going to inform the standard terrain piece of it And then, [inaudible] materials, simple rocks, scree piles, things that can be scattered around the environment to do a lot of the set dressing From there we’re going to take over the images of each one of those assets The work flow is relatively straightforward for the artists at this point It’s a lot of reimportation, retopology We’re using a combination of either retopologizing and 3D Studio Max for relatively simple assets We’re taking it into Zebrush for other assets And so, this is where we’ll go in, we’ll chop out the elements that we captured in a lot of cases, there’s elements like the bottom this is going to be missing The artists will go in, a lot of kind of bespoke modeling work to fill in those elements but then, what they’re left with at the end is a very reasonable asset that we can start making these sets to use around the environment And it’s letting us capture a lot of fidelity in ways that we just couldn’t before So, just like the laser scan allows us to get that accuracy for our environments, the photogrammetry is a key piece for how we get the fidelity into all these pieces So, we are going through, we’re capturing everything from road edges, large scale environment pieces Here’s some examples of some of the terrain materials are feeding into our terrain system We use a combination of substance either

to regenerate substance versions of the materials, or as a processing stage to capture the photogrammetry materials, run it through substance and then create tileable PBR materials out of that From there all, that gets brought back into our engine And so, this is one of the materials that was brought from our photogrammetry and created into a final PBR engine material that is rendered inside of our engine And so, this is an arch reactor is hugely inspiring because it lets me go around and choose the assets that were captured at the reference trap and really pick out the ones that are both reusable and defined the overall character of the environment Was also the ones that are very bespoke and had a lot of the visual interesting can give character to individual areas And what we’re left with at the end, is something that is really representative of that initial photography for it So, we’ve got some of the large scale photogrammetry that we’re using to create that area and then, at the bottom you’ve got the gift there of what it looks like inside the final fours of seven And we’ve got some of the final images from the track So, this is the game running on the Xbox One X, Photogrammetry on the Hafeet pass and that’s what I’ve got So, thank you all for watching, appreciate it and hopefully you’ve got some ideas for how you can leverage some of those elements to kind of feed into your production and really inform how to streamline your production, and get real world values Thank you very much. Appreciate it