Bring Mobile Scanner Data from TRK700 Neo to BeamNG.Drive with Cyclone 3DR and Blender

Preview:

  • Classify point clouds

  • Generate DTM

  • Create buildings and static objects

  • Texture your model

  • Adjust textures

  • Add all models into the game

  • Add trees

In the previous tutorial, I worked with the TRK Neo 500 mobile scanner data and demonstrated a lot of tools that could help you to digitize your site and extract all relevant features. If you didn't see it then go to the link. In today's tutorial, I would like to continue working with the mobile scanner data from Pegasus TRK and try to create my own rally racing map based on this data for BeamNG.Drive.


So the first step in processing Leica Mobile Scanner Data is Pegasus Office. It's very friendly user software that allows you to calculate the trajectory, combine different tracks, generate a high-resolution point cloud, georeferenced panoramic images, and the most useful feature for us - automatic classification of the point cloud. The software can determine more than 20 different layers in the point cloud and export all data separately by layer. For the digital terrain model creation we'll use the below layers:

  • Road surface

  • Curbstone

  • Natural Terrain

Leica Pegasus Office - Point Cloud Classification

If you working with a point cloud that wasn't split by different classes, for example from the terrestrial scanner, then you can run the Auto Classification tool directly in the Cyclone 3DR and select the Outdoor type of classification. The classification process is based on a machine learning algorithm that relies on GPU computation. And it can determine the surface very precisely with any type of point cloud.

For the Pegasus TRK data, select all three layers with surface point clouds and import them into Cyclone 3DR. Next, just select all of them and merge in one single layer with the Group button. Then click the Scan to Mesh button. In the setting select the medium quality, it will be enough to keep all details and get not too many triangles. But you can choose other settings, based on your requirements. This command creates a textured mesh from a cloud by segmenting in boxes. When the mesh will be created select it and change the representation mode from Textured to Smooth. You'll see that your mesh has two colors: gold and blue, It'll give you an understanding of how the normals look. A normal in 3D modeling is a depiction of the orientation of a polygon's surface. The Gold color means the outside of the mesh, the Blue color means the inside. For the BeamNG all visible parts of the model should have a golden color, or be reversed as an outside surface. To do that select the mesh, click with the right mouse button, and choose the Ungroup (Ctrl + Shift + G) in the pop-up window. After ungrouping you'll see that all parts will have different colors, select all of them and assign the Gold color to them with the right mouse button, select in the pop-up window menu Color and choose Gold. Next, select all blue parts of the mesh and reverse them with the right mouse

button, click the Reverse button in the appeared window.

Cyclone 3DR - Reverse Mesh

In Cyclone 3DR we have different options for how to improve our mesh surface, but before that I recommend you to stitch it, because after the Scan to Mesh tool we get a lot of meshes segmented by boxes, and tools with filling holes don't work correctly with it. Select all parts of the mesh and click the Stich Meshes tool, overlapping areas between two meshes will be identified according to the tolerance distance and the meshes will be connected with a clean mesh in the overlapping areas. After that, we can reduce the number of triangles to make the surface more light. Select it and click the Reduce button, this command simplifies or reduces the number of vertices of the selected mesh(es) by applying an overall reduction. There are two methods of reduction: Control the deviation and Keep the aspect. You can play with different settings to get the best result.


Use the Fill Holes tool to fill or modify the free borders of the selected mesh(es). Select the holes you want to fill. Selected holes are displayed in pink. Click preview, the triangles are built in prolongation of the free edge to fill. Try to use additional options such as the Add points inside holes and the Prevent self-intersection to get a better result. Improve the border of the road with the Smooth Border tool, this command modifies the borders of the selected mesh(es). These borders will be smoothed. If your holes have not closed contour then the tool will not help, use the Bridge tool in that case. This command is used to make bridges to separate big holes into smaller holes, which are easier to fill with curvature continuity using the command Fill Holes.

Cyclone 3DR - Fill Holes

If your surface has some unwanted triangles you can improve them with the Local Smoothing tool, this command allows you to smooth local parts of a mesh with a brush. Also, you can use Replace a Part tool, this command deletes triangles and vertices inside a zone and replaces this local area with a new smoothed mesh. If you want to add some sharp edges to your mesh then first you need to create polylines which will be used to reorganize the triangles. Select your mesh and polylines and click the Sharp Edges tools, the meshes will be locally modified and stretched to pass through the polylines. It can replace rounded edges, fillets, or chamfers.


Sometimes could be helpful just to delete redundant triangles on the border of the road that are stuck too much from the main surface, extract the external contour, apply the Offset tool on it and then use the Join 2 Contours tool, this command creates a mesh from two contours (polyline or hole border). You can adjust tangency and surface curvature. It will let you easily create the surface in that area where you don't have enough data. After all, manipulations check your mesh with the Check Integrity tool, this command detects integrity errors on a mesh and offers the option to correct most of them.


After editing the surface textures will be destroyed or corrupted, to fix this select your mesh and remove the existing textures with the Remove Texture button. Then select the mesh and point cloud and click the Texture from Cloud button, this tool uses the colors from point clouds to add a texture to the mesh. Optionally, set the Maximum distance: it defines a threshold above which the closest point of the cloud is rejected and will not be used for texturing the face.

Cyclone 3DR - Mesh textured from Point Cloud

For buildings we can use different kinds of approaches, I decided to create manually a simple shape of each building because a lot of walls are covered by trees, and creating nice triangulation from this data is not possible. For the same reason, I didn't use here Building Extractor tool, but in some other projects, we can use this tool. The main goal here is to get a very simple shape for each building to get a similar to-reality visual representation and at the same time not overload our computer. So I used Cyclone MODEL to create all buildings and fences because this software has a lot of tools for auto-fitting different elements into the point cloud. I used below tools:

  • Fit Patch

  • Fit Cylinder

  • Draw Polyline

  • Convert polyline to patch

  • Extrude patch

  • Insert Box


And from the segmented point cloud layers I took the below layers:

  • Buildings

  • Hardscape

  • Retaining wall

  • Wall


When all elements are done we should transfer them to Cyclone 3DR via .dxf format. While importing the 3D model to Cyclone 3DR I choose the Force Unit option to keep the right scale. You can create these sample models in any other software you liked and copy them into Cyclone 3DR, it supports many popular formats like .obj .fbx .dwg .ifc .stl .glb .iges and others. Next, we need to check that all elements have the right normals orientation. Ungroup it and colorize it to Gold color then reverse all blue elements in the same workflow as with the road surface.

Cyclone 3DR - Buildings textured from panoramic images

The next step is to colorize all buildings to real color, for that we can use data from Pegasus TRK exported in E57 format. This format allows keeping panoramic images so when we bring it to the Cyclone 3DR we can disable the option for importing cloud and leave only panoramic images. To colorize the mesh select it and some images, either several spherical images or several perspective images, then launch the Smart Texture tool. The mesh will be textured based on the closest selected images, and sometimes side walls could be colorized from the wrong panorama. To fix this select the mesh and run the Adjust Texture tool, choose the Remove current texture mode, and fence the triangles which should be modified, this tool will erase the texture information from them. Next, click the Explode by Texture tool to split the mesh on different surfaces based on texture information. Now select the required part of the mesh and panoramic images which you want to use for texturing it and apply the Smart Texture again. Repeat this workflow again if you'll be not satisfied with the result. Also optionally you can select other surface objects and use them as obstacles. They will filter images and thus the corresponding images won't be applied locally to the mesh.


Before exporting all models to other software I recommend you to set the coordinate system somewhere in the middle of your scene, because not all software can work with big coordinates. To do that just run the Define UCS tool, in the appeared window click plus symbol, and specify the place where you want to move the origin of your coordinate system. Then export all models to .glb format, keep the option Export in current UCS and leave meters as units of measurement. Import that model to the Blender with the standard option File/Import/glTF 2.0 (glb/gltf). After import check that all normals have the right direction, go to the Overlays menu and select the Face Orientation option. You will see two colors, blue will be visible in the game and red will be hidden. To change the orientation select the object and switch to Edit mode, select faces that should be reversed and click Alt + N on the keyboard, in the appeared window click the Flip button.


In Blender, we have options to adjust the textures of our objects, for that select the mesh and switch mode to Texture Paint. Use the Clone tool to copy the colors from the specified area to the active location. Set the reference area with Ctrl+LMB and next just move your left mouse button over the area that should be modified. Change the radius of the cursor and Strength parameter if it'll be required. You can repeat the same workflow for buildings and other elements. When your model will be ready export it to DAE format but create a separate folder and save all files there. Choose the Global Orientation option and leave Z as a default Up axis.

Blender - Before and after Texture Adjustments

Next, let's move to BeamNG.Drive. You can buy and install it via the Steam platform. After installation go to the C:\Program Files (x86)\Steam\steamapps\common\BeamNG.drive\content\levels folder, here you can see all available levels by default. We can choose any of them to start building our map, it's easier than creating the level from scratch because these levels already include some basic elements like trees, poles, and other static objects. You can try any level you want. Open the archive file, go to Art/Shapes folder, create here new empty folder, and put there every DAE model that you exported from Blender. Run the game and select the same level in which you put the DAE models, when the level will be uploaded click the F11 button to switch to World Editor mode. Click the Window / Asset Browser tool. In the appeared window you'll see that percentage of loading assets, just wait until it will be fully loaded. After all, assets will become visible find the folder with your models, select it and drag and drop it somewhere into the scene. Use the standard translation and rotation tools to pace your model into the desired area. To be able to interact with the model you should apply collision on it, for that select it and in the Inspector tab choose the Visible Mesh mode for the Collision Type and enable the option prebuildCollisionData after that Save your level and restart the game. Open again the same level, click F11 to open the World Editor menu, and move the Player car on the road which you added just right now. Click F11 again to close the Wold Editor menu and now you can test your model. Use arrow keys on your keyboard for movement, the car will interact with your surface and you can simulate your virtual driving in a real environment.


Repeat the previous step for other elements. Next, you can add some repeatable objects like poles, signs, or traffic lights from the standard library assets. For adding vegetation use the Forest Editor tool, it also has preloaded models of trees which could be easily placed on your surface. Use the point cloud as a reference to place them. And when everything will be done enjoy the game.

BeamNG.Drive - Crash Scene Simulation

I choose the BeamNG Drive as an engine because it has the best realistic physics that I've ever seen and it has a lot of tools for recreating the movement of vehicles and simulating crashes. This engine could be very useful in combination with forensic software for investigating car accidents, especially if you working with reality capture data. You can use data from the mobile or terrestrial laser scanner to analyze and investigate the crash scenes in MAP360 software. Point clouds give you the most informative data about the forensic scene and this software allows you to calculate a variety of crash reconstruction formulas; speed, deceleration factor, kinetic energy, momentum, kinematics, and more. So you can use this information in BeamNG to recreate hyper-realistic crash scenes and make your reports more informative.

MAP360 - Crash Scene Investigation

For me it was also interesting can I drive a rally racing car in that environment, let's see what comes out of it.

BeamNG.Drive - Rally Racing in Real Environment

If you still have a question or you want to repeat this tutorial by yourself, but don't have a valid license then you could reach me by clicking the below button or leave a comment.