BLK2GO, Cyclone 3DR, and Augmented Reality (AR) with IMMERSAL and Unity 3D
Used software:
Immersal SDK v. 1_16_1
Unity 3D v. 2020.3.37f1
REGISTER 360 v. 2022.1
Cyclone 3DR v. 2022.1
Used hardware:
Preview:
BLK2GO capturing process
Colorize the BIM model based on its status in Cyclone 3DR
Create the floor surface for navigation in Cyclone 3DR
Create an as-built model in CloudWorx for AutoCAD
Upload .b2g file to the Immersal server
Import all assets to the Unity 3D
Build mobile application
Augmented Reality or AR applications enhance real-world experiences by augmenting a user’s visual perception with the display of digital content in the physical world.
AR’s ability to weave context-specific, 3D information into physical spaces provides endless opportunities to save time, improve performance and reduce costs across a wide range of industries and applications — from surveying, construction, public safety and manufacturing to maintenance, training and navigation applications. In 2021 Hexagon acquires an innovator of spatial mapping and visual positioning solutions for producing augmented reality (AR) applications - the Immersal company. These guys have an SDK that allows you to build centimeter-accurate, persistent multiuser AR applications that can be used on AR-capable mobile devices and with glasses. Their technology uses images from any device to extract features and you could try it by yourself just with your phone, there is a free option for that workflow, but it has some limits on capacity. If you want to use the unlimited version with BLK2GO files support, you should get the Enterprise license from them, but you could take it for free, to test it, just message the Immersal guys.
So in this tutorial, I'll show you how to combine the Leica BLK2GO handheld scanner with the Immersal SDK.
First that you need to do is scan your area with the Leica BLK2GO handheld scanner. The scanning process is very simple, you should do the below steps:
Put the scanner on the platform and turn on it
Open BLK2GO Live mobile application and connect it to the scanner
Push the power button to start the scanning
Walk around your area with the scanner
Go back to the starting position and put the scanner on the platform
Keep in mind that the Immersal SDK uses images to extract features, so be sure the pictures will be good. Avoid these issues:
Redundant objects (your face on the picture, another person that walk with you, parts of a vehicle if you use it for movement)
Too fast movements (pictures could look blurred)
Too dark areas (could be not enough data to extract features)
Generally, its basic recommendations for scanning also because for the VIS requirements are almost the same. So you could use the below link to download the best practices guide for BLK2GO.
The next step is to download data from the scanner and use it to digitize the current environment and for creating maps that will be used by the mobile application. You could use the BLK Data Manager to download the data from your devce. This simple, lightweight utility is designed to help you quickly download data from your BLK and make minor modifications to your device.
Run this tool, choose your device and click the connect button. Select required scans and click the Download button. After downloading you'll get the .b2g files that should be uploaded to different places, depending on their purpose:
Immersal Development Portal - to extract features and create a map file
REGISTER 360 - to prepare the data and send it further to another software for creating deliverables
So if you already have some object that you want to add to your scene you could use this workflow:
Download data with BLK Data Manager
Import it into immersal portal
Use the map created by Immersal portal and any other 3D objects created in other software in Unity 3D for creating a mobile app.
Install the mobile application to your phone or tablet to use the AR
But if you want to use the point cloud from BLK2GO to create some content that should be placed into your scene, then you should prepare the data in REGISTER family software and bring all files to Cyclone 3DR or CADs to extract the required materials. In this scenario use the below workflow:
I'll show you the second scenario with creating some 3D content, which could be used in your scene. To prepare the data I'll be using REGISTER 360. I have a clean point cloud and only one track so I don't need to clean the data or use registering tools, so I'll skip the part on how to work with the BLK2GO data in the REGISTER 360, you could find many tutorials on Youtube on how to do that. Here is one example - Complete Leica BLK2GO Workflow. Let's export the data to LGS format and import it into Cyclone 3DR for analysis and convert it to the native Cyclone 3DR point cloud. One of the latest features added to Cyclone 3DR it's the Progress Monitoring tool, which helps you to identify which parts of a construction site have been built or not by analyzing the coverage of a scan on a BIM model. So let's import the BIM model which should be analyzed to Cyclone 3DR, I'll be using the IFC format for that. Most of the time, BIM files contain the full assembly of the object so you could just select only required and import only them:
Drag and drop the IFC file to the Cyclone 3DR window
Select required elements
Select at least one cloud and a BIM object then launch the Progress Monitoring command
Set percentage of coverage threshold for considering the sub-element as Incomplete, Not installed, and Installed
Left on the screen only Not installed elements and export them to FBX format
As you could see in my case all engineering was not installed and was colored grey. The software marks them as undefined because they don't have relevant points in a defined radius. But some of them were colored red because they got some overlapping points. So I separate them into different folders for easy management.
For one more application, I modeled some elements inside the wall to imitate the scenario when you make scans on different construction stages, so we could see the number, type, and position of elements that was installed before the wall was covered finish layer. I combined all elements in the Cyclone 3DR just to be sure that all of them were in the same coordinate system and then exported them to FBX format.
One of the main objects that will be needed for creating an AR navigation application is the floor surface, which will be used for calculating the closest trajectory to the specified point. There are different ways how to create it, I decided just to:
Draw manually a simple contour for the required area
Extract the best plane from ground points and convert it to a mesh object
Cut the mesh with the contour and next export it to FBX also
The next step is to combine all materials in Unity 3D and build your own mobile application. For that you'll need to install the below components:
I'll not be going to show how to install Unity, just install it with default settings, next download and install the Immersal SDK from https://developers.immersal.com/ and after that download from the same page the sample project that will be suitable for your device, it could be Android, Huawei, iOS, etc. Add it to the Unity Hub and run it. Detailed information about the installation and capabilities of the Immerse SDK you could find in the below resources:
FAQ and instructions on GitBook
Vimeo channel with tutorials and promo materials
Discord group where you could ask any advice from other users and developers
Few words about Immersal Developer Portal, you'll get access here after registration and the main use for the Developer Portal is to download and manage your maps constructed using the Immersal Mapper mapping app or automatically generated from .b2g files. You can also find links to the latest SDK core package and Unity sample projects in the portal:
The latest version of SDK
Project samples
Private developer token
Upload BLK2GO files for creating a maps
Downloadable files (maps)
When you'll get access to the Developer Portal, click the Upload.b2g button and choose your blk2go files. After processing the data you'll see that now there are some files that have appeared. Download the file with the .bytes extension, this is the actual map file used by Immersal SDK for localization. Now you have all that you need to start creating your own AR application in Unity 3D. Make the next steps in the below order:
Run the Unity Hub and add there project with samples from Immersal
Open it and load all packages that the software will ask you to add
After everything will be loaded go to the Assets/ ImmersalSDK/Samples/Scenes folder and double-click on Navigation Sample
Add created in Cyclone 3DR models to some folder
Add created map from BLK2GO to any folder and replace AR Map in your Scene
Align the model and map if you changed the coordinate system in REGISTER 360. If you didn't touch it, then the map and model should be aligned automatically
Delete the existing mesh floor and add your floor to the AR Space, select it, and change its layer to Navigation. And I recommend you disable the visibility of this floor. For that switch Rendering Layer Mask to Nothing for that object
Select Generate NavMeshSurface here file. Click Clear and Bake to generate a new surface for navigation, it will be based on your floor
Change name and location of navigation markers
Add any materials to your objects. I added a Red transparent color for the elements that clash with existing objects. Orange transparent material for elements inside the wall. And other colors for not installed elements.
After all required elements will be added to your scene, you could start to build your application. Go to File/Build Settings choose the required platform and click Build. I used an Android template to test this application, so I built the android version and I deleted all scenes in Build except Navigation Sample, because I didn't use them right now. Next, click the Build button. The Unity 3D will generate a .apk file that you should copy and install on your android device.
When you'll run the application it requires a few seconds to define your position in the scene, after it will be done, you'll see the Successful Localization green icon. Now you could see on your screen all added models:
Orange for inside wall elements
Red for clashes
Other colors for non-installed objects
In the Navigation sample, you also could select one of your Navigation Markers, and the application will generate the closest route to your target and avoid obstacles because previously we generated floor mesh only in areas where is possible to walk. You could use this sample for different scenarios which require visualizing the routes to different places.
Immersal team created samples for different scenarios, which you could find there:
Multiplayer sample
Mapping sample
Content Placement
And others, go to their Discord channel to see how users create different applications
As you could see it's not so hard to create your own AR application and you should remember that in theory, the Immersal SDK could work with any imaging device so if you have some specific task just discuss it with the Immersal team, they will help you to find the right solution. But if you'll use the BLK2GO scanner it will expand your capabilities because it delivers to you not only images but also accurate point clouds.
If you still have a question or you want to repeat this tutorial by yourself, but don't have a valid license then you could reach me by clicking the below button.