Embedded Files
Preview:
Preview:
Leica Cyclone ENTERPRISE intro
Pycharm installation
Computer Vision - OpenCV libraries
Tesseract installation
First image recognition test
Binding hotkeys for point ID
Virtual Surveyor test with TruView LIVE
Import all points into Autodesk Civil 3D
Benefits of this approach
Hey Everyone! 🌟 Get ready for an exciting tutorial where I'll show you how to transform a regular point cloud viewer into a virtual surveyor tool with a dash of computer vision magic! 🚀 Building on my previous tutorial on AI tools for custom object detection using panoramic images from a laser scanner (check it out here - AI tools in Reality Capture), this time, we're taking it up a notch. Learn how to teach your PC to read text from images through computer vision and leverage it for seamless Plan generation in AutoCAD Civil3D.
As the point cloud viewer, I'm going to use the Leica Cyclone ENTERPRISE platform with the Leica TruView LIVE web browser viewer. If you're not familiar with Leica Cyclone ENTERPRISE, it delivers a simplified management & collaboration platform. Powered by Leica Geosystems’ JetStream technology, Cyclone ENTERPRISE facilitates:
Reality Capture Project Management.
Control user & data management features from a simple, online, browser-based administration portal.Secure, flexible, self-managed deployment so you can meet your unique IT needs
Provide secure access to internal & external users on a per-project basis. No data leaves the premise. Sessions are managed centrally & remotely. You can assign different roles to users with different abilities.
Access to rich, full 3D Reality Capture data through desktop programs and web browsers Point clouds, imagery, models, and more.
Interoperability with all major CAD products from Hexagon, Autodesk, Bentley & others via CloudWorx & JetStream powered technology, delivering rich, reality capture-based feature extraction & analysis workflows.
My favorite advantage of this server-based approach is that you can store all your vast amounts of data in a single space and access it easily through your browser, desktop viewer, or CAD software without downloading it. The JetStream technology streams the data directly from the server to your device.
Leica Cyclone ENTERPRISE - your own server storage for Reality Capture data
To extract coordinates from this browser viewer, we'll need to create some custom scripts. For that, I'll use the Python language and PyCharm software. PyCharm is an integrated development environment (IDE) specifically designed for Python development. Let's install PyCharm; you can download the free Community Version from this link. I'll use the version for Windows. After installation, create a new project and specify the location for it. When you open the new project, you'll see that a virtual environment will be created, and by default, there will be a main.py script, which you can run just to check that everything is working. To use modules for computer vision, you need to install some libraries. For installation, go to the terminal window and type the following commands:
''pip3 install opencv-python" - used to install the OpenCV library for the Python programming language. OpenCV (Open Source Computer Vision) provides a wide range of functions and tools for image processing and computer vision. This library is widely used in areas such as computer vision, machine learning, image processing, and robotics.
''pip3 install PyAutoGUI'' - the module that provides functions for programmatically controlling the mouse and keyboard.
''pip install pyscreenshot" - the module that provides a simple interface for taking screenshots programmatically.
''pip install pillow'' - powerful image processing library
''pip install numpy'' - library for numerical and mathematical operations in Python
Also, you need to download and install Tesseract separately. Tesseract is an open-source optical character recognition (OCR) engine developed by Google. OCR technology is designed to recognize and extract text content from images or scanned documents. You can download the Windows version here. While installing, please remember the path of the installation folder, you'll need it later.
To use Tesseract from any location in the command prompt, you can add the Tesseract installation directory to the system's PATH environment variable. To do that, go to the Environmental Variables menu:
Under "System variables," find the "Path" variable and click "Edit."
Click "New" and add the path to the directory where Tesseract is installed (e.g., C:\Program Files\Tesseract-OCR).
Click "OK" to close the windows.
Go back to the Pycharm terminal and type the ''pip install pytesseract'' command. Be sure that in your python project folder "Your python project/.venv/Scripts/"- pytesseract.exe will be installed.
PyCharm interface
To perform a simple test and ensure everything is working, follow these steps:
Create a screenshot with any text.
Save the image into your project folder.
Run the script provided below:
Simple Image to Text script
Simple Image to Text script
#1. Import Libraries
import pytesseract
from PIL import Image
#2. Open Image
img = Image.open("image_coordinates.JPG")
#3. Specify Tesseract Command Path
pytesseract.pytesseract.tesseract_cmd = r"C:\Users\soal\AppData\Local\Programs\Tesseract-OCR\tesseract.exe"
#4. Extract text from the image with OCR
text = pytesseract.image_to_string(img)
#5. Print Result
print (text)
This script uses the Tesseract OCR (Optical Character Recognition) engine, along with the pytesseract and PIL (Pillow) libraries, to extract text from an image. Let's break down the script step by step:
Import Libraries
Open Image
Specify Tesseract Command Path - The path to the Tesseract executable (tesseract.exe) is specified. This is done using pytesseract.pytesseract.tesseract_cmd. Make sure to provide the correct path to the Tesseract executable on your system.
Extract text from the image
Print Result
If you run it, you'll see in the console that the text from your image will be recognized and extracted, which means everything is working.
Image to Text script
Next, we should teach the program to take a screenshot of the desired area and save it as a file. To define the area, we'll use the coordinates of your screen. You can extract the coordinates using the simple script below:
Get Mouse Cursor Coordinates
Get Mouse Cursor Coordinates
#1. Import Libraries
import pyautogui
import time
#2. Pause Execution for 2 Seconds: This allows the user to position the mouse cursor where they want before any subsequent actions are performed.
time.sleep(2)
#3. Print Mouse Cursor Position.
print(pyautogui.position())
The script waits for 2 seconds to give the user time to position the mouse cursor, and then it prints the current coordinates (X, Y) of the mouse cursor on the screen. You need to get the coordinates of the left upper corner and right bottom corner to define the area for a screenshot. Add these coordinates to the below script.
Script - Create screenshot of desired area and extract text from it
Script - Create screenshot of desired area and extract text from it
#1. Import Libraries
import time
import numpy as np
import pyscreenshot as ImageGrab
import cv2
import os
import pytesseract
#2. Initialize Variables
filename = 'Image.png'
x = 1
last_time = time.time()
#3. Capture Screenshots in a Loop
while (True):
screen = np.array(ImageGrab.grab(bbox=(0, 200, 992, 771)))
print('loop took {} seconds'.format(time.time() - last_time))
last_time = time.time()
cv2.imshow('window', cv2.cvtColor(screen, cv2.COLOR_BGR2RGB))
cv2.imwrite(filename, screen)
x = x + 1
print(x)
if x == 2:
cv2.destroyAllWindows()
break
#4. Perform OCR on Captured Screenshot
img = cv2.imread('Image.png')
text = pytesseract.image_to_string(img)
print(text)
Run this script and test that it's working fine. You can also check the captured screenshot by opening the "Image.png" file.
Script - Create screenshot of desired area and extract text from it
As the next step, I decided to modify the script to keep only numerical values and save all extracted coordinates into a .txt file. To make it faster, I created one more script for hotkeys. It runs the previous script by pressing a specified key on the keyboard and assigns different names to the coordinates. Name codes are based on the Description Key Set parameter from Autodesk Civil 3D. A Description Key Set is a set of rules or instructions that defines how survey data, specifically points, is translated or converted into objects within the Civil 3D environment. If you use codes from this Description Key Set, then when you import your .txt file into Civil 3D, all objects will be generated automatically based on the rules from the Description Key Set. I already demonstrated this approach in my previous tutorial about Road feature extraction in Cyclone 3DR software.
So I decided to assign the below codes to the hotkeys:
1 - STR: for regular structure elements
2 - TR: for trees
3 - PL: for poles
4 - SWMH: for manholes
5 - SIGN: for road signs
Of course, you can change these namings or assign another hotkey. Just make sure that these keys are not already in use by default in the software.
Autodesk Civil 3D - Description Key Set
To understand which points were extracted, I decided to create another script that will generate a Geotag at each extracted point. The script should also assign related names and categories to the Geotag for easy management. For that, we can use a Python module that provides functions for programmatically controlling the mouse and keyboard.
The new script allows the user to define specific actions associated with certain keys. When a defined key is pressed, the script performs a series of mouse and keyboard actions, including running an external script (Find.py) and interacting with specific areas of the screen. The script is set up to be responsive to specific hotkeys and demonstrates basic automation capabilities. The Find.py script performs image-to-text conversion. The Hot Keys 3.py script is in charge of hotkeys.
Note that the script relies on specific screen coordinates, and you may need to adjust them based on your application's UI. All final scripts can be downloaded below.
Leica TruView free viewer - Geotags colored by category (Leica BLK2GO data)
Now, all Geotags are generated in the specified location automatically by pressing the related hotkey. If you stream this point cloud to desktop Leica TruViewer, you'll be able to colorize and filter them by category. Point coordinates can be extracted in 3D point cloud view or in Image Panoramic view. A text file is generated simultaneously, so after finishing, you can import it into your CAD software, and all icons will appear automatically.
You may ask why this is needed if I can just import a point cloud into my CAD software and start generating drawings there based on the point cloud data. I agree that it's easier, but imagine a scenario where you have 20-30 drawing engineers in your organization, and all of them should work on a big project (around 200-300 GB or even more).
Data size: If you're working with a file-based approach, it means all users should store the file on their own PCs. This implies that 20 users will hold 4TB of data on their PCs. With the Leica Cyclone ENTERPRISE Server-based approach, they can simply open this data via a browser or desktop viewer and use only server storage to keep this data. An additional benefit is that everyone will use a single source of information, and you can share links to this project with stakeholders outside of your organization, allowing them to view the data online without downloading.
Data management: If you're going to split the project between your engineers, you'll likely be forced to segment the point cloud by blocks or create some borders for each user to let them work with specified regions to avoid duplication of the extracted features. All these operations will take additional time and may lead to inconsistencies in recording measurements and details over time or across different observers. With the latest point cloud processing software, Leica REGISTER 360 Plus 2023.1, you can easily divide the whole area into multiple Limit Boxes, and information about these Limit Boxes will be published to the Leica Cyclone ENTERPRISE server. This means that each user can work with a related Limit Box only. If someone prefers to use a tablet in the field to collect asset information for Geotags, then they can easily merge all collected data with the existing project on the Leica Cyclone ENTERPRISE server.
Sometimes, even if you have 20 engineers within your organization, it doesn't mean that all of them have licenses for the software. Perhaps they are using 10 floating licenses when they are available. That means you'll be forced to invest additionally in extra licenses. If the user is not familiar with point cloud processing, it will take additional time to train them. Leica Cyclone ENTERPRISE allows connecting an unlimited number of users via the free Leica TruView viewer with a very simple and user-friendly interface.
Computer specification: Even if you provide all your engineers with enough software licenses, it doesn't mean that all of them have suitable computer specifications to handle a 200-300 GB point cloud in their software. That will force you to spend additional investments on computer upgrades. With Leica Cyclone ENTERPRISE, if you open the project via a browser, then all computing will rely on server resources. This means end users don't need to have powerful PCs to work with this data; it can be opened even via a tablet.
As you can see, for a big project or companies with a large number of employees, divisions, or stakeholders, this server-based approach is a must-have. And keep in mind that if needed, you can stream the reality capture data from the Leica Cyclone ENTERPRISE server to your CAD software via the Leica CloudWorx plugin.
Leica Cyclone Enteprise benefits table
These were my top 10 favorite tools and approaches in Leica Cyclone 3DR for construction analysis applications. Instead of ranking them by popularity, I've categorized them based on the main construction stages – from groundwork to concrete works, installation analysis, and so on.
Of course, there are many more tools you can utilize in your daily work. If you're eager to learn more, head over to the Leica E-learning platform, where you'll find three comprehensive online courses covering various tools within Leica Cyclone 3DR. These courses include detailed workflow descriptions, assessment tests, and data samples. Upon completion, you'll even receive an official certificate from Leica Geosystems.
So, don't miss out on the opportunity to enhance your skills and boost your expertise. Explore these courses, and let's continue to build a brighter future in construction analysis with Leica Cyclone 3DR. Thanks for watching!
Page updated
Google Sites
Report abuse