Processing Data in NavVis IVION

With NavVis IVION Processing and NavVis IVION Processing Enterprise, you can post-process your datasets in a cloud-based environment within NavVis IVION.

Prerequisites

NavVis IVION Processing

• A NavVis Cloud account.

• Data scanned with NavVis M6 or NavVis VLX with a valid NavVis IVION Processing license. Contact your Customer Success Manager or NavVis Technical Support for details.

• Administrator rights to a NavVis IVION instance or membership in a NavVis IVION user group with the Data Manager role.

NavVis IVION Processing Enterprise

• An active NavVis IVION Enterprise instance

• Data scanned with NavVis M6 or NavVis VLX with a valid NavVis IVION Processing Enterprise license. Contact your Customer Success Manager or NavVis Technical Support for details.

Usage Metrics

These following metrics will give you an overview of your data usage for all sites and instances.

Cloud Storage

Cloud storage displays the storage size of all the datasets uploaded (recorded data) and processed in NavVis IVION Processing and NavVis IVION Processing Enterprise.

Note: The counting does not affect the storage size and usage of NavVis IVION instances.

For information of how the cloud storage is calculated and storage management tips refer to Calculating the Storage and Processing Units

Processing Balance

Processing balance accounts for post-processing and usage of other features such as Person blurring. The area is counted only within 10 meter range of mapping trajectory. Data outside of the radius is processed, but not billed. Overlapping area is counted only once against the balance.

Note: Successful reprocessing of datasets is also counted against the balance.

Processing Units are units of usage which can be used for all of data processing functionalities, such as post-processing and Person Blurring.

Note: Processing Units are non-refundable and expire after one year. They require at least one active NavVis IVION Core or NavVis IVION Enterprise instance.

For information on how processing units are calculated, refer to Calculating the Storage and Processing Units

Uploading Data

Follow these steps to upload data for processing:

1. Go to Site Setup > Upload Unprocessed Data.

   

2. Upload your data. You have two options to do that:
   • Click Upload Data and choose the file(s) or folder(s) you want to upload.
   • Drag and drop the file(s) or folder(s) you want to upload.

Once the upload has started, your datasets will be listed on the Upload Unprocessed Data page with the following information:

• Name
• Status
• Date and time the dataset was recorded.
• Number of panorama images included in the dataset.
• Size of the scanned area.

• Size of the recorded dataset.

Processing Data

You can create a processing task while your data is still uploading. Once the upload is complete, the data will be processed.

Follow these steps to create a processing task:

1. Go to Site Setup > Processing Tasks.

   

2. Click Create processing task.

3. Enter a name for the processing task or continue with the assigned default name.

4. Select the dataset(s) you want to process.

5. Perform the following steps to crop datasets before processing:
   1. Click the three dots next to the dataset you want to crop.
   2. Click Define processing time frame.
   3. In the screens that displays, use the slider to choose the part of the dataset you want to process.
   4. Click Define time frame.

      

6. Click Next.
7. Choose the desired point cloud and panorama settings.
8. Optionally, upload measured control points in .txt or .csv format. To do that:
   1. Click Upload or drop a control points file (.csv, .txt) here to upload a file or drag and drop a file.
   2. In the dialog box that opens, select Local reference system if your control points file is in a local coordinate system.

      

   3. Select Spatial reference system if your control points file is in a global coordinate system. Start typing a name and select the desired spatial reference system from the options that appear.

      

   4. Click Upload file.
   The control points file will be validated. If there are problems with your control points file, this will be indicated and you will need to resolve the problem to ensure the successful processing of your data.
9. Optionally, add up to five email addresses to receive updates on the completion or failure of your processing task.
10. After finishing configuration for new processing task, you can start processing the task. Click Start Processing.

Once you have created a processing task, it will be listed on the Processing Task page with the following information:

• Name

• Status

• Starting date

• Number of datasets included in the processing task.

• Number of datasets that have been successfully processed.

• Number of datasets for which processing has failed.

• Units consumed for all datasets within the processing task.

• Size of all datasets within the processing task.

• Progress of the processing task.

You can expand a processing task to see the following information:

• The datasets included in a processing task.
• The settings chosen for a processing task.

• The Status of the individual datasets:

  • Finished (in green): The processing task has completed successfully. The amount shown in the Units Consumed field will be deducted from your processing units balance.

  • Failed (in red): The processing task has failed. This could be caused by data incompleteness or an issue with the processing software. The Units Consumed filed will be empty and no units will be subtracted from your processing units balance.

  • Canceled: You have canceled the processing task. The Units Consumed field will be empty and no units will be subtracted from your processing units balance.

  • Finished (in yellow): The processing task has completed but a large residual in the control points has prevented the task from finishing successfully. Click View control point configuration to see which controls points have large residuals. For more information on this issue go to Knowledge Based Articles.

• Billed area This is displayed after processing is completed, this will be deducted from the Processing Balance.

• Units Consumed:
  • The unit factor of 10 Units/m² processed as Billed Area will be consumed from remaining Processing Balance for datasets captured with NavVis M6 or NavVis VLX 2.

  • The unit factor of 25 Units/m² processed as Billed Area will be consumed from remaining Processing Balance for datasets captured with NavVis VLX 3.

  • The unit factor of 100 Units/Panorama will be consumed for detection and blurring from remaining Processing Balance.

Running Multiple Operations

NavVis IVION allows you to queue operations to reduce waiting time.

You can interact with the Dataset Management page even when an operation is running. That way, you can start multiple operations simultaneously.

If you are running operations on a bundle or dataset, you can start operations on other bundles and datasets.

Blocked Operations

Actions that impact in-progress jobs will be blocked. If an operation on a bundle or dataset is already running, you will not be able to start any other operations on that bundle or dataset.

Blocked operations appear grayed out.

Calculating the Storage and Processing Units

This section describes how the amount of storage is calculated and how to manage the storage space.

Calculating the cloud storage

This section describes how Cloud Storage and Processing Units are calculated.

Cloud Storage (GBs): NavVis IVION Processing and NavVis IVION Processing Enterprise provides you with a specific amount of cloud storage space. The storage consumption is calculated based on all recorded datasets that you upload to your NavVis IVION site. Additionally every file that you can download after data processing is also calculated in the space, such as; panoramic images, Point cloud (*.ply,*.e57 with and without embedded pano), quality map, and log files.

Calculating the processing units

Processing Units: These units are calculated based not only on standard post-processing usage, but also on the advanced features and functions such as Person Blurring. Successful reprocessing of datasets are also counted against the balance while failed processing tasks are not counted. The processing units are shared between different devices belonging to the same account and they are non-refundable.

Note: The storage metric in data processing is used across all instances, this means that if you have multiple instances, they will all be consuming the same storage

Managing Cloud Storage

There are two options when your instance is running low on storage capacity.

• Remove unwanted files by deleting old uploaded and processed datasets from sites and instances.

• Upgrade your NavVis IVION Processing or NavVis IVION Processing Enterprise package.

Checking the Storage in your NavVis Cloud Instances

The following describes how to view the amount of cloud storage being used if you have multiple instances.

1. Go to NavVis Cloud to view a list of all of your instances.
2. Open each instance and check which sites are consuming the cloud storage.

   

   Note: To check storage consumption for NavVis IVION Processing Enterprise, please contact your Customer Success Manager or NavVis Technical Support.

Removing Unwanted Data

As the cloud storage is shared across all your instances, it is possible that you could run out of storage even on a newly created site or instance. When the amount of storage space has been exceeded, the screen shown below will display. However, removing old datasets uploaded and processed data from sites and instances can free up additional storage space.

Before deleting any datasets, it is important to keep in mind the following:

• Have a back up ready either on premise.

• To have successfully processed the data already so that you can delete the raw data.

• To have moved the data to NavVis IVION so that you can delete the processed data.

Remove Processed Data from a Site

1. Go to the Home page and select the site where you want to remove data.
2. Go to Site Setup.
3. Click Processed Data.
4. Check the corresponding box of the dataset you want to delete.
5. Click Delete.

Removing Uploaded Datasets from a Site

1. Go to the Home page and select the site where you want to delete data.
2. Go to Site Set-up.
3. Click Upload Processed Data.
4. Check the corresponding box of the dataset you want to delete.
5. Click Delete.

Upgrading your Data Processing Package

NavVis IVION Processing and NavVis IVION Processing Enterprise offer several packages depending on your business needs. If an instance runs out of cloud storage or processing units you can upgrade to a package with a higher storage capacity. The table below shows the different packages available.

For further questions or assistance with upgrading your storage and processing balance capacity, please contact your Customer Success Manager or send an email to sales@navvis.com

For information on how the billed area is calculated, refer to the KBA here.

NavVis IVION Processing packages

	Processing Units	Storage for Data
IVION Processing 0.25 (Add-On)	250,000	No Additional GB
IVION Processing 1 (Add-On)	1,000,000	No Additional GB
IVION Processing 5	5,000,000	2,500
IVION Processing 10	10,000,000	5,000
IVION Processing 15	15,000,000	7,500
IVION Processing 20	20,000,000	10,000
IVION Processing 25	25,000,000	12,500
IVION Processing 30	30,000,000	15,000
IVION Processing 35	35,000,000	17,500
IVION Processing 40	40,000,000	20,000

NavVis IVION Processing Enterprise packages

	Processing Units	Storage for Data
IVION Processing Enterprise 5	5,000,000	2,500
IVION Processing Enterprise 10	10,000,000	5,000
IVION Processing Enterprise 15	15,000,000	7,500
IVION Processing Enterprise 20	20,000,000	10,000
IVION Processing Enterprise 25	25,000,000	12,500
IVION Processing Enterprise 30	30,000,000	15,000
IVION Processing Enterprise 35	35,000,000	17,500
IVION Processing Enterprise 40	40,000,000	20,000
IVION Processing Enterprise 50	50,000,000	25,000
IVION Processing Enterprise 60	60,000,000	30,000
IVION Processing Enterprise 100	100,000,000	50,000

Point Cloud Settings

The following point cloud settings are available for data captured with NavVis M6 and NavVis VLX:

• Point Cloud Resolution
• Point Cloud Color

The following point cloud settings are only available for data captured with NavVis M6:

• Sharp Edges (default): This mode produces point clouds with details and edges in the scanned area reconstructed with a low amount of smoothing during filtering. However, the point cloud may contain noise and rough-looking surfaces.

• Balanced: This mode is a balance between smooth surfaces and sharp edges. The point cloud might contain more noise on surfaces than smooth-surfaces mode, but will have sharper edges.

• Smooth Surfaces: Surfaces will appear smooth and with a low amount of noise, but edges will appear less sharp compared to other modes. The point clouds will contain the lowest amount of noise with this mode.

The following point cloud settings are only available for data captured with NavVis VLX:

• High Confidence: In this mode only points with high confidence are retained and regions that have not been scanned densely are removed. This mode is recommended for narrow indoor environments.

• Standard: This mode is for general use and yields high-quality point clouds in a wide range of scenarios. This mode is recommended for larger indoor environments.

• Outdoor: In this mode points from sparsely scanned areas are retained. This mode is recommended for outdoor environments.

Panorama Settings

The following panorama settings are available for data captured with NavVis M6 and NavVis VLX:

• Person Blurring

• Floor Filling

• Include panoramas in .e57 point cloud

Control Points File Structure

TXT Files

To avoid problems with processing, a control points file in .txt format needs to be structured in the following way:

• The file needs to have four columns with the control point name first followed by the x, y, and z coordinates.

• The columns need to be separated by at least one space.

• Decimal places need to be indicated by periods. Commas are not allowed.

CSV Files

To avoid problems with processing, a control points file in .csv format needs to be structured in the following way:

• The file needs to have four columns with the control point name first followed by the x,y, and z coordinates.

• An optional header line (similar to: id, x, y, z) describing the columns can be present.

• Data beyond the fourth column will be ignored.

• Empty rows will be ignored.

• Fields should be separated by commas. Semicolons are supported but not suggested.

• Decimal places should be indicated by periods. Commas are supported but not suggested.

  

Resolving Problems with Control Points

When you upload a control points file, the file will be validated. If there are any warnings or errors you can deselect or edit the control points that are causing problems.

Excluding Control Points from Processing

Follow these steps to exclude control points from processing.

1. In your processing task, click View control point configuration.
2. Deselect the check boxes next to the control point(s) you want to exclude from processing.

   

3. Click Save changes.

Renaming Control Points

If you cannot find a control point, it might be due to inconsistent naming. For control points to be recognized by the system, the names that appear in the control points file need to correspond to the names used during mapping.

Follow these steps to rename control points:

1. In your processing task, click View control point configuration.
2. Click the pencil icon next to the control point that you want to rename.
3. In the window that opens, select the new name for the control point.
4. Click Rename.

   

5. Click Save changes.

After Processing

After processing your data, you can download the processed data and copy it to Dataset Management to view it in your NavVis IVION site.

Adding Processed Data to Dataset Management

Follow these steps to add the processed data to Dataset Management:

1. Go to Site Setup > Processed Data.

   

2. Select the dataset(s) you want to add.

3. Click Add to Dataset Management.

Downloading Processed Data

Follow these steps to download the processed data:

1. Go to Site Setup > Processed Data.

   

2. Select the dataset(s) you want to download.

3. Click the Download button.

4. Choose an output file and, if applicable, a data format.

   

   The following output files are available for download:

   • Control points alignment log: the log file generated during the alignment stage of post-processing.

   • Control points optimization log: the log file generated during the optimization stage of post-processing.

   • Control points verification log: the log file generated during the verification stage of post-processing.

   • Surveyed control Points: the file uploaded during configuring new processing task.

   • Dataset archive: an archive of the dataset, e.g. for backing up the data.

   • Filtered point cloud: the filtered and smoothed point cloud.

   • Filtered point cloud in control points coordinate system: the filtered and smoothed point cloud in the coordinate system of the control points used during processing.

   • Raw point cloud (NavVis M6 only): the filtered but unsmoothed point cloud.

   • Raw point cloud in control points coordinate system (NavVis M6 only): the filtered but unsmoothed point cloud in the coordinate system of the control points used during processing.

   • Other files: you can download panorama images by selecting Panorama images (.zip) from the list of data formats.

   If you download a point cloud file, you can also choose a data format. The following data formats are available:

   • .e57

   • .e57 with panoramas

   • .las

   • .ply

   • .pod (Pointools)

   • .rcs

   Note:

   The .e57 files contain unstructured point clouds and panorama images in a spherical representation if you enabled the option Include panoramas in .e57 point cloud in the processing settings. The files are compatible with the following third-party software:

   • Leica Cyclone 3DR Viewer

   • Cloud Compare

   • AVEVA

   • Prevu3D

   • PTC

5. Click Download.

Downloading Data Processing Log Files

If you need support using NavVis IVION Processing or NavVis IVION Processing Enterprise, contact support@navvis.com. To speed up the process, include the log files from the time of the problem in your support request.

Follow these steps to download data processing log files:

1. Go to Site Setup > Processing Tasks.

   

2. Click on the arrow next to the processing task for which you want to download log files.
3. Click on the three dots.
4. Click Download logs.