FactVerse Omniverse Adaptor User Guide

Overview

FactVerse Omniverse Adaptor is an extension plugin that connects the FactVerse platform with NVIDIA Omniverse. It is designed to help users quickly import digital twin scenes into the Omniverse environment and, in combination with DFS (Data Fusion Services), enable dynamic simulation and interaction within USD scenes.

Core features

This plugin offers the following core capabilities within the Omniverse application:

• Quick Import: Load USD models and configurations directly from the FactVerse platform.
• Real-time Driving: Connect to DFS data to drive dynamic scene changes.
• Multi-application Compatibility: Can be integrated with applications like USD Composer, Viewer, Explorer, and others.
• Flexible Deployment: Supports local operation or embedding into custom applications.
• Web Visualization: Works with WebRTC and web front-end for remote interaction and viewing in the browser.

Example use cases

The plugin is suitable for the following typical scenarios:

Plugin loading methods

System environment requirements

For development or testing environments, it is recommended to install the following components on the same high-performance workstation to ensure stable communication links and sufficient rendering performance:

Development tools and dependencies

Recommended hardware configuration (Standalone deployment)

Preparation

Before integrating the FactVerse Omniverse Adaptor plugin, please complete the following steps to ensure the environment is properly configured, resources are available, and data connections are established.

Obtain testing platform and account information

To perform plugin testing and build scenes, you will need to use the FactVerse platform and DFS platform provided by DataMesh.

Platform Information

Account Application and Resource Access

• Please send an email to lab@datamesh.com to request a trial account and testing materials. You can also download testing resources from the associated Hands-on Lab.
• Note: The above platforms are intended for use with the provided testing materials only.
  If you need to use your own enterprise data for testing or require an internal testing environment (private deployment), please contact lab@datamesh.com to request support solutions, ensuring data security and compliance.

Download the plugin toolkit

Please visit the following link to download and extract the required toolkit: FactVerse-Omniverse-Adaptor-v1.0.0.zip

Toolkit contents:

Build a digital twin scene

Before creating digital twin templates and scenes, please install the following essential design tools:

• DataMesh Importer: Used for importing and preprocessing 3D model files.
• FactVerse Designer: Used for template creation, scene building, and interaction logic configuration.

Steps

1. Prepare the model:

a) Open the model in DataMesh Importer.

b) Select the nodes in the model hierarchy based on the motion requirements for data-driven actions.

c) Upload the model to your FactVerse resource library.

2. Create and configure the template:

a) Create a new template in FactVerse Designer.

b) Drag in the model and add attributes, such as the rotation angle.

c) In the data binding area, link the model hierarchy to the attributes, then save the template.

3. Build the digital twin scene:

a) Create a new digital twin scene and batch-generate digital twins using the templates you’ve created.

b) The digital twins represent the actual devices—make sure to name them clearly for easier data binding later.

c) Adjust the position and layout of the digital twins to complete the basic scene setup, then save it.

Upload USD model resource

Convert your 3D model to .usdz format and upload it to the platform’s resource library.

Steps

1. Convert the model format: Use the conversion tool provided by DataMesh to convert common model formats (like FBX or OBJ) to .usdz files.

a) Extract the USDConnectorUnityTool_v0.0.1.zip from the plugin toolkit.

b) Run USDConvertUnityTool.exe.

c) Open the original model.

d) Once the model is open, click Export USD to export it as a .usdz file.

2. Upload the .usdz file

a) Log in to the FactVerse platform.

b) Navigate to Digital Assets-> Resources, select the model you want to upload and go to its details page.

c) Click Omniverse to upload the .usdz file.

Access data

Connect device data through the DFS (Data Fusion Service) platform and set up data-driven configurations for digital twin scenarios.

Steps

1. Create simulation data source:

a) Log in to the DFS platform, go to Data Source Management > Simulation Data Source, create a new task, and upload the data.

b) Bind the data to the corresponding device and set the data update frequency.

2. Import digital twin scene: In the DFS platform, go to Digital Twin Association > Scene Configuration, click Import, and select the digital twin scene to which the data is bound.

After importing, you can view the list of digital twins on the scene detail page.

3. Bind data to the digital twin:

a) Go to Digital Twin Association > Device Binding, find the device to bind (e.g., Robot), and click the details button to open the device’s detail page.

b) In the Device Configuration section, click the Configure Digital Twin button to open the configuration window and bind the corresponding digital twin.

c) In the Attribute section, click the binding icon next to each device attribute and associate it to the corresponding digital twin attribute.

4. Verify the scene: Go back to FactVerse Designer, open the digital twin scene with the bound data, click play, and observe the movement of the digital twin.

For more details, refer to DataMesh FactVerse DFS User Manual.

Install Omniverse Nucleus

Before using the Adaptor plugin, you must install and configure Omniverse Nucleus, which plays two essential roles in the system:

Role 1: USD Scene Resource Server

Stores and synchronizes .usdz files generated by the plugin. These resources are later loaded and rendered by Omniverse applications.

Role 2: Local API Provider

The Adaptor plugin calls local Nucleus APIs to perform operations such as uploading and querying scene resources.

Deployment Scenarios

Note: The machine running the Adaptor plugin must be able to access the remote Nucleus server over the network, or else USD loading and data-driven updates will fail.

Installation steps

1. Open the Omniverse Launcher and navigate to the Nucleus
2. Click Add Local Nucleus Service.

3. Set up the administrator account.

Configure the Nucleus service address

Steps

1. Visit http://localhost:3080/ to open the Nucleus service management page.
2. On the Apps page, click Enable sharing.

3. After starting Nucleus, click Add Server in the server list and enter the Nucleus server address (use the local machine address).

4. Once the configuration is complete, check the server list to confirm that the server has been successfully added.

Test connection and data-driven effects

You can import the Adaptor plugin into official Omniverse applications (such as USD Composer or Isaac Sim) to verify connectivity with the FactVerse platform, test DFS data-driven effects, and ensure the scene loads correctly.

Application scenarios:

• Verify that the plugin can successfully connect to the FactVerse platform.
• Test if DFS data driving works correctly (e.g., check if the robotic arm moves as expected).
• Provide visual verification support for future remote deployment and custom app development.

Import the plugin

Using USD Composer (USD Composer 2023.2.3) as an example, follow these steps to import the plugin:

1. Install USD Composer.
2. Import the plugin:

a) Open USD Composer and go to Window > Extensions in the menu.

b) Click Options > Import Extension in the top-right corner.

c) Select the datamesh.factverse_ovadaptor_extension-1.0.0.zip file and click Import.

d) In the Extension search box, type DataMesh and find FACTVERSE OMNIVERSE ADAPTOR, then click Enable.

Verify connection and data driving

Important Notes: Make sure you are in Edit Mode when importing the scene and integrating data.

1. After enabling the plugin, a new DataMesh menu will appear in the top menu bar. Click DataMesh > FactVerse Omniverse Adaptor to open the connection window.

2. In the connection window, enter the following information:

• • Nucleus IP: The address of the resource server that the plugin accesses. If the plugin and the resource server are on the same machine, enter localhost. If it’s a remote deployment, enter the server’s IP address (e.g., 192.168.1.1).
  • FactVerse Server URL: The access URL for the FactVerse platform.
  • Account ID: Your FactVerse account ID.
  • Password: Your login password.
  • Scene ID: The digital twin scene ID.

How to get the Scene ID:

a) Open the target digital twin scene’s detail page on the FactVerse platform.

b) Copy the ID from the URL in this format:

https://dtcs-local-demo.datamesh.com/…/6fa55ac9f44f4523ae0c166e299176d9/…

6fa55ac9f44f4523ae0c166e299176d9 is the Scene ID.

3. After clicking the CONNECT button, the plugin will automatically load the scene and bind the data.

4. After completing the connection, click [OPEN STAGE AND JOIN SESSION].

Once these steps are complete, you will be able to see the digital twin scene driven by real-time FactVerse data in USD Composer. This setup provides the foundation for remote publishing and debugging.

5. After completing the verification, click the [STOP] button to stop the Adaptor.

Plugin integration and custom development

This chapter explains how to develop a customized USD Viewer Streaming application based on the Omniverse Kit framework and integrate the Adaptor plugin into it. With this integration, users can achieve the following key capabilities within the application:

• Load digital twin scenes from the FactVerse platform.
• Bind DFS (real-time or simulation) data to drive dynamic changes in the scene.
• Render the scene within the USD Viewer Streaming
• Enable remote access and interaction control via a web browser.

This solution is suitable for enterprise-level deployments, remote visualization, project delivery, and multi-user interactions. It is the primary integration method for transitioning the plugin from local debugging to a production environment.

Data flow explanation:

1. The Adaptor plugin downloads the USD model and digital twin scene configuration from the FactVerse platform and is responsible for writing scene content and data changes to the local Nucleus.
2. DFS continuously pushes device or sensor data to the Adaptor plugin.
3. The Adaptor plugin writes the data to USD attributes in the scene (such as position, angle, color, etc.) based on the mapping, enabling dynamic control of the scene.
4. The USD Viewer loads and renders the current scene from Nucleus, reflecting the real-time changes written by the Adaptor plugin.
5. The Messaging plugin responds to web requests, calls the USD Viewer API to load the target scene, and monitors events and state changes in the scene.
6. The Web Client sends load/control commands to the Messaging plugin, receives feedback, and enables remote scene control and state synchronization.
7. The Streaming module captures the rendered scene from the USD Viewer, encodes it into a video stream, and uses WebRTC to push it to the Web Client, enabling 3D scene visualization in the browser.

Create a USD Viewer Streaming application

1. Initialize the App project:

2. Fill in the required information:

• • Type: Application
  • Template: [omni_usd_viewer]
  • App Name: my_usd_viewer
  • Version: 1.0

3. Configure extension plugins:

During the initialization process, you will need to configure the following two core extension plugins:

a) USD Viewer Messaging Extension

This extension handles command communication and status feedback between the web client and the viewer application.

b) USD Viewer Setup Extension

This extension manages in-app configuration, communication, layout, and other settings.

4. Enable Streaming:
   During application initialization, select [omni_default_streaming]: Omniverse Kit App Streaming (Default) to enable WebRTC streaming for browser-based visualization and interaction.
5. Build the project

 Integrate the Adaptor plugin and Messaging plugin

1. Extract plugins to the extensions directory: Unzip the following two files into the source/extensions/ directory:

• factverse_ovadaptor_extension.zip
• factverse_viewer_messaging_extenslon.zip

2. Configure dependencies:

a) In datamesh.usd_viewer.setup/config/extension.toml, add a dependency entry for the Messaging plugin.

b) In datamesh.factverse_viewer_messaging_extension/config/extension.toml, add the Adaptor plugin dependency.

c) In source/apps/datamesh.my_usd_viewer_streaming.kit, add a dependency for the Live Session feature.

3. Set connection parameters: Edit factverse_ovadaptor_extension\config\extension.toml

Parameter description

4. Create hard links (Optional): To avoid path length limitations on Windows, you can create hard links to shorten file paths.

Development and debugging

1. Add the Adaptor plugin source code by adding an Extension Search Path in USD Composer. For example: d:/temp/kit-app-template/source/extensions

2. Enable the plugin

3. Use VSCode to open the plugin’s source code and begin debugging and development.

4. Launch the application

After confirming that development and debugging are successful, you can run the following command to package and publish the application:

Web frontend deployment

After completing the plugin integration and launching the USD Viewer application, if you want to remotely access the scene and perform interactive operations via a browser, you can deploy the accompanying Web frontend project. This frontend is based on modern frameworks (such as Vite + React) and uses WebRTC to display USD renderings in real-time, while also communicating with the plugin to control data.

Reference Example: Omniverse Configurator Sample Application (provided by DataMesh).

1. Start the frontend application

The default access URL is: http://localhost:5173. Open this URL in a browser to enter the Web interface for operations.

2. Start the Adaptor plugin

Click the Start Adaptor button on the interface to start the backend data-driving plugin.

3. Check plugin status

Click the Get Adaptor Status button to check if the Adaptor plugin has successfully started.

4. Load the USD scene

After confirming the Adaptor is running, click the Open USD and Join Session buttons to load the scene. The USD Viewer will retrieve and render the scene from Nucleus, with data driving automatically applied.

After loading, the page will display the USD Viewer’s rendering and allow for interactive operations within the scene.