How to Use Gaussian Splatting with NeRF Studio on Nebula Cloud

3D Modeling Deep Tech Nebula Cloud Computer Vision Generative AI Rendering

Are you eager to harness the power of Gaussian Splatting with NeRF Studio but don't want the hassle of dealing with installations, configurations, and hardware maintenance? Look no further! Nebula Cloud offers a seamless solution, allowing you to utilize Gaussian Splatting with NeRF Studio effortlessly. Follow our step-by-step guide to get started on Nebula Cloud.

Getting Started on Nebula Cloud

Step 1: Sign-up and Activate

Begin by signing up for Nebula Cloud (www.nebulacloud.ai) and activating your user account. This straightforward process sets the stage for your cloud computing journey.

Step 2: Search and Select

Use the search window to find "Gaussian Splatting with NeRF Studio." Choose the configuration that fits your GPU preferences and region. Ensure a minimum of 250GB SSD storage for an optimal experience.

Step 3: Workbench Creation

Within 3-5 minutes, Nebula Cloud will create the "Gaussian Splatting with NeRF Studio" workbench. This virtual environment comes preconfigured with NeRF Studio, Gaussian Splatting, and various multimedia software.

Step 4: Connect and Go

Simply connect to your virtual cloud workstation using the provided login credentials. Start running Gaussian Splatting with NeRF workflows without the hassle of software and hardware management.

Key Features of Nebula Cloud's Gaussian Splatting Environment

• - Preconfigured Workstation: Virtual Windows 10 Professional Workstation with NVIDIA Instant NeRF, Gaussian Splatting with NeRF Studio, and multimedia software.
• - Rapid Deployment: Start using the cloud workstation in less than 5 minutes.
• - No Installation Hassles: Nebula Cloud takes care of installation, configuration, and administration of software and hardware.
• - Data Migration Tool: Effortlessly move data between on-premises systems and the cloud workstation.
• - Flexible Scaling: Adjust hardware and storage based on project needs and data volume.
• - Built-in Storage: Up to 5TB SSD storage for efficient data processing and an additional 5TB HDD storage for storage and archival.
• - Visualization Tools: Preconfigured with NICE DCV, Agisoft Metashape, MeshLab, Blender, and more.
• - Pay-as-You-Go: Pay only for the time and resources used on the Nebula Cloud platform.

Gaussian Splatting Workflow on Nebula Cloud

Step 1: Setting Up NERFStudio Environment

Open Anaconda Command Prompt

bashCopy code
conda activate nerfstudio
cd C:\Users\Administrator\nerfstudio

Step 2: Processing Video Data

Command to Process Video Data

bashCopy code
ns-process-data video --data <location> --output-dir <location>

Example:

bashCopy code
ns-process-data video --data C:\nerfstudio-with-guassian-splatting\data\nerfstudio\ship\ship.mp4 --output-dir data\nerfstudio\ship

Step 3: Processing Images in NERFStudio

Command to Process Images

bashCopy code
ns-process-data images --data <location> --output-dir <location>

Example:

bashCopy code
ns-process-data images --data \data\nerfstudio\swiss-centre\images --output-dir \data\nerfstudio\swiss-centre\output

Step 4: Training

Command to Train

bashCopy code
ns-train splatfacto --data <data>

Example:

bashCopy code
ns-train splatfacto --data data/nerfstudio/poster

After Training Completion

The system will display the path where you can find your model in the output folder inside C:\Users\Administrator\nerfstudio\outputs\poster\splatfacto. It's recommended to rename it for easier use.

Adjusting Settings for Quality

bashCopy code
ns-train splatfacto --pipeline.model.cull_alpha_thresh=0.005 --pipeline.model.continue_cull_post_densification=False --data <data>

Step 5: Visualizing the Trained Model

Command to Visualize

bashCopy code
ns-train splatfacto --data <data> --load-dir <output-directory>

Example:

bashCopy code
ns-train splatfacto --data data/nerfstudio/poster --load-dir C:\nerfstudio-with-guassian-splatting\nerfstudio\outputs\poster\splatfacto\2024-01-26_134421\nerfstudio_models

Step 6: Rendering the Video

Command to Render

bashCopy code
python nerfstudio/scripts/gaussian_splatting/render.py camera-path --model-path GAUSSIAN_TRAINING_OUTPUT_MODEL_DIR -camera-path-filename YOUR_CAMERA_PATH_FILE.json --output-path YOUR_OUTPUT_MP4_FILE.mp4

Example:

bashCopy code
python nerfstudio/scripts/gaussian_splatting/render.py camera-path --model-path ..\..\gaussian-splatting\output\hanoi --camera-path-filename C:\gaussian-splatting\data\hanoi\camera_path.json --output-path  C:\gaussian-splatting\data\hanoi\hanoi-dragon.mp4

Step 7: Exporting Splats

Command to Export Splats

bashCopy code
ns-export gaussian-splat --load-config <config> --output-dir exports/splat

Third-Party Splat Viewers

1. Polycam Viewer: https://poly.cam/tools/gaussian-splatting
2. Playcanvas SuperSplat: https://playcanvas.com/super-splat
3. WebGL Viewer by antimatter15: https://antimatter15.com/splat/
4. Spline: https://spline.design/
5. Three.js Viewer by mkkellogg: https://github.com/mkkellogg/GaussianSplats3D

Note: Gaussian splats can only be exported from trained splats, not from nerfacto.

Feel free to copy and paste these steps into your development environment. For further assistance or inquiries, please contact support@nebulacloud.in. Welcome to the future of hassle-free computing with Nebula Cloud!