3. 3D / Spatial Modality (3D / Spatial / XR)

As applications move from "flat images/video" to autonomous driving, robotics, AR/VR/XR, and similar scenarios, systems are no longer satisfied with merely looking at "2D pixels" — they need to understand the three-dimensional structure, scale, and pose relationships of the real world. These tasks are collectively referred to as the 3D / spatial modality: they encompass both precise geometric and topological modeling, as well as semantic understanding, localization and navigation, and content generation within 3D space. On one end, they connect to various sensors such as LiDAR, RGB‑D, and IMU; on the other, they connect to autonomous driving perception modules, robot navigation systems, ARKit/ARCore environment models, mobile 3D scanning and modeling applications, and digital twin platforms.

3.1 3D Perception & Reconstruction

In 2D vision, we only see "the world after it has been photographed"; but in scenarios like autonomous driving, robotics, and AR/VR, what matters more is: the position, shape, and structure of the real world in 3D space. 3D perception and reconstruction aims to recover the three-dimensional geometric information of the environment from multiple sensors (cameras, LiDAR, depth cameras, etc.) and express it in forms such as point clouds, voxels, meshes, and implicit fields, providing the foundation for path planning, physics simulation, digital twins, and 3D content generation.

In engineering practice, this layer covers multiple technical directions from point cloud processing to multi-view geometric reconstruction to neural radiance fields / neural field rendering, corresponding to product forms such as autonomous driving 3D perception modules, ARKit/ARCore environment modeling, mobile 3D scanning/modeling apps, and digital twin city/campus modeling platforms. The following expands from three angles — scenarios, principles, and models — and further subdivides into several key sub-directions.

• Scenarios
  • Autonomous driving and assisted driving: perceiving 3D structures such as vehicles, pedestrians, curbs, lane lines, and traffic facilities from onboard LiDAR point clouds and multi-camera images, used for path planning and safety decisions.
  • Indoor/outdoor environment scanning: using phones/tablets (structured light / ToF / stereo) or handheld scanners to capture multi-view data, building 3D models of rooms, buildings, and neighborhoods in real time for AR modeling, home design, and digital twins.
  • Digital twins and BIM: reconstructing actual factories, campuses, and cities into high-precision 3D models through multi-view imagery and point clouds for operations management, simulation, and visualization.
  • Consumer-grade 3D scanning: mobile 3D scanning apps, one-click "photo-to-3D-model" tools that provide raw geometry for 3D printing, virtual try-on, and game/film asset production.
• Principles
  • Point cloud processing: treating sparse/dense point sets obtained from LiDAR or multi-view reconstruction as 3D sampling point sets, performing filtering, registration, downsampling, and feature learning, then doing classification, semantic/instance segmentation, or 3D object detection.
  • Multi-view geometry and 3D reconstruction: using SfM (Structure‑from‑Motion) to estimate camera poses and sparse 3D point clouds across multiple images, then using MVS (Multi‑View Stereo) to generate dense point clouds, followed by mesh reconstruction and texture mapping.
  • Neural radiance fields / neural implicit fields: using methods such as NeRF, Instant‑NGP, and Gaussian Splatting to represent a 3D scene as a continuous volume density/color field or a collection of Gaussian particles, generating images through volume rendering or rasterization and learning from multi-view supervision; once trained, they can perform novel view synthesis and geometry extraction.
• Models
  • Point cloud networks: PointNet / PointNet++, PointCNN, DGCNN, MinkowskiNet, etc. learn features directly on points or sparse voxels for point cloud classification, segmentation, and 3D detection. In autonomous driving, 3D detection frameworks such as VoxelNet, SECOND, and CenterPoint are commonly used, converting point clouds into voxel or BEV (Bird's Eye View) features before detection.
  • Geometric reconstruction toolchains: traditional SfM/MVS systems such as COLMAP and OpenMVG / OpenMVS can recover camera poses and dense point clouds from multi-view photos and construct high-quality meshes.
  • Neural field reconstruction and rendering: NeRF / Instant‑NGP, Gaussian Splatting, and numerous improved models encode scenes in neural networks or Gaussian clouds, achieving high-fidelity novel view synthesis and 3D scene reconstruction, progressively forming engineered products. The industry has also seen 3D AI services targeting developers and content production, such as "Hunyuan 3D" and "Tripo," which package NeRF/Gaussian technologies into cloud APIs or interactive tools.

Starting from this layer, traditional geometry, deep learning, implicit representations, and explicit meshes are closely intertwined — the goal is both to solve "how to accurately reconstruct the real world" and to balance real-time performance and usability, serving higher-level 3D scene understanding, 3D generation, and editing.

3.1.1 Point Cloud Processing and 3D Object Detection

For autonomous driving, robotics, and high-precision surveying, LiDAR point clouds are one of the most critical forms of 3D sensing information. A point cloud is a sparse set of points composed of 3D coordinates (sometimes accompanied by reflection intensity, timestamps, etc.), lacking a regular grid structure, which poses challenges for traditional convolution. The goal of point cloud processing is to extract useful geometric and semantic information from these unstructured points — for example, "this is a car," "this is a curb/ground," "this is a building."

In point cloud classification and segmentation tasks, we typically focus on: which category a given point (or point cluster) belongs to, such as car, pedestrian, ground, curb, building, vegetation, etc., or performing semantic/instance segmentation on the scene. From a modeling perspective, approaches can be roughly divided into three categories:

1. Direct point cloud networks: PointNet / PointNet++, PointCNN, DGCNN, etc. define permutation-invariant operations directly on point sets, building hierarchical features through local neighborhood aggregation, suitable for classification and segmentation of small-to-medium-scale point clouds.
2. Voxelization and sparse convolution: rasterizing point clouds into 3D voxels, then using sparse 3D CNNs (such as VoxelNet, MinkowskiNet) for convolution, balancing structural regularity with spatial sparsity, widely used in autonomous driving 3D detection.
3. Projection and multi-view: projecting point clouds into BEV (Bird's Eye View), front-view depth maps, or multi-view perspectives, then using 2D CNNs to extract features, relatively easy to combine with mature 2D detection networks.

In 3D object detection, the goal is no longer simply labeling points, but predicting 3D bounding boxes (position, size, orientation) and their categories — this is the core of autonomous driving environment perception. Typical methods such as VoxelNet, SECOND, PointPillars, and CenterPoint usually convert point clouds into voxel or pillar representations and perform detection regression in BEV or 3D space. Methods like CenterPoint use a "center point detection" paradigm, directly detecting object centers and their sizes/orientations on BEV, balancing accuracy and speed. As deep learning and sensor hardware evolve, 3D detection can now achieve real-time inference on automotive-grade chips, becoming one of the foundational modules of the autonomous driving perception stack.

3.1.2 Multi-View Geometry and 3D Reconstruction: From Photos to Mesh

Without LiDAR, can we still "understand" 3D? The answer is yes — multi-view geometry and 3D reconstruction rely on "multiple photos + camera motion." By capturing the same scene from different viewpoints, we can use geometric constraints to recover camera poses and spatial structure — this is the classic SfM/MVS pipeline.

SfM (Structure‑from‑Motion) primarily solves two problems:

1. Estimating the camera extrinsic parameters (position and orientation) for each image from multiple pairwise or multi-view images;
2. Recovering a set of sparse 3D feature points in a unified coordinate system.

Typical tools such as COLMAP and OpenMVG, through feature extraction and matching (SIFT/ORB, etc.) and incremental or global BA (Bundle Adjustment), can automatically recover sparse point clouds and camera poses from uncalibrated image collections. Building on this, MVS (Multi‑View Stereo) uses multi-view photometric consistency to generate dense point clouds: performing depth estimation for each pixel/ray, gradually filling in the geometric details of the scene.

After obtaining a dense point cloud, the next step is mesh reconstruction:

• Through Poisson Surface Reconstruction, Marching Cubes, or learning-based methods, the scattered point cloud is "wrapped" into a continuous surface, forming a mesh with topological structure.
• This is typically followed by hole filling, smoothing, boundary optimization, and texture mapping, yielding a 3D model that can be directly used for rendering and editing.

In terms of product form, this entire pipeline has been delivered through desktop software, cloud services, and SDKs. For example: 3D scanning apps on phones call SfM/MVS-like processes in the background, automatically outputting a mesh model that can be imported into game engines after the user "circles around and takes photos" or "scans a video"; digital twin platforms run large-scale reconstruction at the city/campus scale using aerial imagery + street view data to generate interactive 3D scenes.

3.1.3 Neural Radiance Fields and Volume Rendering: NeRF, Gaussian, and Next-Generation 3D Reconstruction

Traditional SfM/MVS/mesh reconstruction can produce well-structured explicit geometry, but still has limitations in rendering quality, viewpoint continuity, and detail representation; neural radiance fields (NeRF) and its follow-up work have redefined 3D reconstruction and novel view synthesis through implicit fields + volume rendering.

In NeRF, the entire 3D scene is modeled as a continuous function:

$$ F_\theta(\mathbf{x}, \mathbf{d}) = (\sigma, \mathbf{c}) $$

where $\mathbf{x}$ represents the position of a point in 3D space, $\mathbf{d}$ represents the viewing direction, $\sigma$ represents volume density, $\mathbf{c}$ represents color, and $\theta$ represents network parameters.

Given a point position x and viewing direction d in 3D space, the network outputs the corresponding volume density σ and color c at that point. By performing volume rendering integration along the camera ray direction on this mapping function, we can obtain the pixel color for that camera pose; conversely, given a set of multi-view photos and their camera parameters, we can solve for the model parameters θ by minimizing the error between rendered results and real images. Once the model is trained, simply changing the camera pose allows synthesizing novel view images that were "never actually photographed" (Novel View Synthesis).

Traditional NeRF has relatively slow training and rendering speeds; subsequent work such as Instant‑NGP uses techniques like multi-resolution hash grid encoding to dramatically accelerate convergence and inference; Gaussian Splatting replaces scene representation with 3D Gaussian particles, achieving high-quality, real-time novel view rendering through efficient rasterization strategies. Meanwhile, a large body of work has extended NeRF/Gaussian with editability, multimodality, and composability, gradually moving from research prototypes to engineering systems.

At the productization level, NeRF/Gaussian technologies have been embedded into various 3D AI products:

• Mobile/PC "multi-view video → 3D scene" tools, whose underlying layers often use neural fields or Gaussian particles for reconstruction and rendering;
• Game/film asset pipelines that use neural fields for rapid scene capture and lighting recovery, then export as Mesh + textures for use in traditional DCC tools;
• 3D AI services launched by major cloud providers and content platforms, such as Tencent's "Hunyuan 3D" and Tripo, typically supporting "multi-view photos/short videos → editable 3D models/scenes," internally combining neural radiance fields, SDF/Gaussian representations, and subsequent explicit reconstruction to package high-quality 3D results into developer-friendly APIs or interactive products.

3.2 3D Scene Understanding & SLAM

If 3D perception and reconstruction answers "what does this world look like," then 3D scene understanding and localization further answers: "Where am I in this world? Which places in this world are traversable, and which are obstacles?" For robot vacuums, AGV robots, drones, AR navigation, and indoor positioning systems, the ability to self-localize, self-map, and autonomously plan paths in a 3D environment is a prerequisite for survival.

This body of work primarily revolves around 3D semantic understanding and SLAM (Simultaneous Localization and Mapping): the former performs semantic segmentation and traversable region identification within reconstructed 3D scenes, while the latter uses visual/IMU/LiDAR and other sensors for camera/robot pose estimation and map construction. In engineering, this layer is typically embedded as SDKs or algorithm modules into robot chassis, drone flight controllers, or mobile AR engines.

• Scenarios
  • Home and service robots: robot vacuums, food delivery/patrol robots build maps in indoor environments, identify room types and obstacles, and automatically plan cleaning or patrol paths.
  • Warehousing and logistics: AGV/AMR robots perform autonomous navigation in warehouses, identifying shelves, aisles, and restricted areas to complete transport and inventory tasks.
  • Drones and outdoor robots: build 3D maps in outdoor environments, avoiding buildings, trees, power lines, and other obstacles to perform inspection, surveying, and security tasks.
  • AR navigation and indoor positioning: phones/AR glasses obtain camera pose through SLAM, overlay navigation arrows, room information, and POIs on semantic maps, enabling immersive guided tours and navigation.
• Principles
  • 3D semantic segmentation and scene understanding: performing semantic segmentation on point cloud or voxel representations to distinguish walls, floors, tables and chairs, shelves, doors and windows, and other structures, while identifying traversable regions and obstacles, providing semantic-layer information for navigation and behavioral decisions.
  • Pose estimation and SLAM: using Visual SLAM (monocular/stereo / RGB‑D) or LiDAR‑SLAM to estimate the 6D pose of the camera/robot from continuous sensor data, handling loop closure detection and map optimization, and fusing multi-source information such as IMU, wheel odometry, and GNSS when necessary to improve robustness.
  • Map construction and navigation: overlaying geometric and semantic information on local/global maps to form 2D/3D/topological/semantic maps, and performing path planning, obstacle avoidance, and task assignment on this basis.
• Models
  • SLAM systems: classic feature-point methods such as the ORB‑SLAM series, direct methods such as DSO, and inertial-fused systems such as VINS‑Mono / VINS‑Fusion, achieving precise pose estimation and dense/semi-dense maps through frontend feature tracking + backend optimization. In LiDAR/visual‑LiDAR fusion, frameworks such as LIO‑SAM are common.
  • 3D semantic segmentation networks: 3D U‑Net, MinkowskiNet, and other 3D CNNs, as well as point-cloud-based PointNet++ / KPConv / SparseConv series for semantic and instance segmentation of point clouds/voxels.
  • Multi-sensor fusion localization: methods based on graph optimization or filtering (EKF/UKF) that fuse visual, IMU, LiDAR, odometry, and other multi-source information in a unified state space, improving localization stability in harsh lighting, texture-poor, or dynamic environments.

Overall, 3D scene understanding and localization form the foundation for robots to "get moving": they must both build a reliable self-localization framework in complex 3D worlds and make maps "meaningful," thereby supporting high-level task planning and human-robot interaction.

3.2.1 3D Semantic Segmentation and Traversable Region Understanding

In a purely geometric map, all structures are just undifferentiated points/voxels; in real applications, what we care about is: where is the ground, where are the walls, where are tables or shelves, and where is traversable. 3D semantic segmentation aims to assign a semantic label to every point or voxel, transforming "pure geometry" into "geometry + semantics."

In indoor/outdoor scenes, typical targets include:

• Fixed structures: walls, floors, ceilings, stairs, columns, roads, curbs, etc.;
• Furniture and fixtures: tables and chairs, cabinets, shelves, doors and windows, handrails, etc.;
• Traversable/non-traversable regions: robot-walkable areas, obstacles to circumvent, restricted zones, etc.

In terms of modeling, 3D semantic segmentation commonly uses:

• Voxel/sparse convolution approaches: after voxelizing point clouds, using sparse CNNs such as 3D U‑Net and MinkowskiNet to learn voxel-level features, balancing local details and global structure.
• Direct point cloud approaches: PointNet++, KPConv, and other point cloud networks that perform feature aggregation on local neighborhoods for point-level semantic prediction.

In applications such as robot vacuums and AGV robots, semantic segmentation results are further abstracted into semantic maps: for example, dividing rooms into bedroom/living room/kitchen, and dividing warehouse spaces into shelf areas/aisles/restricted zones. Robots not only know "where they can go" but can also tailor different strategies based on room type (e.g., avoiding carpeted areas in bedrooms, prioritizing certain shelf zones in warehouses).

3.2.2 Pose Estimation, SLAM, and Multi-Sensor Fusion Localization

The goal of SLAM (Simultaneous Localization and Mapping) is: in an unknown environment, to estimate one's own trajectory while moving and simultaneously build a map of the environment. For indoor environments without high-precision external positioning (such as RTK‑GNSS), SLAM is the preferred solution for the vast majority of robots and AR engines.

In visual SLAM, methods represented by ORB‑SLAM, DSO, and VINS‑Mono/VINS‑Fusion are typically divided into several key modules:

• Frontend: extracting and tracking keypoints/image patches from consecutive images, estimating relative pose between adjacent frames.
• Backend: performing BA or graph optimization within a sliding window or global graph, handling drift, loop closure detection, and relocalization.
• Mapping: constructing dense or semi-dense maps based on pose and depth information, providing the foundation for subsequent navigation or rendering.

Pure vision tends to fail in texture-poor areas or under drastic lighting changes, so in practice multi-sensor fusion localization is generally adopted:

• Visual + IMU: frameworks such as VINS‑Mono/VINS‑Fusion combine the high-frequency short-term accuracy of IMU with the scale and geometric constraints of vision, significantly improving stability in short-term and sharp-turn scenarios.
• LiDAR + IMU + vision: odometry frameworks such as LIO‑SAM introduce inertial navigation and optional visual information into LiDAR‑SLAM, leveraging the complementary nature of the three to achieve robust localization, widely used in autonomous driving and high-precision surveying.

At the product level, these methods are typically encapsulated as part of robot chassis controllers, drone flight controllers, AR engines (such as Visual‑Inertial SLAM in ARKit/ARCore), or indoor positioning SDKs, shielding upper-layer applications from complex state estimation and graph optimization logic, allowing developers to directly obtain "real-time pose + map."

3.2.3 Semantic Maps, Navigation, and Obstacle Avoidance

With stable pose estimation and geometric/semantic maps in place, the next step is to make the robot "move intelligently." This part primarily involves semantic map construction, path planning, and obstacle avoidance.

• Semantic map construction: overlaying semantic information (room types, POIs, region labels) on geometric maps to form map representations suitable for high-level decision-making. For example:
  • In home scenarios, dividing the map into areas such as bedrooms, living room, kitchen, and bathrooms;
  • In warehouse scenarios, labeling shelf locations, loading/unloading zones, hazardous areas, etc.;
  • In large malls/exhibition halls, labeling stores, service desks, restrooms, and other POIs for AR navigation and guided tours.
• Path planning and obstacle avoidance: building grid maps or topological maps, using planning algorithms such as A*, D* Lite, and RRT to find feasible paths from start to goal for the robot; simultaneously combining real-time perception (obstacles ahead, dynamic pedestrians/vehicles) for local replanning and obstacle avoidance, ensuring operational safety and efficiency.
• Navigation behavior and task scheduling: in AGV robots and drones, task scheduling and multi-robot coordination modules are layered on top of navigation: assigning tasks, avoiding congestion, and optimizing overall paths and energy consumption.

AR navigation and indoor positioning systems also essentially rely on similar semantic maps and path planning, except the "executor" changes from a robot to a person: the system obtains the user's device pose through SLAM, plans a walking path on the semantic map, and then visualizes the path as an augmented reality overlay on the real-world view.

3.3 3D Generation & Editing

If 3D perception and SLAM are about "capturing and understanding" geometry from the real world, then 3D generation and editing take the perspective of content production: how to use AI to automatically produce and modify 3D assets. This directly addresses the enormous content demands of gaming, film, digital humans, virtual spaces, e-commerce displays, 3D printing, and more.

In the past two to three years, with breakthroughs in technologies such as NeRF/Gaussian, SDF representations, and multimodal diffusion models, 3D generation has entered a period of rapid development: one-click generation of 3D models or scenes from text, images, and video has become a reality, and major cloud providers and startup teams have launched products such as "Hunyuan 3D," Tripo, and the DreamFusion / Magic3D series of methods, implemented as online tools, gradually moving 3D production toward "accessible to everyone." 3D generation and editing can be roughly divided into four capabilities: text-to-3D, image/video-to-3D, model optimization and editing, and rigging and animation.

• Scenarios
  • Game / film asset production: rapidly generating usable 3D models for characters, props, buildings, and scenes, significantly reducing art workload.
  • E-commerce and product display: automatically generating 3D display models from product copy or photos for 3D preview, AR placement, and interactive advertising.
  • Digital humans and virtual content: rapidly generating 3D assets such as virtual humans, virtual fitting models, and virtual streamer scenes, supporting live streaming, short videos, and interactive applications.
  • 3D printing and personalized modeling: generating printable models from sketches/photos/text, enabling personalized gifts, prototype design, and educational scenario applications.
• Principles
  • Text‑to‑3D: encoding text descriptions as semantic vectors, then generating 3D representations (NeRF/SDF/Gaussian/Mesh) through multi-stage optimization or diffusion processes, often leveraging powerful 2D text-to-image models as "scorers" or priors.
  • Image / video‑to‑3D: using single or multiple images, or multi-view video as supervision, combined with NeRF, SDF, or implicit/explicit hybrid representations, to reconstruct 3D models with geometry and textures.
  • 3D model optimization and editing: performing retopology, simplification, detail enhancement, LOD generation, UV unwrapping, and texture generation on existing models, as well as language/image-based deformation and stylization.
  • Rigging and animation: automatically inferring skeletal structures for 3D characters and completing rigging, supporting skeletal animation and physics simulation (cloth, soft bodies, rigid bodies), forming drivable dynamic assets.
• Models
  • Foundational 3D generation representations: NeRF / Instant‑NGP, SDF (implicit surfaces), Gaussian Splatting, and mesh-based generation networks, constituting the expression space for 3D data.
  • Text‑to‑3D methods: typical approaches such as DreamFusion, Magic3D, and Fantasia3D, completing end-to-end generation from text to 3D through "2D text-to-image model + 3D optimization" or "3D diffusion models," laying the technical foundation for later products such as Hunyuan 3D and Tripo.
  • Image/video‑to‑3D models: reconstruction and optimization frameworks based on NeRF/SDF/Gaussian, recovering stable 3D geometry and textures from multi-view consistency and single-view priors.
  • Rigging and animation algorithms: automatic skeleton extraction, skeletal weight prediction, deep-learning-based retargeting and motion generation, providing one-click tools for virtual human/character animation.

In this layer, traditional 3D DCC (Maya/Blender/3ds Max, etc.) and AI toolchains are gradually merging: many 3D AI services are embedded into existing production workflows as plugins or cloud interfaces, allowing modelers/artists to rapidly iterate assets through human-AI collaboration.

3.3.1 Text‑to‑3D and Scene Rough Models

The goal of Text‑to‑3D is: given a natural language description, such as "a cartoon-style yellow duck toy with a blue scarf, suitable for children's toy display," the system automatically generates an editable 3D model (Mesh/NeRF/SDF/Gaussian, etc.). This is a classic application of combining large language models / multimodal models with 3D representations.

Typical technical paths include:

1. Optimization based on 2D text-to-image models (e.g., DreamFusion, Magic3D):
2. Using powerful Text‑to‑Image models (such as diffusion models) as "evaluators," given an image rendered from a 3D representation at a certain viewpoint, assessing how well it matches the text description.
3. Through gradient optimization or diffusion processes, iteratively adjusting the 3D representation (NeRF/SDF/Mesh) so that images rendered from multiple viewpoints all conform to the text semantics.
4. 3D diffusion models / direct generation:
5. Treating 3D data (point clouds, voxels, implicit field parameters, Gaussian particles, etc.) as the generation target of diffusion models, pretrained on large-scale 3D datasets;
6. Achieving end-to-end Text‑to‑3D sampling through text-conditioned control.

At the scene level, scene rough model capabilities allow users to describe spatial layouts using natural language or rough sketches — for example, "a living room with floor-to-ceiling windows, an L-shaped sofa on the left, a coffee table in the middle, and bookshelves and a TV stand on the right" — and the system automatically builds a geometrically and semantically reasonable 3D layout sketch. Subsequently, models and materials can be refined in DCC tools, or usable scene prototypes can be quickly produced directly through the "scene generation" capabilities in tools like Hunyuan 3D and Tripo.

Currently, multiple platforms have launched Text‑to‑3D products for designers and developers:

• "Hunyuan 3D" and similar platforms integrate text-to-3D, multi-view generation, and reconstruction capabilities into a unified interface, supporting rapid generation of characters, props, and scenes from text, then exporting to game engines;
• Tripo-class products emphasize "multimodal input + one-click 3D output," supporting mixed simple text and reference images to guide the generation of 3D assets that meet style and structure requirements.

3.3.2 Image / Video‑to‑3D and Model Optimization & Editing

Compared to pure text, generating 3D models from images or video provides stronger geometric constraints and better visual consistency. Therefore, a large number of 3D AI products support image-to-3D / video-to-3D:

• Single photo → rough 3D: using single-view priors (such as shape priors for faces, human bodies, and common object categories) to infer approximate 3D geometry, generating 3D models suitable for preview or simple interaction.
• Multiple photos / short video → high-quality 3D: comprehensively using NeRF/SDF/Gaussian reconstruction, multi-view geometry, and post-processing to convert dozens of photos or a few seconds of video into high-fidelity 3D models, suitable for game/film assets or high-quality e-commerce displays.

Generating 3D geometry is only the first step; substantial model optimization and editing work is still needed afterward:

• Retopology and simplification: converting implicit fields or high-polygon meshes into structurally regular, face-count-controllable topology for rigging, animation, and real-time rendering.
• LOD generation: automatically generating multi-level detail models (Level of Detail), using low-poly models at a distance and high-poly models up close, balancing image quality and performance.
• UV unwrapping and texture generation: automatically unwrapping UVs for models, generating or optimizing normal maps, displacement maps, roughness/metallic maps, and other PBR materials; some models also support automatically generating stylized textures from text or reference images.
• Geometry and style editing: making localized modifications based on language or example images, such as "make this chair's legs a bit shorter" or "change this building to a cyberpunk style," typically implemented through shape latent space manipulation or neural field editing.

Products such as Hunyuan 3D and Tripo often connect the above workflows end-to-end: users start from photos/videos or simple text, and the system internally completes reconstruction, retopology, texturing, and export, allowing non-professional users to obtain "plug-and-play" 3D models within minutes, dramatically shortening the time from concept to asset.

3.3.3 Rigging, Animation, and Dynamic 3D Assets

Static models are only half the content; 3D assets that "can move" are more critical in gaming, film, virtual humans, and interactive applications. This involves skeletal rigging, weight painting, animation, and physics simulation — traditionally high-barrier professional tasks that are now increasingly assisted or even semi-automated by AI tools.

• Automatic rigging: given a character mesh, the system automatically infers the skeletal hierarchy (spine, limbs, fingers, etc.) and the positions of bones within the model, and predicts the weight of each vertex relative to each bone. Recent deep learning methods can learn this mapping on large-scale rigged character datasets, achieving one-click skeletal rigging.
• Animation and motion generation: overlaying motion data (mocap or AI-generated) on existing skeletons to produce walking, running, facial expressions, gestures, and other animations; deep-learning-based motion generation and retargeting can transfer human motions from video or motions from other characters onto new characters.
• Physics simulation: performing physics simulation on cloth, soft bodies, rigid bodies, etc., making the movement of hair, clothing, flags, and soft objects more natural. Some systems use neural networks to accelerate or approximate physics, making physical effects in real-time engines more realistic.

In terms of products and ecosystems, these capabilities are often embedded in:

• Game / film asset toolchains: providing modelers with one-click rigging, automatic weight assignment, and basic motion libraries, significantly reducing repetitive labor;
• Virtual human / digital asset creation platforms: starting from character photos or scans, going through 3D reconstruction + automatic rigging + motion driving to output virtual humans that can be driven in live streaming, short videos, and interactive applications;
• 3D AI platforms (such as Hunyuan 3D, Tripo, and similar products): adding rigging and simple animation capabilities on top of 3D generation, so that "generated characters can immediately start moving" without requiring complex DCC tool operations.

As 3D generation and editing technologies mature, the entire 3D content production pipeline is evolving from "centered on professional DCC tools" to "AI-driven human-AI collaboration": AI handles generation and extensive foundational work, while humans make more decisions on style definition, quality control, and key design nodes. Hunyuan 3D, Tripo, and other next-generation 3D AI products are a concentrated manifestation of this trend, providing faster and more accessible 3D infrastructure for upstream gaming, film, AR/VR, digital twin, and virtual human applications.