The creation of 3D scenes has traditionally been both labor-intensive and costly, requiring designers to meticulously configure 3D assets and environments. Recent advancements in generative AI, including text-to-3D and image-to-3D methods, have dramatically reduced the complexity and cost of this process. However, current techniques for editing complex 3D scenes continue to rely on generally interactive multi-step, 2D-to-3D projection methods and diffusion-based techniques, which often lack precision in control and hamper real-time performance. In this work, we propose 3DSceneEditor, a fully 3D-based paradigm for real-time, precise editing of intricate 3D scenes using Gaussian Splatting. Unlike conventional methods, 3DSceneEditor operates through a streamlined 3D pipeline, enabling direct manipulation of Gaussians for efficient, high-quality edits based on input prompts.The proposed framework (i) integrates a pre-trained instance segmentation model for semantic labeling; (ii) employs a zero-shot grounding approach with CLIP to align target objects with user prompts; and (iii) applies scene modifications, such as object addition, repositioning, recoloring, replacing, and deletion directly on Gaussians. Extensive experimental results show that 3DSceneEditor achieves superior editing precision and speed with respect to current SOTA 3D scene editing approaches, establishing a new benchmark for efficient and interactive 3D scene customization.
3D场景的创建传统上既耗时又昂贵，需要设计者精心配置3D资产和环境。近年来，生成式人工智能的进步（如文本到3D和图像到3D方法）显著降低了这一过程的复杂性和成本。然而，当前用于编辑复杂3D场景的技术依然依赖于交互性强、多步骤的2D到3D投影方法和扩散模型技术，这些方法往往缺乏精确控制并且阻碍了实时性能的实现。 在本研究中，我们提出了3DSceneEditor，一种基于完全3D的实时精确编辑复杂3D场景的范式，利用高斯散点（Gaussian Splatting）实现。与传统方法不同，3DSceneEditor通过精简的3D管道运行，允许直接操控高斯，以高效、高质量地根据输入提示进行编辑。 所提出的框架具有以下特点：(i) 集成了预训练的实例分割模型以进行语义标注；(ii) 使用基于CLIP的零样本（zero-shot）定位方法，将目标对象与用户提示对齐；(iii) 支持直接在高斯上进行场景修改，如对象添加、重新定位、重新着色、替换和删除。 大量实验结果表明，3DSceneEditor在编辑精度和速度方面显著优于当前最先进的3D场景编辑方法，为高效、交互式3D场景定制设立了新的标杆。