Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
Paper
•
2412.13389
•
Published
•
6
Photogrammetry and Remote Sensing Lab of ETH Zurich
university
AI & ML interests
None defined yet.
Recent Activity
View all activity
prs-eth's activity
toshas
authored
a
paper
6 days ago
toshas
updated
a
Space
7 days ago
Post
1161
Introducing ⇆ Marigold-DC — our training-free zero-shot approach to monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you!
Depth Completion addresses sparse, incomplete, or noisy measurements from photogrammetry or sensors like LiDAR. Sparse points aren’t just hard for humans to interpret — they also hinder downstream tasks.
Traditionally, depth completion was framed as image-guided depth interpolation. We leverage Marigold, a diffusion-based monodepth model, to reframe it as sparse-depth-guided depth generation. How the turntables! Check out the paper anyway 👇
🌎 Website: https://marigolddepthcompletion.github.io/
🤗 Demo: prs-eth/marigold-dc
📕 Paper: https://arxiv.org/abs/2412.13389
👾 Code: https://github.com/prs-eth/marigold-dc
Team ETH Zürich: Massimiliano Viola ( @mviola ), Kevin Qu ( @KevinQu7 ), Nando Metzger ( @nandometzger ), Bingxin Ke ( @Bingxin ), Alexander Becker, Konrad Schindler, and Anton Obukhov ( @toshas ). We thank
Hugging Face for their continuous support.
Depth Completion addresses sparse, incomplete, or noisy measurements from photogrammetry or sensors like LiDAR. Sparse points aren’t just hard for humans to interpret — they also hinder downstream tasks.
Traditionally, depth completion was framed as image-guided depth interpolation. We leverage Marigold, a diffusion-based monodepth model, to reframe it as sparse-depth-guided depth generation. How the turntables! Check out the paper anyway 👇
🌎 Website: https://marigolddepthcompletion.github.io/
🤗 Demo: prs-eth/marigold-dc
📕 Paper: https://arxiv.org/abs/2412.13389
👾 Code: https://github.com/prs-eth/marigold-dc
Team ETH Zürich: Massimiliano Viola ( @mviola ), Kevin Qu ( @KevinQu7 ), Nando Metzger ( @nandometzger ), Bingxin Ke ( @Bingxin ), Alexander Becker, Konrad Schindler, and Anton Obukhov ( @toshas ). We thank
Hugging Face for their continuous support.
ghjuliasialelli
updated
a
dataset
22 days ago
PeterTor
updated
a
dataset
23 days ago
toshas
authored
a
paper
24 days ago
PeterTor
authored
a
paper
24 days ago
Bingxin
authored
a
paper
24 days ago
toshas
authored
a
paper
5 months ago
nandometzger
authored
a
paper
5 months ago
Bingxin
authored
a
paper
5 months ago
toshas
authored
a
paper
5 months ago
toshas
authored
2
papers
6 months ago
Post
991
Join us at our remaining CVPR presentations this week! Members of PRS-ETH will be around to connect with you and discuss our presented and ongoing works:
💐 Marigold: Discover our work on sharp diffusion-based computer vision techniques, presented in Orals 3A track on "3D from Single View", Thu, June 20, 9:00-9:15 AM. Also, drop by Poster Session 3 later that day for more tangible matters! 🌚
Project page: https://marigoldmonodepth.github.io/
Paper: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (2312.02145)
Collection: https://huggingface.co./collections/prs-eth/marigold-6669e9e3d3ee30f48214b9ba
Space: prs-eth/marigold-lcm
Diffusers 🧨 tutorial: https://huggingface.co./docs/diffusers/using-diffusers/marigold_usage
⚙️ Point2CAD: Learn about our mechanical CAD model reconstruction from point clouds, presented in Poster Session 1, Wed, June 19, 10:30 AM - 12:00 PM.
Project page: https://www.obukhov.ai/point2cad.html
Paper: Point2CAD: Reverse Engineering CAD Models from 3D Point Clouds (2312.04962)
🎭 DGInStyle: Explore our generative data synthesis approach as a cost-efficient alternative to real and synthetic data, presented in the Workshop on Synthetic Data for Computer Vision, Tue, June 18, at Summit 423-425.
Details and schedule: https://syndata4cv.github.io/
Project page: https://dginstyle.github.io/
Paper: DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control (2312.03048)
Model: yurujaja/DGInStyle
💐 Marigold: Discover our work on sharp diffusion-based computer vision techniques, presented in Orals 3A track on "3D from Single View", Thu, June 20, 9:00-9:15 AM. Also, drop by Poster Session 3 later that day for more tangible matters! 🌚
Project page: https://marigoldmonodepth.github.io/
Paper: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (2312.02145)
Collection: https://huggingface.co./collections/prs-eth/marigold-6669e9e3d3ee30f48214b9ba
Space: prs-eth/marigold-lcm
Diffusers 🧨 tutorial: https://huggingface.co./docs/diffusers/using-diffusers/marigold_usage
⚙️ Point2CAD: Learn about our mechanical CAD model reconstruction from point clouds, presented in Poster Session 1, Wed, June 19, 10:30 AM - 12:00 PM.
Project page: https://www.obukhov.ai/point2cad.html
Paper: Point2CAD: Reverse Engineering CAD Models from 3D Point Clouds (2312.04962)
🎭 DGInStyle: Explore our generative data synthesis approach as a cost-efficient alternative to real and synthetic data, presented in the Workshop on Synthetic Data for Computer Vision, Tue, June 18, at Summit 423-425.
Details and schedule: https://syndata4cv.github.io/
Project page: https://dginstyle.github.io/
Paper: DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control (2312.03048)
Model: yurujaja/DGInStyle
Post
1984
Another gem from our lab — DGInStyle! We use Stable Diffusion to generate semantic segmentation data for autonomous driving and train domain-generalizable networks.
📟 Website: https://dginstyle.github.io
🧾 Paper: https://arxiv.org/abs/2312.03048
🤗 Hugging Face Paper: DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control (2312.03048)
🤗 Hugging Face Model: yurujaja/DGInStyle
🐙 Code: https://github.com/yurujaja/DGInStyle
In a nutshell, our pipeline overcomes the resolution loss of Stable Diffusion latent space and the style bias of ControlNet, as shown in the attached figures. This allows us to generate sufficiently high-quality pairs of images and semantic masks to train domain-generalizable semantic segmentation networks.
Team: Yuru Jia ( @yurujaja ), Lukas Hoyer, Shengyu Huang, Tianfu Wang ( @Tianfwang ), Luc Van Gool, Konrad Schindler, and Anton Obukhov ( @toshas ).
📟 Website: https://dginstyle.github.io
🧾 Paper: https://arxiv.org/abs/2312.03048
🤗 Hugging Face Paper: DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control (2312.03048)
🤗 Hugging Face Model: yurujaja/DGInStyle
🐙 Code: https://github.com/yurujaja/DGInStyle
In a nutshell, our pipeline overcomes the resolution loss of Stable Diffusion latent space and the style bias of ControlNet, as shown in the attached figures. This allows us to generate sufficiently high-quality pairs of images and semantic masks to train domain-generalizable semantic segmentation networks.
Team: Yuru Jia ( @yurujaja ), Lukas Hoyer, Shengyu Huang, Tianfu Wang ( @Tianfwang ), Luc Van Gool, Konrad Schindler, and Anton Obukhov ( @toshas ).
bobboli
authored
a
paper
9 months ago
toshas
authored
3
papers
9 months ago
DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
Paper
•
2211.12131
•
Published
EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation
Paper
•
2304.14291
•
Published
DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control
Paper
•
2312.03048
•
Published
•
1