Part Finder

Introduction

What is 6DoF Object Pose Estimation?

When a robot picks up an object, or an augmented reality app overlays a 3D model onto a scene, it needs to know not just what the object is, but exactly where it is and how it is oriented in 3D space. This is the problem of 6DoF (six degrees of freedom) pose estimation: recovering an object's full position (X, Y, Z) and orientation (roll, pitch, yaw) from a camera image.

What This Site Does:

This interface lets you run our state-of-the-art 6DoF pose estimation system on your own objects and scenes, with no training required.

You upload a 3D model of your object (PLY format), an RGB image and depth map of a scene containing a number of instances of the object, and the RGB camera intrinsic parameters. The system will locate each instance of the object in the image and return its precise 6DoF pose.

Under the hood, the method uses a large-scale vision foundation model to extract rich visual features that generalise across the gap between synthetic 3D renderings and real photographs. No task-specific fine-tuning, no labelled data, and no GPU training on your object is required. It works on objects it has never seen before (i.e. zero zhot).

Processing Time:

Onboarding (~5 minutes):Each new part that the system encouters must be preprocessed in an onboarding phase, which generates a data structure based on a textured 3D model. Onboarding is computationally intensive, and takes roughly 5 minutes on consumer hardware. Generating a GPU cache for the onboarded data structure takes an additional 2 minutes.

For this demo, onboarding must be re-run for each submission to conserve disk space on the server.

Inference (~5-10 seconds): Onboarding only needs to be done once per object. Subsequent detections of the same object only take 5-10 seconds per image, depending on the number of parts detected.