Member Avatar for vs.vaidyanathan

can anyone please help me with converting from image to world coordinates? Once i get the world coordinates how do i make use of the Z coordinate to show the 3d? thanks in advance:)

  • 4 Contributors
  • 7 Replies
  • 307 Views
  • 4 Days Discussion Span
  • Latest Post Latest Post by vs.vaidyanathan

Dani AI

Generated

Short, practical roadmap (builds on ’s setup and the hints from and ):

Start by undoing lens distortion and converting pixel coordinates to normalized camera coordinates with K^{-1}[u v 1]^T. With known intrinsics the fundamental matrix maps to the essential matrix; decompose the essential matrix (SVD) to get four candidate (R,t) solutions. Choose the correct solution by triangulating a handful of correspondences and enforcing cheirality (points must have positive depth in both camera frames). Note: the overall reconstruction is determined up to a global scale unless a metric baseline or a known object size is provided.

Triangulation and a simple OpenCV workflow:

  • Build projection matrices P1 = K [I | 0] and P2 = K [R | t].
  • Triangulate each matching pair (DLT or cv::triangulatePoints) to get homogeneous 4D points, then divide by w to get Euclidean (X,Y,Z).
  • Check reprojection error by projecting X back with P1 and P2; discard bad matches.
// K1,K2 (3x3), R (3x3), t (3x1), pts1/pts2 as cv::Mat 2xN (CV_64F)
cv::Mat P1 = K1 * cv::Mat::eye(3,4, CV_64F);
cv::Mat RT; cv::hconcat(R, t, RT);
cv::Mat P2 = K2 * RT;
cv::Mat pts4D;
cv::triangulatePoints(P1, P2, pts1, pts2, pts4D); // 4xN
// convert to 3D
std::vector<cv::Point3d> X;
for (int i=0;i<pts4D.cols;i++){
  cv::Mat x = pts4D.col(i); x /= x.at<double>(3);
  X.emplace_back(x.at<double>(0), x.at<double>(1), x.at<double>(2));
}

Rendering / using Z

  • For real 3D display, feed the (X,Y,Z) cloud to a 3D renderer (OpenGL/DirectX) and set a viewing camera (gluLookAt / equivalent). Z is the camera-space depth: positive Z typically means “in front”.
  • For 2D outputs that look 3D, use Z for depth sorting, point size, and shading or produce a depth map (grayscale or colored by distance).

Troubleshooting notes: undistort first, use RANSAC when estimating F/E, discard triangulated points with negative depth, check reprojection error, and refine everything with bundle adjustment to reduce bias from noisy matches.

Recommended Answers

All 7 Replies

Member Avatar for Member #46692

You need to give more details than that. What exactly are you using?

Member Avatar for vs.vaidyanathan

i am using Visual C++. its for 3-D reconstruction from multiple views. I have image coordinates of the same point from two cameras. Their calibrations are known. So i should be able to get the world coordinates. having got that, how do i implement the z coordinate in my output image? which should look 3-d..

Member Avatar for amrith92

i am using Visual C++. its for 3-D reconstruction from multiple views. I have image coordinates of the same point from two cameras. Their calibrations are known. So i should be able to get the world coordinates. having got that, how do i implement the z coordinate in my output image? which should look 3-d..

An image has to be described on a 2D plane, so the z-coordinate does not come into play here. If you just want stills of your 3D simulation, maybe you should just try writing the pixel information of the current sub-window(or the simulation-containing window) that contains the 3D simulation? This would result in a 2D image that would be something of a screen capture of your 3D simulation. I am thinking along the lines of what the CIMG Image library provides, which allows us to do this quite easily...

Member Avatar for Lerner

I find terminology used to describe 3D coding tend to be confusing---just like any new language when I start using it. To me, a point has no 3D representation visually. However, multiple points making up a single image can be displayed simulating 3D on a 2D surface. Then the "Z" component can come into play, assuming you don't want to visualize "hidden" points/surfaces, etc. If this is what you want to do, then I strongly suggest you pick up a text book or look at some of the 3D tutorials you can find on the net as it's not something likely to be readily explained in a post to a bulletin board. At least I've never seen a brief description of it without all the necessary background information needed to understand it.

Member Avatar for vs.vaidyanathan

Thanks amritha and lerner...
I have come as far as estimating fundamental matrices for the cameras. I am lost as to which direction I should proceed next. help please?:?:

Member Avatar for amrith92

> I have come as far as estimating fundamental matrices for the cameras.

You have mentioned in an earlier post that you know the camera's calibration. When you talk about a fundamental matrix under a known calibration, it is called an essential matrix.

This web page, which is a part of the Gandalf(computer vision lib) project, and gives the algorithm\formulae you may require to proceed with your project.

Member Avatar for vs.vaidyanathan

thanks folks... all of your advice was helpful:)

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.