Useful code snippets for CVB 3D image processing

Intrinsically & extrinsically calibrate point clouds using a laser triangulation sensor

The following describes an example workflow in CVB.Net to:

  • reconstruct an intrinsically calibrated point cloud from a range map and an intrinsic calibration file
  • extrinsically calibrate a system using an AQS12 calibration target
  • create a rectified range map from the calibrated point cloud
using Stemmer.Cvb;
using Stemmer.Cvb.Foundation;

namespace cvbNet_ExtrinsicCalibration_Console

  class Program
    static void Main(string[] args)
      // load range map
      var rangemap = Image.FromFile(@"%CVB%\Tutorial\Metric\Images\RangeMap_CalibrationPattern.tif");

      // load intrinsic calibration (e.g. from an AT compact sensor)
      var calibrator = Calibrator3D.FromFile(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\Test.xml"));

      // reconstruct point cloud without extrinsic calibration
      PointCloud pointCloud = calibrator.CreatePointCloud(rangemap, PointCloudFlags.Double);

      // set pattern points
      AQS12Piece aqs12 = new AQS12Piece();
      aqs12.TopBasePlaneDistance = 60;
      aqs12.ReferencePoints[0] = new Point3Dd(0, 40, 60);
      aqs12.ReferencePoints[1] = new Point3Dd(10, 53, 60);
      aqs12.ReferencePoints[2] = new Point3Dd(10, 96, 60);
      aqs12.ReferencePoints[3] = new Point3Dd(0, 110, 60);
      aqs12.ReferencePoints[4] = new Point3Dd(-10, 96, 60);
      aqs12.ReferencePoints[5] = new Point3Dd(-10, 53, 60);
      aqs12.ReferencePoints[6] = new Point3Dd(0, 0, 30);
      aqs12.ReferencePoints[7] = new Point3Dd(30, 40, 30);
      aqs12.ReferencePoints[8] = new Point3Dd(30, 110, 30);
      aqs12.ReferencePoints[9] = new Point3Dd(0, 150, 30);
      aqs12.ReferencePoints[10] = new Point3Dd(-30, 110, 30);
      aqs12.ReferencePoints[11] = new Point3Dd(-30, 40, 30);
      // perform calibration
      Point3Dd[] residuals = new Point3Dd[12];
      calibrator.ExtrinsicMatrix = Metric.ExtrinsicTransformationFromPiece(rangemap.Planes[0], aqs12, calibrator, out residuals);
      // reconstruct point cloud with extrinsic calibration
      PointCloud calibratedPointCloud = calibrator.CreatePointCloud(rangemap, PointCloudFlags.Double);
      //create rectified range map 
      //(parameters for rectification can be set differently)
      var boundingBox = calibratedPointCloud.CalculateBoundingBox();
      ValueRange<double> xRange = new ValueRange<double>(double.MinValue, double.MaxValue);
      ValueRange<double> yRange = new ValueRange<double>(double.MinValue, double.MaxValue);
      xRange.Min = boundingBox.X.Min;
      xRange.Max = boundingBox.X.Max;
      yRange.Min = boundingBox.Y.Min;
      yRange.Max = boundingBox.Y.Max;
      int targetWidth = rangemap.Width;
      int targetHeight = rangemap.Height;
      double bgValue = boundingBox.Z.Min - 10;
      var rectifiedRangemap = calibratedPointCloud.ToRangeMap(xRange, yRange, targetWidth, targetHeight, bgValue);

Aligning a point cloud using a plane fit

In many cases it is necessary to transform 3D point clouds in a post processing. It is e.g. much easier to detect surface defects on flat objects, when the surface is aligned to the xy-plane of the coordinate system. Espacially when working with rectified range maps (2.5D images) it gets handy to do these kind of alignments before analyzing.

The following example demonstrates how to fit a plane into a subset of points of a point cloud and how to transform the point cloud with the resulting normal vector of the plane. The result will be a point cloud where the z-axis points into the same direction as the normal vector of the plane/surface.

First, we load an example point cloud:

var pc = PointCloud.FromFile(@"D:\MyPointCloud.ply");

For verification we can display it in an activeX control (Common Vision Blox 3D Viewer):

axCVCore3DViewer1.AddPointCloudVariable(pc.Handle.ToInt64(), 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0);

The sample point cloud is an AQS12 calibration target, already aligned to its base plane. In this example workflow we will fit a plane into one of the side planes.

For the plane fit we define a certain 3D region of interest (cuboid) within the point cloud:

var range = new ValueRange<double>(double.MinValue, double.MaxValue);
      var planeCuboid = new Cuboid(range, range, range);
      planeCuboid.X.Min = 700;
      planeCuboid.X.Max = 860;
      planeCuboid.Y.Min = 1035;
      planeCuboid.Y.Max = 1540;
      planeCuboid.Z.Min = 1080;
      planeCuboid.Z.Max = 1240;
      var plane = pc.FitPlane(planeCuboid);

Again, for verification, we can display the points of the cuboid used for the plane fit in the 3D viewer.

axCVCore3DViewer1.AddPointCloudSolid(pc.Crop(planeCuboid).Handle.ToInt64(), 0xFFFFFF, 1,1,1,2);

For the transformation of the point cloud we need to set up a rotation matrix first with the entries of the normal vector:

var roll = new Angle();
var pitch = new Angle();
var yaw = new Angle();

roll.Rad = Math.Atan(plane.Normal.Y/plane.Normal.Z);
pitch.Rad = Math.Atan(plane.Normal.X/plane.Normal.Z);
yaw.Rad = 0; // not necessary for an alignment into the xy-plane

var rotAngles = new RotationAngles3D(roll, pitch, yaw);
var rotMatrix = new Matrix3D(rotAngles);

Afterwards we can fill the transformation matrix with our calculated rotation matrix and a translation vector, that can be set to (0,0,0) in this case:

var translation = new Point3Dd(0, 0, 0);
var trafoMatrix = new AffineMatrix3D(rotMatrix, translation);

The transformation itself is done with the following method:

var pc_transformed = pc.Transform(trafoMatrix);

The transformed point cloud is now aligned to the fitted plane and can be visualized:

axCVCore3DViewer1.AddPointCloudVariable(pc_transformed.Handle.ToInt64(), 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1);

This is how rectified range maps look like when being calculated from the original point cloud (left) and transformed point cloud (right):