Useful code snippets for CVB 3D image processing

Intrinsically & extrinsically calibrate point clouds using a laser triangulation sensor

The following describes an example workflow in CVB.Net to:

  • reconstruct an intrinsically calibrated point cloud from a range map and an intrinsic calibration file
  • extrinsically calibrate a system using an AQS12 calibration target
  • create a rectified range map from the calibrated point cloud
using Stemmer.Cvb;
using Stemmer.Cvb.Foundation;

namespace cvbNet_ExtrinsicCalibration_Console
{

  class Program
  {
    static void Main(string[] args)
    {
      // load range map
      var rangemap = Image.FromFile(@"%CVB%\Tutorial\Metric\Images\RangeMap_CalibrationPattern.tif");

      // load intrinsic calibration (e.g. from an AT compact sensor)
      var calibrator = Calibrator3D.FromFile(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\Test.xml"));

      // reconstruct point cloud without extrinsic calibration
      PointCloud pointCloud = calibrator.CreatePointCloud(rangemap, PointCloudFlags.Double);
      pointCloud.Save(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\PointCloud.ply"));

      // set pattern points
      AQS12Piece aqs12 = new AQS12Piece();
      aqs12.TopBasePlaneDistance = 60;
      aqs12.ReferencePoints[0] = new Point3Dd(0, 40, 60);
      aqs12.ReferencePoints[1] = new Point3Dd(10, 53, 60);
      aqs12.ReferencePoints[2] = new Point3Dd(10, 96, 60);
      aqs12.ReferencePoints[3] = new Point3Dd(0, 110, 60);
      aqs12.ReferencePoints[4] = new Point3Dd(-10, 96, 60);
      aqs12.ReferencePoints[5] = new Point3Dd(-10, 53, 60);
      aqs12.ReferencePoints[6] = new Point3Dd(0, 0, 30);
      aqs12.ReferencePoints[7] = new Point3Dd(30, 40, 30);
      aqs12.ReferencePoints[8] = new Point3Dd(30, 110, 30);
      aqs12.ReferencePoints[9] = new Point3Dd(0, 150, 30);
      aqs12.ReferencePoints[10] = new Point3Dd(-30, 110, 30);
      aqs12.ReferencePoints[11] = new Point3Dd(-30, 40, 30);
           
      // perform calibration
      Point3Dd[] residuals = new Point3Dd[12];
      calibrator.ExtrinsicMatrix = Metric.ExtrinsicTransformationFromPiece(rangemap.Planes[0], aqs12, calibrator, out residuals);
      calibrator.Save(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\TestOut.json"));
      
      // reconstruct point cloud with extrinsic calibration
      PointCloud calibratedPointCloud = calibrator.CreatePointCloud(rangemap, PointCloudFlags.Double);
      calibratedPointCloud.Save(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\calibratedPointCloud.ply"));
      
      //create rectified range map 
      //(parameters for rectification can be set differently)
      var boundingBox = calibratedPointCloud.CalculateBoundingBox();
      ValueRange<double> xRange = new ValueRange<double>(double.MinValue, double.MaxValue);
      ValueRange<double> yRange = new ValueRange<double>(double.MinValue, double.MaxValue);
      xRange.Min = boundingBox.X.Min;
      xRange.Max = boundingBox.X.Max;
      yRange.Min = boundingBox.Y.Min;
      yRange.Max = boundingBox.Y.Max;
      int targetWidth = rangemap.Width;
      int targetHeight = rangemap.Height;
      double bgValue = boundingBox.Z.Min - 10;
      var rectifiedRangemap = calibratedPointCloud.ToRangeMap(xRange, yRange, targetWidth, targetHeight, bgValue);
      rectifiedRangemap.Save(Environment.ExpandEnvironmentVariables(@"%cvb%\Tutorial\Metric\Images\rectifiedRangemap.tiff"));
    }
  }
}
3 Likes

Aligning a point cloud using a plane fit

In many cases it is necessary to transform 3D point clouds in a post processing. It is e.g. much easier to detect surface defects on flat objects, when the surface is aligned to the xy-plane of the coordinate system. Espacially when working with rectified range maps (2.5D images) it gets handy to do these kind of alignments before analyzing.

The following example demonstrates how to fit a plane into a subset of points of a point cloud and how to transform the point cloud with the resulting normal vector of the plane. The result will be a point cloud where the z-axis points into the same direction as the normal vector of the plane/surface.

First, we load an example point cloud:

var pc = PointCloud.FromFile(@"D:\MyPointCloud.ply");

For verification we can display it in an activeX control (Common Vision Blox 3D Viewer):

axCVCore3DViewer1.AddPointCloudVariable(pc.Handle.ToInt64(), 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0);

The sample point cloud is an AQS12 calibration target, already aligned to its base plane. In this example workflow we will fit a plane into one of the side planes.

For the plane fit we define a certain 3D region of interest (cuboid) within the point cloud:

var range = new ValueRange<double>(double.MinValue, double.MaxValue);
      var planeCuboid = new Cuboid(range, range, range);
      planeCuboid.X.Min = 700;
      planeCuboid.X.Max = 860;
      planeCuboid.Y.Min = 1035;
      planeCuboid.Y.Max = 1540;
      planeCuboid.Z.Min = 1080;
      planeCuboid.Z.Max = 1240;
      var plane = pc.FitPlane(planeCuboid);

Again, for verification, we can display the points of the cuboid used for the plane fit in the 3D viewer.

axCVCore3DViewer1.AddPointCloudSolid(pc.Crop(planeCuboid).Handle.ToInt64(), 0xFFFFFF, 1,1,1,2);

For the transformation of the point cloud we need to set up a rotation matrix first with the entries of the normal vector:

var roll = new Angle();
var pitch = new Angle();
var yaw = new Angle();

roll.Rad = Math.Atan(plane.Normal.Y/plane.Normal.Z);
pitch.Rad = Math.Atan(plane.Normal.X/plane.Normal.Z);
yaw.Rad = 0; // not necessary for an alignment into the xy-plane

var rotAngles = new RotationAngles3D(roll, pitch, yaw);
var rotMatrix = new Matrix3D(rotAngles);

Afterwards we can fill the transformation matrix with our calculated rotation matrix and a translation vector, that can be set to (0,0,0) in this case:

var translation = new Point3Dd(0, 0, 0);
var trafoMatrix = new AffineMatrix3D(rotMatrix, translation);

The transformation itself is done with the following method:

var pc_transformed = pc.Transform(trafoMatrix);

The transformed point cloud is now aligned to the fitted plane and can be visualized:

axCVCore3DViewer1.AddPointCloudVariable(pc_transformed.Handle.ToInt64(), 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1);
axCVCore3DViewer1.ResetCamera();

This is how rectified range maps look like when being calculated from the original point cloud (left) and transformed point cloud (right):

3 Likes

Hi,help me, I’m trouble. About your tutorial , my vs2017 editor make error where ‘Metric’ does not contain the definition of ‘ExternalicTransformationFromPiece’. CVB version is 13.04.

Hi @wugx

Sorry for the late response du to the vacation season.
In order to reproduce the error we need to know:

  • which tutorial are you using?
  • In which line the error happens?
  • What is exactly the error?

A minimal executable code that only contains the required lines to reproduce the error is appreciated!

About this topic content, eg. picture,

Hi @wugx

The name of the function is changed to Metric.CalculateRigidBodyTransformationFromAqs12Piece(). For more details please refer to %cvb%Doc/Cvb.Net.chm, in Stemmer.Cvb.Foundation>Metric Class

Thanks for your patience. But have a new question, about aqs12 calibration related function, Is valid that this function are applied in other calibrate target?what’s boundary condition that I can’t find this in the api doc?

Hi @wugx

The extrinsic calibration in CVB only supports aqs12 as calibration target.
I do not know exactly what you mean with “Boundary condition”, but in one of the override function of Metric.CalculateRigidBodyTransformationFromAqs12Piece(), there is an argument “rect aoi” which is the region of interest.

Thank you. If my calibration target is of other styles, are there other existing classes or functions? eg. the
section


Multi cylinder calibration target where parameter is (Center.x,Center.y,Radius) .

No, CVB does not support other calibration targets.

ok,thank you. it’s unreasonable.

But , Has’t support that we can independent develop the calibration function to caculate affine matixs by CVB. I can’t find the optimization solver for Multivariate linar equations.

We do not offer such a functionality in our SDK, we use of course optimizations method internally in CVB which are not offered in our SDK. If you plan to write your own calibration, you may find useful methods in Blob and edge modules. %cvb%Doc/blob.chm, %cvb%Doc/edge.chm.

3Dpoint, 3DMatrix and matrix multiplication and point cloud conversions can be found under the namespace Stemmer.Cvb.

2 Likes

Thank you for your kindness,Happy life.

1 Like

Hi, is this code still valid as of 10/2022? I seem to fail to find the Calibrator3D class actually. Thank you.

@MarSch

There were some changes and lots of developments in CVB 3D module since starting this thread, but the Calibrator3D class still exists. Please take a look at the CVB 14 online documentation:
https://help.commonvisionblox.com/NextGen/14.0/cvbnet/db/d65/class_stemmer_1_1_cvb_1_1_calibrator3_d.html

Hi Mandana, thank you very much.

I see that this is a protected class: https://help.commonvisionblox.com/NextGen/14.0/cvbnet/db/d65/class_stemmer_1_1_cvb_1_1_calibrator3_d.html#details

Is this intentional?

Hi @MarSch

Yes, it is intentional. This class is inherited by LaserPlaneCalibrator3D, MatrixCalibrator3D, and PinholeCameraCalibrator3D.

Thank you. However I can’t get around the fact that regardless of what I do, I’m able to access any ImageManager subclasses in the Net-API, however PointCloud and Calibration3D are not accessible with CVB 14.

As far as I can see in the object browser, I could do it unmanaged, but I would prefer not to do this.

@MarSch I noticed that you sent a same request to our support mailing account. As soon as this matter gets clear for you in our support case, we will update this post accordingly.