I have seen from the Intel D400 page you will be supporting the realsense drivers out of the gate. I am very interested in the potential for using multiple cameras to build up a highly detailed colour scan of our environment and see CVB as a good option for that. Especially if it can pre stitch the point clouds before passing them to ROS modules and image analysis modules.
Intel Realsense D400 Stitching
Is it possible to use CVB for the acquisition of 4+ D400 point clouds and stitch them together? For example if I know the transforms between the cameras will it be able to deliver both the acquisition and the stitching then output an image?
Do you have any example applications of 3D stitching?
Passing pointclouds/images to ROS
Is it possible to pass pointclouds straight from CVB to ROS modules? Or use CVB within the ROS architecture?
If I can would it be able to pass the pre processed stitched pointcloud discussed above?
Iâm terribly sorry for the delay. I should have answered earlier.
Regarding Release:
Our intention is to deliver integration with CVB 13.1 however we might not be able to deliver on the first release of CVB 13.1. Short answer: It will be available soon rather than now.
Regarding Realsense, CVB and ROS:
Well yes and no. A camera like Realsense is the typical 3D camera for a ROS system, so there are several examples for this.
With CVB we donât actually have plans to create a CVB-ROS integration (right now!). However we wonât change datatypes in a way to make these systems incompatible (short answer: ârawâ datatypes will exist and should play well with ROS).
@FredI Could you tell me what âimage analysis modulesâ you meant? This is useful information for us. @FredI Secondly, could you tell me the parameters of âhighly detailed colour scanâ?
Once the Realsense integration for CVB is finished / released, you can aquire via CVB and pass the image to ROS. Technically it is possible right now, with some tweaking (GIGE or U3V cameras, we just donât have the realsense integration in CVB) .
Regarding Stitching Pointcloud:
This is a planned feature of our 3D Tools. I canât actually name a release date.
Again an answer with your timeframe for your projects is useful to prioritize development.
No worries for the delay @c.hartmann it was a speculative enquiry.
Realsense + ROS
We are already using this integration but thank you for highlighting it. Intel support this code base very well and its very simple to use.
In regards to the questions
1.We are doing a couple of different detectionâs using Tensorflow and Keras with some custom algorithms. Not any CVB modules unfortunately.
2. As we are using deep learning the parameters for a âhighly detailed colour scanâ are detailed enough for our detector to work. Its not a fixed thing, we believe a number of D415s could be the correct solution.
Stitching
Once you have this module developed we would be extremely interested in purchasing this. The effort of syncing, calibrating and stitching cameras is significant so if you provided a solution for 2+ 3D camera stitching from any setup(as long ad they overlap) that would be great.
Project Timeframe
ASAP, we are actively developing this at the moment. If you had it now we would buy it.
What we will be releasing in 2018 in terms of 3D processing and handling capabilities will be a first step and a foundation to build upon. To give some background: The 3D functionality in in the past (in the version range [10.x, ⌠12.1]) was based on parts of the Sal3D library by AQSense. In 2016, AQSense was bought by Cognex and the Sal3D library was discontinued as a product. As a side effect of that decision, availability of the 3D functionality in ended in October 2017 (which is why 13.0 has no 3D tools in it), and STEMMER had to start building a replacement from scratch.
This is why initially in the 2018 release, we will basically cover what was there before with a few added details here and there:
Functionality to acquire, handle, transform and work on 3D data (this time, however, with GenICam compliant data format coverage, i. e. we will be able to talk directly to GenICam compliant cameras capable of outputting 3D point clouds once they become available).
A component for displaying 3D point clouds interactively.
An ICP-based point-cloud-matching algorithm comparable to what was called CVB Match 3D before.
A 3D calibration tool mostly for extrinsic camera calibration - this time with support for intrinsic calibration as well (initially intrinsic calibration will cover the calibration models stored in AT cameras, later on also LMI and and custom calibration).
Once we have that rolled out, we will of course not stop there, but if you ask about the time frame for the release of e. g. a general purpose 3D stitching implementation, we are rather looking at 12 months or more.
@illusive thanks for the clarification. It is good to understand a bit more about the road map, when the CVB 2018 3D release and the intel realsense support is released we will definitely investigate using it. If the stitching tool was there we would definitely be using it.
i thought about this a little bit. And realized the pro point âBetter imageâ might be complete nonsense. These cameras donât use the same optics / lenses. So this statement is a bad guess at best.
Any new information would generally have a news item on www.stemmer-imaging.com (it is searchable too), but I am not sure what information you are looking for - are you just asking if there are updates?
For using Realsense with ROS, there are some options:
CVB
There has never been any development for a ROS-CVB bridge by Stemmer Imaging.
However https://index.ros.org/r/ros_cvb_camera_driver/ is available.
I.e. Using CVB Acquisition (+ the TransportLayer developed by Stemmer) with a Wrapper for ROS.
This is not Realsense specific.
2018 in terms of 3D processing and handling capabilities will be a first step and a foundation to build upon. To give some background: The 3D functionality in