The article describes a multi-sensor dataset of human-human handovers composed of over 1000 recordings collected from 18 volunteers. The recordings refer to 76 test configurations, which consider different volunteer׳s starting positions and roles, objects to pass and motion strategies. NuTonomy released a self-driving dataset called nuScenes that it claims is the “largest open-source, multi-sensor self-driving dataset available to public.” According to nuTonomy, other self-driving datasets such as Cityscapes, Mapillary Vistas, Apolloscapes, and Berkeley Deep Drive focused only on camera-based object detection.
Super-resolution algorithms reconstruct high-resolution images from low-resolution input images. For this purpose, multi-sensor super-resolution describes a technique to reconstruct high-resolution images from low-resolution data for one modality under the guidance of another modality. In our work, we investigate this concept for hybrid range imaging to super-resolve low-resolution 3-D range data that is fused with complementary photometric information (RGB data). In order to improve robustness, different computational steps of the super-resolution algorithm can be applied on the guidance data instead of using the low-resolution input images directly. This is beneficial e.g.
For motion estimation required for a multi-frame super-resolution reconstruction as higher accuracy can be achieved on the guidance images.We plan to include more datasets to consider different applications of our method in the future. The use of this data is free but please cite our corresponding paper if you would like to use it in your next publication.
We employed multi-sensor super-resolution to indoor range data. Our database consists of simulated data with known ground truth as well as real data acquired with real sensors (Microsoft's Kinect).Simulated dataset. We created 4 synthetic datasets consisting of 3-D range data fused with photometric data (VGA resolution, 640 x 480 px) rendered from a an artificial scene. The data was generated using the. Low-resolution range images are simulated by disturbing the ground truth with Gaussian blur and adding zero-mean Gaussian noise (use subsampling with desired factor to simulate reduced spatial resolution). Each dataset consists of 40 consecutive frames of the underlying scene.Organization of the data:. RGB (.png): Photometric information encoded as RGB image (resolution: 640 x 480).
Range (.txt): Range data with simulated Gaussian blur and zero-mean Gaussian noise (resolution: 640 x 480). Please note that downsampling was omitted for this dataset.
Use nearest-neigbour interpolation to obtain the desired pixel resolution. Ground truth (.txt): Ground truth range data (resolution: 640 x 480). Kinect datasets. We captured three datasets of indoor scenes using Microsoft's Kinect.
Range and photometric data was captured in VGA resolution (640 x 480 px) using a frame rate of 30 fps. During the acquisition, the device was held in the hand such that a small shaking ensured the required motion for super-resolution over consecutive frames.Organization of the data:.
RGB (.png): Photometric information encoded as RGB image (resolution: 640 x 480). Range (.txt): Range data (resolution: 640 x 480). In terms of image-guided surgery, multi-sensor super-resolution is applied in hybrid 3-D endoscopy based on Time-of-Flight (ToF) imaging.
Here, we provide the datasets used for our experiments in the related publications. In the current state, the database contains synthetic endoscopic images with ground truth range data.Baseline dataset. We created 6 synthetic datasets consisting of 3-D range data (64 x 48 px) fused with high-resolution RGB images (640 x 480 px) from a laparoscopic scene. The ground truth data was generated using the.
Low-resolution range images were simulated by disturbing the ground truth with Gaussian blur, downsample it and adding zero-mean Gaussian noise. Random movements of the virtual camera was used to simulate movements of the endoscope held by the surgeon. Small displacements of endoscopic tools and organs simulated minimally invasive surgery in a realistic manner.
This images can be used as a baseline dataset, as endoscope motion simulates a small jitter of the which can be registered using optical flow and the associated 3-D data is not affected by sensor-specific errors, e.g. Flying Pixels or specular reflections. Each dataset consists of 40 frames.Organization of the data:. RGB (.png): Photometric information encoded as RGB images(resolution: 640 x 480). Range (.txt): Ground truth range data and low-resolution data(resolution: 64 x 48). Outlier dataset.
In addition to the baseline dataset, we provide 4 datasets to simulate more challenging scenarios. We simulated distance-dependent Gaussian noise, Perlin noise to simulate specular highlights in range images and flying Pixels on depth edges. In terms of motion, we considered small random endoscope movements (data set S1), larger movements of the endoscope (S2), larger tool movements (S3) and organ movements due to respiratory motion (S4). Each dataset consists of 40 frames.Organization of the data:. RGB (.png): Photometric information encoded as RGB images(resolution: 640 x 480).
Ground truth (.txt) Ground truth range data (resolution: 640 x 480). Range (.txt): Simulated, noisy range data (resolution: 640 x 480). Please note that downsampling was omitted for this dataset. Use nearest-neigbour interpolation to obtain the desired pixel resoution.
The table below shows the actual list of recorded sequences. Sequences marked with ^ are not distributed via web.
Only sample images are available. Further information about how to get these sequences can be found in the. Sequences marked with. can be downloaded fully. Those sequences are short ones for demonstration purpose. Was your download successful?Due to the high frame rate and resolution of the cameras, the number of the cameras and the lossless raw image storage one whole sequence can easily reach several Gigabytes.
Therefore a download of the sequences is hardly possible.We offer instead to send the whole set of sequences on an external hard drive disk by a price of 4000 Swedish Krona(SEK) incl. Worldwide shipping (as of 2013-06-23, this price may change). Feel free to.Note: Shipping starts as soon as possible. Information about shipping can be found here.For reading the sensor data stored instreamfilesan API is needed, theStreamReader. There are different APIs for different programming languages. The specific APIs can downloaded by clicking on the corresponding programming language.
Documentation of the functions is inline. A common description of the functions can be found in the.ROSFor initialization the API needs the paths to the project on the one hand and to a XML file on the other hand which describes the IDs and messages used in the streamfiles.
These paths can be committed directly or stored in a XML file which is used instead (more information in the API documentation). Below are the mandatory message/ID XML file and an example file of the initialization XML file.Furthermore, information about the experimental setup, the stream format, etc. Can be found in the links below.Sketches of the experimental setupDetailed sensor description incl. Camera calibration parameterStream format descriptionGUIDELINES FOR EXPANDING OUR WORKInformation to be published soon.