How to cite

J. Martinez-Gomez, M. Cazorla, I. Garcia-Varea and V. Morell.- “ViDRILO: The Visual and Depth Robot Indoor Localization with Objects information dataset”. International Journal of Robotics Research. 34:14 pp. 1681-1687. 2015. doi: 10.1177/0278364915596058

News

October 2015

Vidrilo is now in the media section of the PCL.

June 2015

Accepted a paper describing the dataset for IJRR.

March 2015

A new experimentation page has been created

New classification models have been added to ViDRILO: kNN and Random Forest

New descriptors have been added to ViDRILO: GIST, PHOG, and ESF.

August 2014

The ViDRILO web page has been released

ViDRILO: The Visual and Depth Robot Indoor Localization with Objects information dataset

Experimentation Results

In order to analyze the dataset, we performed a set of experiments using the toolbox provided.

  • Classification Models
    • Support Vector Machines (SVM)
      • We have used the libSVM implementation with an exponential chi-square kernel
    • k-Nearest Neighbor (kNN)
      • Matlab implementation with k=7
    • Random Forest
      • Matlab implementation with 50 classification trees
  • Visual Descriptors
    • Pyramid Histogram of Oriented Gradients (PHOG)
      • We used the code proposed by Anna Bosch and Andrew Zisserman: Phog Code We used 2 levels and 30 bins.
      • Citation: A. Bosch, et al. (2007). Representing Shape with a Spatial Pyramid Kernel. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp. 401–408.
    • GIST
      • We used the code proposed by Aude Oliva and Antonio Torralba: Gist Code
      • Citation: A. Oliva & A.Torralba (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International journal of computer vision 42(3):145–175.
    • Greyscale Histogram
      • The perspective image is directly mapped into a 256 bins histogram
  • Depth Descriptors
    • Ensemble of Shape Function (ESF)
      • We used the implementation included in the PCL library
      • Citation: W. Wohlkinger & M. Vincze (2011). Ensemble of shape functions for 3D object classification. In Robotics and Biomimetics (ROBIO), 2011 IEEE International Conference on, pp. 2987–2992.
    • Deph Histogram
      • The RGB-D image is directly mapped into a 30 bins histogram using the z-coordinate

Room Classification - Accuracy (% of test rooms correctly classified)

  SVM kNN Random Forest
PHOG     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 90.8 56.5 52.1 63.3 33.0 TRAIN S1 91.9 51.5 48.4 58.6 31.7 TRAIN S1 100 61.1 52.0 61.9 31.8
S2 64.8 97.4 62.2 70.3 27.4 S2 60.1 97.3 56.5 65.1 26.2 S2 63.1 100 57.1 65.8 30.2
S3 46.0 61.9 94.2 55.1 23.5 S3 45.4 58.4 96.3 46.3 22.3 S3 52.8 59.0 98.3 47.2 28.2
S4 65.3 70.0 51.9 96.8 30.8 S4 61.7 65.1 47.8 97.3 28.8 S4 73.0 69.7 56.8 98.3 28.6
S5 44.5 43.1 37.5 39.3 98.7 S5 32.7 39.7 32.5 30.1 98.2 S5 36.3 33.7 26.7 32.5 99.2
GIST     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 98.7 64.4 61.0 67.8 35.0 TRAIN S1 93.8 65.5 58.9 71.3 34.3 TRAIN S1 100 62.9 57.1 64.6 35.4
S2 67.1 99.6 61.1 72.9 33.3 S2 67.8 97.6 65.0 73.4 25.8 S2 63.2 100 58.9 70.0 32.8
S3 56.1 59.8 99.5 55.3 32.1 S3 52.8 59.0 98.3 47.2 28.2 S3 52.0 58.1 100 49.2 31.2
S4 69.3 71.5 56.3 99.7 33.3 S4 73.0 69.7 56.8 98.3 28.6 S4 63.0 66.0 52.8 100 28.9
S5 53.2 46.9 43.5 46.4 99.7 S5 34.3 33.7 26.7 32.5 99.2 S5 44.0 38.0 33.0 29.0 100
GrayScale Histogram     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 73.9 58.4 51.1 58.6 38.3 TRAIN S1 88.0 53.3 45.0 51.7 30.3 TRAIN S1 100 59.6 49.8 58.2 34.0
S2 60.7 82.9 53.0 57.8 37.8 S2 55.4 93.4 48.1 49.8 30.8 S2 58.1 100 50.9 52.5 32.1
S3 50.2 50.9 86.6 48.4 36.9 S3 50.3 49.6 92.6 47.0 28.3 S3 47.8 51.3 100 48.3 31.0
S4 56.6 55.4 50.2 87.4 33.0 S4 48.5 45.7 40.8 94.9 24.2 S4 56.1 54.3 49.8 100 28.6
S5 44.7 43.5 39.4 37.8 84.7 S5 32.1 29.6 26.6 33.0 96.0 S5 36.8 39.4 35.3 30.6 100
ESF     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 68.9 61.9 57.0 59.9 35.2 TRAIN S1 82.6 57.4 54.1 59.8 34 TRAIN S1 100 64.8 63.0 64.2 36.2
S2 64.5 78.2 61.4 63.3 34.2 S2 58.5 91.8 65.3 65.3 30.3 S2 64.1 100 70.2 67.1 34.2
S3 59.5 64.7 81.0 62.3 34.3 S3 55.9 63.5 90.9 62.3 29.3 S3 61.6 65.8 100 65.6 34.5
S4 63.5 65.3 65.5 80.1 36.7 S4 61.0 61.0 67.7 93.4 32.2 S4 67.1 69.3 73.8 100 35.6
S5 49.6 47.0 46.6 46.8 84.2 S5 44.5 39.2 38.7 36.9 95.1 S5 52.0 48.2 48.6 49.8 100
Depth Histogram     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 71.2 59.3 54.6 61.3 38.2 TRAIN S1 77.7 55.2 50.3 56.3 28.5 TRAIN S1 98.5 61.6 58.0 62.2 33.1
S2 60.2 79.1 32.1 62.7 36.7 S2 56.7 86.7 59.5 59.7 28.6 S2 60.1 98.1 67.2 64.1 33.2
S3 57.6 60.8 82.7 60.0 37.6 S3 54.7 59.0 87.9 59.4 29.1 S3 57.9 62.4 98.8 64.1 31.5
S4 62.7 52.6 62.9 81.9 32.7 S4 58.7 61.1 63.8 88.6 26.8 S4 63.4 65.0 70.3 98.7 30.9
S5 47.6 48.4 44.4 47.4 87.3 S5 41.8 39.4 34.0 36.6 93.8 S5 45.0 43.3 40.5 45.3 99.3

Object Recognition - F1 Score (2* (PREC*REC)/(PREC+REC))

  SVM kNN Random Forest
PHOG     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 1.00 0.36 0.37 0.43 0.37 TRAIN S1 0.91 0.39 0.39 0.42 0.38 TRAIN S1 1.00 0.24 0.23 0.31 0.18
S2 0.35 1.00 0.59 0.54 0.21 S2 0.40 0.96 0.53 0.49 0.34 S2 0.16 1.00 0.36 0.43 0.02
S3 0.27 0.49 1.00 0.34 0.21 S3 0.36 0.48 0.95 0.36 0.27 S3 0.20 0.36 1.00 0.26 0.09
S4 0.48 0.53 0.37 1.00 0.18 S4 0.46 0.48 0.35 0.95 0.24 S4 0.24 0.33 0.24 1.00 0.04
S5 0.44 0.23 0.27 0.25 1.00 S5 0.40 0.31 0.36 0.29 0.98 S5 0.40 0.22 0.25 0.30 1.00
GIST     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 1.00 0.45 0.36 0.55 0.37 TRAIN S1 0.93 0.46 0.45 0.64 0.40 TRAIN S1 1.00 0.35 0.24 0.38 0.34
S2 0.40 1.00 0.42 0.53 0.14 S2 0.53 0.97 0.56 0.63 0.26 S2 0.33 1.00 0.33 0.37 0.15
S3 0.28 0.40 1.00 0.34 0.13 S3 0.38 0.44 0.97 0.42 0.25 S3 0.27 0.39 1.00 0.33 0.11
S4 0.57 0.57 0.43 1.00 0.26 S4 0.66 0.61 0.49 0.97 0.34 S4 0.45 0.47 0.33 1.00 0.23
S5 0.44 0.26 0.22 0.32 1.00 S5 0.42 0.34 0.37 0.41 0.99 S5 0.42 0.23 0.25 0.28 1.00
GrayScale Histogram     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 0.99 0.42 0.25 0.36 0.36 TRAIN S1 0.86 0.35 0.17 0.32 0.36 TRAIN S1 1.00 0.29 0.11 0.27 0.31
S2 0.41 1.00 0.24 0.35 0.30 S2 0.41 0.90 0.19 0.28 0.34 S2 0.32 1.00 0.13 0.23 0.21
S3 0.25 0.27 0.99 0.31 0.24 S3 0.25 0.24 0.90 0.30 0.28 S3 0.12 0.14 1.00 0.14 0.11
S4 0.34 0.28 0.29 0.99 0.33 S4 0.34 0.30 0.25 0.92 0.32 S4 0.29 0.19 0.24 1.00 0.26
S5 0.32 0.27 0.25 0.31 1.00 S5 0.31 0.29 0.26 0.25 0.97 S5 0.31 0.22 0.15 0.25 1.00
ESF     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 1.00 0.47 0.55 0.43 0.40 TRAIN S1 0.80 0.42 0.51 0.47 0.38 TRAIN S1 1.00 0.40 0.44 0.40 0.33
S2 0.53 1.00 0.65 0.54 0.39 S2 0.44 0.88 0.59 0.51 0.37 S2 0.45 1.00 0.56 0.45 0.30
S3 0.49 0.53 1.00 0.55 0.34 S3 0.45 0.50 0.90 0.50 0.36 S3 0.44 0.47 1.00 0.50 0.26
S4 0.61 0.54 0.67 1.00 0.36 S4 0.53 0.51 0.60 0.90 0.40 S4 0.46 0.43 0.59 1.00 0.25
S5 0.38 0.36 0.42 0.35 1.00 S5 0.47 0.33 0.38 0.32 0.96 S5 0.36 0.35 0.38 0.32 1.00
Depth Histogram     TEST     TEST     TEST
  S1 S2 S3 S4 S5   S1 S2 S3 S4 S5   S1 S2 S3 S4 S5
TRAIN S1 0.99 0.43 0.50 0.52 0.36 TRAIN S1 0.67 0.38 0.41 0.41 0.24 TRAIN S1 0.99 0.38 0.44 0.50 0.27
S2 0.44 0.99 0.59 0.55 0.34 S2 0.36 0.80 0.48 0.49 0.26 S2 0.36 0.99 0.56 0.52 0.23
S3 0.40 0.49 0.99 0.54 0.31 S3 0.34 0.42 0.83 0.46 0.26 S3 0.36 0.48 1.00 0.57 0.26
S4 0.44 0.48 0.63 1.00 0.31 S4 0.39 0.46 0.55 0.82 0.25 S4 0.35 0.43 0.57 1.00 0.20
S5 0.31 0.30 0.35 0.32 1.00 S5 0.26 0.30 0.32 0.30 0.94 S5 0.36 0.31 0.35 0.34 1.00