Skip to content

Home

Jump to bottom
Mathieu Brédif edited this page Feb 16, 2016 · 17 revisions

General wiki

To consult the general wiki page of the itowns project, go to https://github.com/iTowns/itowns-project/wiki

Data preparation

Images

Images should be accessible over http using a url pattern (see below) Their format should be compatible with the THREE.js WebGL library, such as JPEG or PNG.

The image locations are provided by an url template, like images/140616/Paris-140616_0740-{cam}-00001_{pano}.jpg, where cam and pano are placeholders for the camera and panoramic names, respectively.

Images are, for now, considered to be composed of a set of panoramics where each panoramic is itself a set of unassembled images from a rigid multi-camera panoramic head. We also assume that the panoramic camera head is rigid such that the relative positions and rotations of image sensors are constant across panoramics. Thus image metadata is provided by two JSON files : panoramic poses and camera calibrations

TODO: Explicit the frame conventions and provide an illustration.

Panoramic poses

The panoramic head moving frame is defined by a JSON file providing an array of panoramic objects:

[
 {
   "pano": "0000482",
   "easting": 651187.76,
   "northing": 6861379.05,
   "altitude": 39.39,
   "heading": 176.117188,
   "roll": 0.126007,
   "pitch": 1.280821,
   "date": "2014-06-16T12:31:34.841"
 },
 ...
]
  • pano is the string id of this panoramic.
  • easting, northing, altitude is the origin of the moving panoramic frame defined in some static world coordinate system. The example above uses Lambert93 coordinates.
  • heading, roll and pitch provide the orientation of the moving frame using three Euler angles (in degrees).
  • date denotes the simultaneous time of acquisition of all the images of this panoramic as an RFC2822 or ISO 8601 date string.

Camera calibrations

Image calibration should be provided using a json file containing an array of object of the following form, which describe all the cameras of the panoramic head :

[
 {
  "cam": "300",
  'mask': 'images/mask300.jpg',
  "rotation": [
    0.000735648, -0.00148884, -0.999999,
    0.999998   ,  0.00205265,  0.000732591,
    0.00205156 , -0.999997  ,  0.00149035
  ],
  "position": [ -0.145711, -0.0008142, -0.867 ],
  "projection": [
    1150.66785706630299,    0               , 1030.29197487242254,
       0               , 1150.66785706630299, 1023.03935469545331,
       0               ,    0               ,    1
  ],
  "size" : [2048, 2048],
  "distortion": {
    "pps": [1042.178,1020.435],
    "poly357": [ -1.33791587603751819E-7, 3.47540977328314388E-14, -4.44103985918888078E-21 ]
  },
  "orientation": 3
 },
 ...
]

  • cam provides a camera name to the calibration (which may be used to generate image urls)

  • mask is an optional attribute which is an URL relative to this JSON file of à gray-level image. The mask is not necessarily of the same size as the image as it will be stretched automatically. White pixels are 100% masked and black pixels are fully kept. As such, not providing a mask image is equivalent to providing a 1x1 black image.

  • rotation is an array of 9 floats, describing a 3x3 rotation matrix from panoramic coordinates (to be defined) to camera coordinates (X along lines, Y along columns, Z along the optical axis)

  • position denotes the image center position with respect to the panoramic coordinate frame

  • projection provides a 3x3 projection matrix from the 3D camera frame to the projective 2D image frame (in pixels). The common case is to provide the following matrix, where focal denotes the focal length along rows and columns in pixels and ppa denotes the principal point of autocollimation in pixels :

      focal.x , 0       , ppa.x,
  0       , focal.y , ppa.y,
  0       , 0       , 1

  • distortion is optional. size is the sensor size (in pixels) and pps is the distortion center (in pixels).

  • poly357 provide the r3, r5 and r7 coefficients of a distortion polynom. ([0,0,0] means no distortion)

  • orientation adds a further rotation of 0,90,180 or 270 degrees around the optical axis. It is redundant with rotation and is likely to be deprecated in future releases. orientation:3 denotes no rotation.

For performance, you should resample some or all your images to remove any distortion and encode the mask in its alpha channel.

Terrain (DTM)

The terrain DTM should be provided as a JSON file containing an array of 3D sensor positions. Here is an example from the sample Data, in Lambert 93:

[
  {"e":651432.81,"n":6861436.46,"h":39.7},
  {"e":651430.81,"n":6861420.57,"h":39.86},
  ...
]

To get ground points from these sensor positions, we currently apply a constant hardcoded vertical offset of 2.1m. We plan These ground positions are then automatically processed to create an elevation grid that is used to guide the immersive navigation and provide the ground geometry during the immersive texture reprojections.

Buildings

Contrary to textured buildings as seen in fly mode, these buildings are never looked at directly and are only used as geometry proxies (similar to the ground mesh) to reproject panoramic images in the immersive mode. The format is in JSON and thus more or less self-describing. Beware that it is not yet a standard GeoJSON (wait for itowns2!).

PointCloud

Pointclouds should be served as binary files. By default it should be organized as a series of N binary-encoded Float32 without any header, where X,Y,Z is the 3D position and I is the lidar intensity or reflectance :

 X1,Y1,Z1,I1, X2,Y2,Z2,I2,..., XN,YN,ZN,IN.

To prevent precision issues due to float-encoding of large values, we use actually use offseted coordinates, with an laser.offset provided through the main API call (see sample html files)

The general idea to get the corresponding PointCloud for the Panoramic image, is to consider their acquisition times. Panoramic timestamps are provided by their JSON metadata file. The process is to look at the acquisition time of the current panoramic in order to generate the urls of the corresponding Pointclouds with the following pattern : pointclouds/{date}/{LOD}/{floor(seconds*10)}.bin, such as pointclouds/140616/HR/450608.bin:

  • {date} follows the formatting YYMMDD
  • {LOD} is either LR (10% of the data) or HR (the remaining 90%)
  • {floor(seconds*10)} indicates the amount of number of seconds since the start of the day (indicated by the {date}, using a 100ms resolution, effectively grouping points by chunks of 0.1 seconds.

In the sample data, the 2 lidar files LR and HR for the same period of 100ms are used for a simple LOD approach.

More practically what happens is :

  • Look up the timestamp of the current panoramic ex: 12h31 on the 14th of june 2014
  • Derive the date and the number of seconds since the start of the day
  • Load a few seconds of PointClouds around this time, beginning by Low Resolution chunks : 140616/LR/450608.bin, 140616/LR/450609.bin, etc and then 140616/HR/450608.bin, 140616/HR/450609.bin...

Textured buildings

This technological preview uses a custom textured mesh format called B3D. 3DS tiles should also work. This will be heavily reworked in version 2.

Clone this wiki locally