Incremental IO

[mikeboers]

I have some tools I have written against the PRMan point cloud api which I would love to adapt to using this library. However, a couple of the operations we have been running against particularly large point clouds would not be possible on our hardware with this library as it appears to read the entire dataset into RAM.

Is it feasible to adapt this library to also offer a streaming API so I can read/write a point at a time and the entire dataset is not held in RAM?

[aselle]

It is feasible and cached reads (or even serialized streaming reads) are possible in the framework. The basic idea would be to implement a new basic data store rather than ParticlesSimple. It could use mmap() or cache reads. It would need to be adapted to a more generalized reader classes. However, the usage of the library to the user would remain relatively the same. Obviously it would be more restrictive about the manipulation it would allow, but I'm open to this evolution and would love to discuss the design more with someone who is interested in going in this direction.