Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for RosettaScIIO #702

Open
CSSFrancis opened this issue Jan 8, 2025 · 1 comment
Open

Add Support for RosettaScIIO #702

CSSFrancis opened this issue Jan 8, 2025 · 1 comment

Comments

@CSSFrancis
Copy link

CSSFrancis commented Jan 8, 2025

Is your feature request related to a problem? Please describe.

I was going to spend a couple of days adding support for loading different file types using rosetasciio directly into py4DSTEM. Is there is anything that I might need to consider before doing this.

Describe the solution you'd like

Most of the complications are going to be limiting what filetypes should be allowed to be loaded and which filetypes are not supported. Currently Rosettasciio requires dask as a dependancy, is this an issue? I could potentially remove it as a requirement and only allow for non-lazy loading if this is something that you would prefer not to add. Do you have a current process for handling memory mapped arrays, or is treating them like numpy arrays and then having numpy automatically convert to in memory arrays fine.

Additional context

This should be a good start to understanding how to generalize the process of loading/ transferring data. I think that ultimately we need to have a .yaml file for each file format which details if it outputs 4D Data, and what metadata it has.

Edit:

It seems like there is already a quite nice version here for loading using memmap vs RAM that we could extend.

https://github.com/py4dstem/py4DSTEM/blob/966a41c7611161d4b3d8d082af5a145be0653b76/py4DSTEM/io/importfile.py#L17C5-L23

@bsavitzky
Copy link
Member

Thanks @CSSFrancis , this sounds great! Dask dependency is not a problem. And yup, you found our memory mapping code. Eventually having a .yaml per file format is a nice idea too. Please let us know if you have other questions :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants