Extend SIF metadata

[mruijzendaal]

Hi,

It would be great if we could extend the current metadata to incorporate more information on e.g. the gate time.
For reference, I include here the "Acquisition info" that Andor Solis lists for a measurement I took, and the 66-line header for the same file:

measurement_info.txt:

Filename:                     measurement.sif
Date and Time:                Fri Jul 28 13:51:04.409 2023
Software Version:             4.32.30000.0
Temperature (C):              -25
Model:                        DH334T-18F-63
Data Type:                    Counts
Acquisition Mode:             Kinetics
Trigger Mode:                 External
Exposure Time (secs):         3
Number of Accumulations:      1
Number in Kinetics Series:    20
Readout Mode:                 Full Vertical Binning
Horizontal binning:           1
Extended Data Range:          off
Horizontally flipped:         false
Vertical Shift Speed (usecs): 6.5
Pixel Readout Rate (MHz):     1
Clock Amplitude:              Normal
Serial Number:                ICCD-06390
Pre-Amplifier Gain:           4x
Trigger level (V):            1
Termination:                  50 ohms
Inverted:                     false
Intelligate:                  off
Insertion delay:              normal
Gain level:                   2500
Integrate on chip:            on
IOC frequency (Hz):           5e+05
IOC number of pulses:         1
PIV Mode:                     off
Output A Width (nsecs):       10
Output A Delay (nsecs):       0
Output A Polarity:            Negative
Gate Mode:                    DDG on
Gate Width (nsecs):           10
Gate Delay (nsecs):           0
Gate Delay Step (nsecs):      0
Gate Width Step (nsecs):      0
Spurious Noise Filter Mode:   No Filter
Photon counted:               false
Data Averaging Filter Mode:   No Filter
KY328i:
Serial Number:                KY-4231
Wavelength (nm):              561.47
Grating Groove Density (l/mm):599.566
Grating Blaze:                650NM
Output Flipper Port:          Direct
Input Side Slit Width (um):   50
Side Iris Steps:              50

measurement.sif header:

Andor Technology Multi-Channel File
65538 1
65567 0 0 1 1690545259 -25 � � � � � 0 3 3.0221 3.0221 1 � 3.0221 1e-06 0 1 2500 0 0 1e-08 0 0 � � 1 0 5 0 0 0 0 0 0 1 500000 0 6.5e-06 0 4 6390 1 1 -999 0 0 0 0 0 5 4 32 30000 0 0 1 1000 0 0 0 550 0 0 0 0 0 0 0 0 0 1 1 13
DH334T-18F-63 
 1024 1024 69
C:\asdasdasdasd\measurement.sif 
65538 2048
<line 8 with non-ascii characters omitted>
<line 9 with non-ascii characters omitted>
<line 10 with non-ascii characters omitted>
<line 11 with non-ascii characters omitted>
<line 12 with non-ascii characters omitted>
<line 13 with non-ascii characters omitted>
65538 � � � � 0.027 0
65540 1 1 561.47 1 1 599.566 650NM
1 50
1 0 0 0 0  0 0 0 0 0 0 10
0 KY328i
0 10
0 10
0 0 
65537 0 500 1200
0 999 999

65539 1 5 2500 0 0 0 10000 1 2000000 1 100 0 0 100 0 0
1 1 0 10000
0 1 0 2000
0 1 0 2000
1 0 0

-1
65538 0 1024 1024 1024 1024 1 1 1 1
65540 � � � � � �
529.93812442523 0.061715845778342 -2.28349748230931e-07 -5.07163560661353e-11
0 1 0 0
0 1 0 0
422
13
13
1
10
Wavelength6
Counts12
Pixel number65541 1 1024 1024 1 20 1 20480 1024
65538 1 1024 1024 1 1024 1 0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
0

measurement.sif.zip
measurement_info.txt

[mruijzendaal]

#34 implements changes that would partially fix the current incomplete metadata. @fujiisoup Could we create a new release with the proposed changes?

[fujiisoup]

Could we create a new release with the proposed changes?

Yes, I will. Let's finalize the PR.

[mruijzendaal]

I found the following XML-structured metadata at the end of the SIF file:

<?xml version="1.0" ?>
<Signals>
    <Signal>
        <TInstaImageData>
            <PreAmpGainText>4x</PreAmpGainText>
            <SpectrographSerial>KY-4231</SpectrographSerial>
            <mSec>616</mSec>
            <DIRECT_IRIS_PRESENT>0</DIRECT_IRIS_PRESENT>
            <DIRECT_IRIS_POSITION>-1</DIRECT_IRIS_POSITION>
            <SIDE_IRIS_PRESENT>1</SIDE_IRIS_PRESENT>
            <SIDE_IRIS_POSITION>50</SIDE_IRIS_POSITION>
            <FullSerialNumber>ICCD-06390</FullSerialNumber>
            <DARKTHRESHOLD>-999</DARKTHRESHOLD>
        </TInstaImageData>
        <TInstaImage>
            <IRIGAvailable>0</IRIGAvailable>
        </TInstaImage>
        <TCalibImage>
            <TCalibImageData />
            <TAndorImage>
                <TAndorImageData>
                    <ImageFormat>0</ImageFormat>
                    <ReadoutPort>-1</ReadoutPort>
                    <ExposureWindowHeight>0</ExposureWindowHeight>
                    <LineScanSpeed>0</LineScanSpeed>
                    <AlternatingReadoutDirection>0</AlternatingReadoutDirection>
                    <ScanSpeedControl>0</ScanSpeedControl>
                    <ReadoutDirection>-1</ReadoutDirection>
                    <FastKineticsStorageMode>0</FastKineticsStorageMode>
                    <FastKineticsTimeScanMode>-1</FastKineticsTimeScanMode>
                    <TickInterval>10</TickInterval>
                    <PIVMode>0</PIVMode>
                    <ExtendedDynamicRange>0</ExtendedDynamicRange>
                </TAndorImageData>
            </TAndorImage>
        </TCalibImage>
    </Signal>
</Signals>
<References />
<Backgrounds>
    <Background>
        <TInstaImageData>
            <PreAmpGainText>4x</PreAmpGainText>
            <SpectrographSerial>KY-4231</SpectrographSerial>
            <mSec>0</mSec>
            <DIRECT_IRIS_PRESENT>0</DIRECT_IRIS_PRESENT>
            <DIRECT_IRIS_POSITION>-1</DIRECT_IRIS_POSITION>
            <SIDE_IRIS_PRESENT>1</SIDE_IRIS_PRESENT>
            <SIDE_IRIS_POSITION>50</SIDE_IRIS_POSITION>
            <FullSerialNumber>ICCD-06390</FullSerialNumber>
            <DARKTHRESHOLD>-999</DARKTHRESHOLD>
        </TInstaImageData>
        <TInstaImage>
            <IRIGAvailable>0</IRIGAvailable>
        </TInstaImage>
        <TCalibImage>
            <TCalibImageData />
            <TAndorImage>
                <TAndorImageData>
                    <ImageFormat>0</ImageFormat>
                    <ReadoutPort>-1</ReadoutPort>
                    <ExposureWindowHeight>0</ExposureWindowHeight>
                    <LineScanSpeed>0</LineScanSpeed>
                    <AlternatingReadoutDirection>0</AlternatingReadoutDirection>
                    <ScanSpeedControl>0</ScanSpeedControl>
                    <ReadoutDirection>-1</ReadoutDirection>
                    <FastKineticsStorageMode>0</FastKineticsStorageMode>
                    <FastKineticsTimeScanMode>-1</FastKineticsTimeScanMode>
                    <TickInterval>10</TickInterval>
                    <PIVMode>0</PIVMode>
                    <ExtendedDynamicRange>0</ExtendedDynamicRange>
                </TAndorImageData>
            </TAndorImage>
        </TCalibImage>
    </Background>
</Backgrounds>
<Lives />
<Sources />
�������SIFX

This seems to also be useful metadata.

[mruijzendaal]

As for the XML metadata, I'm not sure how many edge cases there are (multiple signals, references or backgrounds?). I think there are two options:

• Flattening this metadata (e.g. keys such as Signal_0_PreAmpGainText)
• Convert it to a dict structure.

Do you have any preference?

[fujiisoup]

Hi @mruijzendaal

I like this option:

Flattening this metadata (e.g. keys such as Signal_0_PreAmpGainText)

I think we can just ignore edge cases. If someone really needs these metadata, they may implement it.