Table

Suppose you have a variable named table (which is an instance of Table) that represents the following information in an equipment register for equipment that measures spectral irradiance

<table comment="Spectral">
  <type>   int       ,    double ,    double     </type>
  <unit>   nm        ,    W/m^2  ,    W/m^2      </unit>
  <header> Wavelength, Irradiance, u(Irradiance) </header>
  <data>   250       ,    0.01818,   0.02033
           300       ,    0.18478,   0.01755
           350       ,    0.80845,   0.01606
           400       ,    2.21355,   0.01405
           450       ,    4.49004,   0.01250
           500       ,    7.45135,   0.01200
           550       ,   10.75753,   0.01152
           600       ,   14.03809,   0.01102
           650       ,   16.99469,   0.01103
           700       ,   19.44093,   0.01077
  </data>
</table>

The table instance is a numpy structured array that has the header values as the field name of each column

>>> table.header
array(['Wavelength', 'Irradiance', 'u(Irradiance)'], dtype='<U13')
>>> table["Wavelength"]
Table([250, 300, 350, 400, 450, 500, 550, 600, 650, 700])
>>> table.types["Irradiance"]
array(dtype('float64'), dtype=object)
>>> assert table.units["u(Irradiance)"] == "W/m^2"

Since table is a numpy array, you can index it

>>> print(table[0])
(250, 0.01818, 0.02033)
>>> sliced=table[:3]
>>> print(sliced)
[(250, 0.01818, 0.02033) (300, 0.18478, 0.01755) (350, 0.80845, 0.01606)]

and since sliced is another Table instance, the attributes of the original table are available

>>> sliced.comment
'Spectral'
>>> sliced.header
array(['Wavelength', 'Irradiance', 'u(Irradiance)'], dtype='<U13')

You can also perform mathematical operations and call numpy functions directly with the table instance

>>> np.cos(1 + table["Irradiance"])
Table([ 0.52491592,  0.37650087, -0.2354229 , -0.99741219,  0.70160756,
       -0.56246854,  0.6903377 , -0.78390036,  0.65631968, -0.0205763 ])

Suppose you wanted to get all Irradiance values in the table that are for UV light (i.e., wavelengths < 400 nm)

>>> table["Irradiance"][ table["Wavelength"] < 400 ]
Table([0.01818, 0.18478, 0.80845])

If you prefer to work with unstructured data, you can convert the table by calling the unstructured method

>>> unstructured = table.unstructured()
>>> unstructured
Table([[2.500000e+02, 1.818000e-02, 2.033000e-02],
       [3.000000e+02, 1.847800e-01, 1.755000e-02],
       [3.500000e+02, 8.084500e-01, 1.606000e-02],
       [4.000000e+02, 2.213550e+00, 1.405000e-02],
       [4.500000e+02, 4.490040e+00, 1.250000e-02],
       [5.000000e+02, 7.451350e+00, 1.200000e-02],
       [5.500000e+02, 1.075753e+01, 1.152000e-02],
       [6.000000e+02, 1.403809e+01, 1.102000e-02],
       [6.500000e+02, 1.699469e+01, 1.103000e-02],
       [7.000000e+02, 1.944093e+01, 1.077000e-02]])
>>> print(unstructured[0, 0])
250.0

Table (ndarray)

Represents the table element in an equipment register.

comment class-attribute instance-attribute

comment: str = ''

A comment that is associated with the table.

header class-attribute instance-attribute

header: NDArray[void] = empty(0, dtype=object)

The header value of each column.

types class-attribute instance-attribute

types: NDArray[void] = empty(0, dtype=object)

The data type of each column.

units class-attribute instance-attribute

units: NDArray[void] = empty(0, dtype=object)

The unit of each column.

from_xml classmethod

from_xml(element: Element[str]) -> Table

Convert an XML element into a Table instance.

Parameters:

Name	Type	Description	Default
element	Element[str]	A table XML element from an equipment register.	required

Returns:

Type	Description
Table	A Table is an subclass of a numpy structured array, where the header is used as the field names. This allows for accessing a column by the header value rather than by the index of a column. If you prefer to work with unstructured data, call unstructured on the returned object.

Source code in src/msl/equipment/schema.py

@classmethod
def from_xml(cls, element: Element[str]) -> Table:
    """Convert an XML element into a [Table][msl.equipment.schema.Table] instance.

    Args:
        element: A [table][type_table] XML element from an equipment register.

    Returns:
        A [Table][msl.equipment.schema.Table] is an subclass of a numpy
            [structured array][structured_arrays], where the `header` is used as
            the *field names*. This allows for accessing a column by the header value rather than by
            the index of a column. If you prefer to work with unstructured data, call
            [unstructured][msl.equipment.schema.Table.unstructured] on the returned object.
    """
    booleans = {"True", "true", "TRUE", "1", b"True", b"true", b"TRUE", b"1"}

    def convert_bool(value: str | bytes) -> bool:
        # the value can be of type bytes for numpy < 2.0
        return value.strip() in booleans

    def strip_string(value: str | bytes) -> str:
        # the value can be of type bytes for numpy < 2.0
        stripped = value.strip()
        if isinstance(stripped, bytes):
            return stripped.decode()  # pragma: no cover
        return stripped

    # Schema forces order
    _type = [s.strip() for s in (element[0].text or "").split(",")]
    _unit = [s.strip() for s in (element[1].text or "").split(",")]
    _header = [s.strip() for s in (element[2].text or "").split(",")]
    _file = StringIO((element[3].text or "").strip())

    # must handle boolean column and string column separately
    conv: dict[int, Callable[[str | bytes], str | bool]] = {
        i: convert_bool for i, v in enumerate(_type) if v == "bool"
    }
    conv.update({i: strip_string for i, v in enumerate(_type) if v == "string"})

    dtype = np.dtype([(h, schema_numpy_map[t]) for h, t in zip(_header, _type)])
    data = np.loadtxt(_file, dtype=dtype, delimiter=",", converters=conv)  # type: ignore[arg-type]  # pyright: ignore[reportCallIssue, reportArgumentType, reportUnknownVariableType]
    data.setflags(write=False)  # pyright: ignore[reportUnknownMemberType]

    header = np.asarray(_header)
    header.setflags(write=False)  # make it readonly by default

    units = np.asarray(tuple(_unit), np.dtype([(h, object) for h in _header]))
    units.setflags(write=False)  # make it readonly by default

    assert data.dtype.fields is not None  # pyright: ignore[reportUnknownMemberType]  # noqa: S101
    types = np.asarray(tuple(v[0] for v in data.dtype.fields.values()), dtype=[(h, object) for h in _header])  # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType, reportUnknownArgumentType]
    types.setflags(write=False)  # make it readonly by default

    return cls(types=types, units=units, header=header, data=data, comment=element.attrib.get("comment", ""))  # pyright: ignore[reportUnknownArgumentType]

to_xml

to_xml() -> Element[str]

Convert the Table class into an XML element.

Returns:

Type	Description
Element[str]	The Table as an XML element.

Source code in src/msl/equipment/schema.py

def to_xml(self) -> Element[str]:
    """Convert the [Table][msl.equipment.schema.Table] class into an XML element.

    Returns:
        The [Table][msl.equipment.schema.Table] as an XML element.
    """
    attrib = {"comment": self.comment} if self.comment else {}
    e = Element("table", attrib=attrib)

    types = SubElement(e, "type")
    dtypes = [numpy_schema_map[t.char] for t in self.types.tolist()]
    types.text = ",".join(dtypes)

    units = SubElement(e, "unit")
    units.text = ",".join(self.units.tolist())

    header = SubElement(e, "header")
    header.text = ",".join(self.header)

    buffer = StringIO()
    newline = "\n" + " " * _Indent.table_data
    np.savetxt(buffer, self, fmt="%s", delimiter=",", newline=newline)
    data = SubElement(e, "data")
    data.text = buffer.getvalue().rstrip() + "\n" + " " * max(0, _Indent.table_data - len("<data>"))

    return e

unstructured

unstructured(
    *,
    dtype: DTypeLike | None = None,
    copy: bool = False,
    casting: Literal["no", "equiv", "safe", "same_kind", "unsafe"] = "unsafe"
) -> NDArray[Any]

Converts the structured array into an unstructured array.

See structured_to_unstructured for more details.

Parameters:

Name	Type	Description	Default
dtype	DTypeLike | None	The dtype of the output unstructured array.	None
copy	bool	If True, always return a copy. If False, a view is returned if possible.	False
casting	Literal['no', 'equiv', 'safe', 'same_kind', 'unsafe']	Controls what kind of data casting may occur. See the casting argument of astype for more details.	'unsafe'

Returns:

Type	Description
NDArray[Any]	The unstructured array. This method may return a numpy ndarray instance instead of a Table instance if the table consists of numbers and strings and the appropriate dtype is not specified.

Source code in src/msl/equipment/schema.py

def unstructured(
    self,
    *,
    dtype: DTypeLike | None = None,
    copy: bool = False,
    casting: Literal["no", "equiv", "safe", "same_kind", "unsafe"] = "unsafe",
) -> NDArray[_Any]:
    """Converts the structured array into an unstructured array.

    See [structured_to_unstructured][numpy.lib.recfunctions.structured_to_unstructured]
    for more details.

    Args:
        dtype: The _dtype_ of the output unstructured array.
        copy: If `True`, always return a copy. If `False`, a view is returned if possible.
        casting: Controls what kind of data casting may occur. See the *casting* argument of
            [astype][numpy.ndarray.astype] for more details.

    Returns:
        The unstructured array. This method may return a numpy [ndarray][numpy.ndarray] instance
            instead of a [Table][msl.equipment.schema.Table] instance if the table consists of
            numbers and strings and the appropriate `dtype` is not specified.
    """
    from numpy.lib.recfunctions import structured_to_unstructured  # noqa: PLC0415

    try:
        return structured_to_unstructured(self, dtype=dtype, copy=copy, casting=casting)
    except (TypeError, ValueError):
        return np.array(self.tolist(), dtype=object)