On-disk format#

Note

These docs are written for anndata 0.8+. Files written before this version may differ in some conventions, but will still be read by newer versions of the library.

AnnData objects are saved on disk to hierarchical array stores like HDF5 (via H5py) and zarr. This allows us to have very similar structures in disk and on memory.

As an example we’ll look into a typical .h5ad/ .zarr object that’s been through an analysis. The structures are largely equivalent, though there are a few minor differences when it comes to type encoding.

Elements#

>>> import h5py
>>> store = h5py.File("for-ondisk-docs/cart-164k-processed.h5ad", mode="r")
>>> list(store.keys())
['X', 'layers', 'obs', 'obsm', 'obsp', 'uns', 'var', 'varm', 'varp']

>>> import zarr
>>> store = zarr.open("for-ondisk-docs/cart-164k-processed.zarr", mode="r")
>>> list(store.keys())
['X', 'layers', 'obs', 'obsm', 'obsp', 'uns', 'var', 'varm', 'varp']

In general, AnnData objects are comprised of various types of elements. Each element is encoded as either an Array (or Dataset in hdf5 terminology) or a collection of elements (e.g. Group) in the store. We record the type of an element using the encoding-type and encoding-version keys in its attributes. For example, we can see that this file represents an AnnData object from its metadata:

>>> dict(store.attrs)
{'encoding-type': 'anndata', 'encoding-version': '0.1.0'}

Using this information, we’re able to dispatch onto readers for the different element types that you’d find in an anndata.

Element Specification#

  • An element can be any object within the storage hierarchy (typically an array or group) with associated metadata

  • An element MUST have a string-valued field "encoding-type" in its metadata

  • An element MUST have a string-valued field "encoding-version" in its metadata that can be evaluated to a version

Escaped keys#

/ separates groups in both HDF5 and zarr, and a zarr store commonly maps each key to a real file, so a key like a/b or a:b cannot be used as a child name. From the "0.14" write compatibility profile on, elements that would need such a key store their keys escaped instead, so a/b is stored as a%2Fb.

The escaped characters are the ones Windows forbids in file names, a superset of what POSIX and macOS forbid, so no file system rejects an escaped key as a path segment. Like zarr, anndata does not work around other limits of the store though: e.g. Foo and foo are the same file on case-insensitive file systems (the default on Windows and macOS), and Windows limits path lengths. They are all ASCII and nothing else is ever escaped, which is what keeps an escape sequence exactly three characters long.

Some keys are unusable even though every character in them is safe, CON and zarr.json for example. Those are wrapped rather than mangled, so they stay legible in a listing: CON is stored as %CON%. Wrapping covers every such key at once, since whatever it wraps is non-empty, contains a %, and begins and ends with one. Marking only one end would not do, because some keys are reserved by how they begin – NUL.txt would still name a device and __x would still start with __.

Nothing else can leave a key ending in %: a % can only ever be the first of an escape sequence’s three characters, and a literal one is itself escaped, so CON% is stored as CON%25. That is what lets a reader recognize a wrapped key, and why a writer may wrap every key it writes if it prefers not to recognize reserved keys at all.

Note that zarr additionally recommends restricting names to a-z, A-Z, 0-9, -, _ and .; anndata does not escape everything outside that set, since doing so would mangle e.g. non-ASCII gene names for no concrete gain.

Escaped key specification#

Which elements escape their keys, and how they signal it, is part of each encoding’s own specification – see mappings.

A key is reserved if it

  • is empty, consists only of periods (., .., …), starts with __, or is zarr.json – all reserved by the zarr v3 spec, or

  • ends in a period or a space, or

  • names a Windows device, i.e. taking the part before its first period and stripping trailing spaces from that yields, case-insensitively, one of CON, PRN, AUX, NUL, CONIN$, CONOUT$, COM1–COM9, COM¹, COM², COM³, LPT1–LPT9, LPT¹, LPT² or LPT³. The part is taken before the first period because such a name keeps referring to the device whatever extension follows it, so NUL.tar.gz is reserved as well (ntpath.isreserved implements this rule)

A key is escaped by

  • replacing each of / \ : * ? " < > |, the control characters 0x00–0x1F, and % itself with % and the two upper-case hexadecimal digits of its code point, and then

  • wrapping that result in a % on either end if necessary

while these rules apply:

  • A writer MUST wrap a key that is reserved, and SHOULD NOT wrap a key that is not reserved

Consequently, a reader of escaped keys MUST, in this order:

  1. remove the first and last character of a key that is at least two characters long and both starts and ends with a %, then

  2. replace each %XX with the character of code point 0xXX, and

  3. reject the key if escaping that result’s characters again does not yield the key from step 1, since no writer can have produced it (e.g. %41, %2f, or a%)

AnnData specification (v0.1.0)#

  • An AnnData object MUST be a group.

  • The group’s metadata MUST include entries: "encoding-type": "anndata", "encoding-version": "0.1.0".

  • An AnnData group MUST contain entries "obs" and "var", which MUST be dataframes (though this may only have an index with no columns).

  • The group MAY contain an entry X, which MUST be either a dense or sparse array and whose shape MUST be (n_obs, n_var)

  • The group MAY contain a mapping layers. Entries in layers MUST be dense or sparse arrays which have shapes (n_obs, n_var)

  • The group MAY contain a mapping obsm. Entries in obsm MUST be sparse arrays, dense arrays, or dataframes. These entries MUST have a first dimension of size n_obs

  • The group MAY contain a mapping varm. Entries in varm MUST be sparse arrays, dense arrays, or dataframes. These entries MUST have a first dimension of size n_var

  • The group MAY contain a mapping obsp. Entries in obsp MUST be sparse or dense arrays. The entries first two dimensions MUST be of size n_obs

  • The group MAY contain a mapping varp. Entries in varp MUST be sparse or dense arrays. The entries first two dimensions MUST be of size n_var

  • The group MAY contain a mapping uns. Entries in uns MUST be an anndata encoded type.

Dense arrays#

Dense numeric arrays have the most simple representation on disk, as they have native equivalents in H5py Datasets and zarr.Array\ s. We can see an example of this with dimensionality reductions stored in the obsm group:

>>> store["obsm/X_pca"]
<HDF5 dataset "X_pca": shape (164114, 50), type "<f4">

>>> store["obsm/X_pca"]
<zarr.core.Array '/obsm/X_pca' (164114, 50) float32 read-only>

>>> dict(store["obsm"]["X_pca"].attrs)
{'encoding-type': 'array', 'encoding-version': '0.2.0'}

Dense arrays specification (v0.2.0)#

  • Dense arrays MUST be stored in an Array object

  • Dense arrays MUST have the entries 'encoding-type': 'array' and 'encoding-version': '0.2.0' in their metadata

Sparse arrays#

Sparse arrays don’t have a native representations in HDF5 or Zarr, so we’ve defined our own based on their in-memory structure. Currently two sparse data formats are supported by AnnData objects, CSC and CSR (corresponding to scipy.sparse.csc_matrix and scipy.sparse.csr_matrix respectively). These formats represent a two-dimensional sparse array with three one-dimensional arrays, indptr, indices, and data.

Note

A full description of these formats is out of scope for this document, but are easy to find.

We represent a sparse array as a Group on-disk, where the kind and shape of the sparse array is defined in the Group’s attributes:

>>> dict(store["X"].attrs)
{'encoding-type': 'csr_matrix',
 'encoding-version': '0.1.0',
 'shape': [164114, 40145]}

The group contains three arrays:

>>> store["X"].visititems(print)
data <HDF5 dataset "data": shape (495079432,), type "<f4">
indices <HDF5 dataset "indices": shape (495079432,), type "<i4">
indptr <HDF5 dataset "indptr": shape (164115,), type "<i4">

>>> store["X"].visititems(print)
data <zarr.core.Array '/X/data' (495079432,) float32 read-only>
indices <zarr.core.Array '/X/indices' (495079432,) int32 read-only>
indptr <zarr.core.Array '/X/indptr' (164115,) int32 read-only>

Sparse array specification (v0.1.0)#

  • Each sparse array MUST be its own group

  • The group MUST contain arrays indices, indptr, and data

  • The group’s metadata MUST contain:

    • "encoding-type", which is set to "csr_matrix" or "csc_matrix" for compressed sparse row and compressed sparse column, respectively.

    • "encoding-version", which is set to "0.1.0"

    • "shape" which is an integer array of length 2 whose values are the sizes of the array’s dimensions

DataFrames#

DataFrames are saved as a columnar format in a group, so each column of a DataFrame is saved as a separate array. We save a little more information in the attributes here.

>>> dict(store["var"].attrs)
{'_index': 'ensembl_id',
 'column-order': ['highly_variable',
  'means',
  'variances',
  'variances_norm',
  'feature_is_filtered',
  'feature_name',
  'feature_reference',
  'feature_biotype',
  'mito'],
 'encoding-type': 'dataframe',
 'encoding-version': '0.2.0'}

These attributes identify the index of the dataframe, as well as the original order of the columns. Each column in this dataframe is encoded as its own array.

>>> store["var"].visititems(print)
ensembl_id <HDF5 dataset "ensembl_id": shape (40145,), type "|O">
feature_biotype <HDF5 group "/var/feature_biotype" (2 members)>
feature_biotype/categories <HDF5 dataset "categories": shape (1,), type "|O">
feature_biotype/codes <HDF5 dataset "codes": shape (40145,), type "|i1">
feature_is_filtered <HDF5 dataset "feature_is_filtered": shape (40145,), type "|b1">
...

>>> store["var"].visititems(print)
ensembl_id <zarr.core.Array '/var/ensembl_id' (40145,) object read-only>
feature_biotype <zarr.hierarchy.Group '/var/feature_biotype' read-only>
feature_biotype/categories <zarr.core.Array '/var/feature_biotype/categories' (1,) object read-only>
feature_biotype/codes <zarr.core.Array '/var/feature_biotype/codes' (40145,) int8 read-only>
feature_is_filtered <zarr.core.Array '/var/feature_is_filtered' (40145,) bool read-only>
...

>>> dict(store["var"]["feature_name"].attrs)
{'encoding-type': 'categorical', 'encoding-version': '0.2.0', 'ordered': False}

>>> dict(store["var"]["feature_is_filtered"].attrs)
{'encoding-type': 'array', 'encoding-version': '0.2.0'}

Dataframe Specification (v0.2.0)#

  • A dataframe MUST be stored as a group

  • The group’s metadata:

    • MUST contain the field "_index", whose value is the key of the array to be used as an index/ row labels

    • MUST contain encoding metadata "encoding-type": "dataframe", "encoding-version": "0.2.0"

    • MUST contain "column-order" an array of strings denoting the order of column entries

  • The group MUST contain an array for the index

  • Each entry in the group MUST correspond to an array with equivalent first dimensions

  • Each entry SHOULD share chunk sizes (in the HDF5 or zarr container)

Mappings#

Mappings are simply stored as Groups on disk. These are distinct from DataFrames and sparse arrays since they don’t have any special attributes. A Group is created for any Mapping in the AnnData object, including the standard obsm, varm, layers, and uns. Notably, this definition is used recursively within uns:

>>> store["uns"].visititems(print)
[...]
pca <HDF5 group "/uns/pca" (3 members)>
pca/variance <HDF5 dataset "variance": shape (50,), type "<f8">
pca/variance_ratio <HDF5 dataset "variance_ratio": shape (50,), type "<f8">
[...]

>>> store["uns"].visititems(print)
[...]
pca <zarr.hierarchy.Group '/uns/pca' read-only>
pca/variance <zarr.core.Array '/uns/pca/variance' (50,) float64 read-only>
pca/variance_ratio <zarr.core.Array '/uns/pca/variance_ratio' (50,) float64 read-only>
[...]

Mapping specifications (v0.1.0)#

  • Each mapping MUST be its own group

  • The group’s metadata MUST contain the encoding metadata "encoding-type": "dict", "encoding-version": "0.1.0"

Mapping specifications (v0.2.0)#

Identical to v0.1.0, except that the child keys are escaped.

  • In a mapping with "encoding-version": "0.2.0", every child key is escaped, and a reader MUST unescape each of them to recover the mapping’s keys

  • A mapping containing a key that needs escaping MUST be written as "0.2.0"

  • A mapping whose keys need no escaping SHOULD be written as "0.1.0", so files that don’t use the feature stay readable by older implementations

Accessors#

anndata.acc accessors – references to a vector or container within an AnnData, such as A.obsm["pca"][0] – are stored as their to_json() form – a list of strings, integers and nulls described by acc-schema-v1.json – serialized to a JSON string.

>>> store["uns/ref"]  # uns["ref"] == A.obsm["pca"][0]
<HDF5 dataset "ref": shape (), type "|O">
>>> store["uns/ref"].asstr()[()]
'["obsm","pca",0]'

Accessor specifications (v0.1.0)#

  • Each accessor MUST be stored as a scalar string, i.e. a 0-dimensional array of variable-length UTF-8 strings

  • The array’s metadata MUST contain the encoding metadata "encoding-type": "accessor", "encoding-version": "0.1.0"

  • The string MUST be a JSON document that validates against #/$defs/ref or #/$defs/acc of acc-schema-v1.json

In zarr v3, this is byte-for-byte the layout of the json data type (zarr-developers/zarr-extensions#9) (which also uses the vlen-utf8 codec), so only the data_type metadata will need to change once zarr implementations support it.

Scalars#

Zero dimensional arrays are used for scalar values (i.e. single values like strings, numbers or booleans). These should only occur inside of uns, and are commonly saved parameters:

>>> store["uns/neighbors/params"].visititems(print)
method <HDF5 dataset "method": shape (), type "|O">
metric <HDF5 dataset "metric": shape (), type "|O">
n_neighbors <HDF5 dataset "n_neighbors": shape (), type "<i8">
random_state <HDF5 dataset "random_state": shape (), type "<i8">

>>> store["uns/neighbors/params"].visititems(print)
method <zarr.core.Array '/uns/neighbors/params/method' () <U4 read-only>
metric <zarr.core.Array '/uns/neighbors/params/metric' () <U9 read-only>
n_neighbors <zarr.core.Array '/uns/neighbors/params/n_neighbors' () int64 read-only>
random_state <zarr.core.Array '/uns/neighbors/params/random_state' () int64 read-only>

>>> store["uns/neighbors/params/metric"][()]
'euclidean'
>>> dict(store["uns/neighbors/params/metric"].attrs)
{'encoding-type': 'string', 'encoding-version': '0.2.0'}

Scalar specification (v0.2.0)#

  • Scalars MUST be written as a 0 dimensional array

  • Numeric scalars

    • MUST have "encoding-type": "numeric-scalar", "encoding-version": "0.2.0" in their metadata

    • MUST be a single numeric value, including boolean, unsigned integer, signed integer, floating point, or complex floating point

  • String scalars

    • MUST have "encoding-type": "string", "encoding-version": "0.2.0" in their metadata

    • In zarr, scalar strings MUST be stored as a fixed length unicode dtype

    • In HDF5, scalar strings MUST be stored as a variable length utf-8 encoded string dtype

Categorical arrays#

>>> categorical = store["obs"]["development_stage"]
>>> dict(categorical.attrs)
{'encoding-type': 'categorical', 'encoding-version': '0.2.0', 'ordered': False}

Discrete values can be efficiently represented with categorical arrays (similar to factors in R). These arrays encode the values as small width integers (codes), which map to the original label set (categories). Each entry in the codes array is the zero-based index of the encoded value in the categories array. To represent a missing value, a code of -1 is used. We store these two arrays separately.

>>> categorical.visititems(print)
categories <HDF5 dataset "categories": shape (7,), type "|O">
codes <HDF5 dataset "codes": shape (164114,), type "|i1">

>>> categorical.visititems(print)
categories <zarr.core.Array '/obs/development_stage/categories' (7,) object read-only>
codes <zarr.core.Array '/obs/development_stage/codes' (164114,) int8 read-only>

Categorical array specification (v0.2.0)#

  • Categorical arrays MUST be stored as a group

  • The group’s metadata MUST contain the encoding metadata "encoding-type": "categorical", "encoding-version": "0.2.0"

  • The group’s metadata MUST contain the boolean valued field "ordered", which indicates whether the categories are ordered

  • The group MUST contain an integer valued array named "codes" whose maximum value is the number of categories - 1

    • The "codes" array MAY contain signed integer values. If so, the code -1 denotes a missing value

  • The group MUST contain an array called "categories"

String arrays#

Arrays of strings are handled differently than numeric arrays since numpy doesn’t really have a good way of representing arrays of unicode strings. anndata assumes strings are text-like data, so it uses a variable length encoding.

>>> store["var"][store["var"].attrs["_index"]]
<HDF5 dataset "ensembl_id": shape (40145,), type "|O">

>>> store["var"][store["var"].attrs["_index"]]
<zarr.core.Array '/var/ensembl_id' (40145,) object read-only>

>>> dict(categorical["categories"].attrs)
{'encoding-type': 'string-array', 'encoding-version': '0.2.0'}

String array specifications (v0.2.0)#

  • String arrays MUST be stored in arrays

  • The arrays’s metadata MUST contain the encoding metadata "encoding-type": "string-array", "encoding-version": "0.2.0"

  • In zarr, string arrays MUST be stored using numcodecs’ VLenUTF8 codec

  • In HDF5, string arrays MUST be stored using the variable length string data type, with a utf-8 encoding

Nullable integers, booleans, and strings#

We support IO with Pandas nullable integer, boolean, and string arrays. We represent these on disk similar to numpy masked arrays, julia nullable arrays, or arrow validity bitmaps (see #504 for more discussion). That is, we store an indicator array (or mask) of null values alongside the array of all values.

>>> from anndata import write_elem
>>> null_store = h5py.File("tmp.h5", mode="w")
>>> int_array = pd.array([1, None, 3, 4])
>>> int_array
<IntegerArray>
[1, <NA>, 3, 4]
Length: 4, dtype: Int64

>>> write_elem(null_store, "nullable_integer", int_array)

>>> null_store.visititems(print)
nullable_integer <HDF5 group "/nullable_integer" (2 members)>
nullable_integer/mask <HDF5 dataset "mask": shape (4,), type "|b1">
nullable_integer/values <HDF5 dataset "values": shape (4,), type "<i8">

>>> from anndata import write_elem
>>> null_store = zarr.open()
>>> int_array = pd.array([1, None, 3, 4])
>>> int_array
<IntegerArray>
[1, <NA>, 3, 4]
Length: 4, dtype: Int64

>>> write_elem(null_store, "nullable_integer", int_array)

>>> null_store.visititems(print)
nullable_integer <zarr.hierarchy.Group '/nullable_integer'>
nullable_integer/mask <zarr.core.Array '/nullable_integer/mask' (4,) bool>
nullable_integer/values <zarr.core.Array '/nullable_integer/values' (4,) int64>

>>> dict(null_store["nullable_integer"].attrs)
{'encoding-type': 'nullable-integer', 'encoding-version': '0.1.0'}

Nullable integer specifications (v0.1.0)#

  • Nullable integers MUST be stored as a group

  • The group’s attributes MUST contain the encoding metadata "encoding-type": "nullable-integer", "encoding-version": "0.1.0"

  • The group MUST contain an integer valued array under the key "values"

  • The group MUST contain an boolean valued array under the key "mask"

Nullable boolean specifications (v0.1.0)#

  • Nullable booleans MUST be stored as a group

  • The group’s attributes MUST contain the encoding metadata "encoding-type": "nullable-boolean", "encoding-version": "0.1.0"

  • The group MUST contain an boolean valued array under the key "values"

  • The group MUST contain an boolean valued array under the key "mask"

  • The "values" and "mask" arrays MUST be the same shape

Nullable string specifications (v0.1.0)#

  • Nullable strings MUST be stored as a group

  • The group’s attributes MUST contain the encoding metadata "encoding-type": "nullable-string-array", "encoding-version": "0.1.0"

  • The group’s attributes MAY contain "na-value" as an indicator for missing value semantics with the possible value "NA" or "NaN" described in Missing value semantics, and the default being "NA"

  • The group MUST contain a string valued array under the key "values"

  • The group MUST contain a boolean valued array under the key "mask"

  • The "values" and "mask" arrays MUST be the same shape

Missing value semantics#

If available in the runtime data model, the following values representing missing value semantics are defined. See the individual specification versions for elements that support them.

"NA" means that a comparison between a missing value and a defined value produces a missing value (e.g. "x"==NA → NA). "NaN" means that a comparison between a missing value and a defined value produces a binary result (e.g. "x"==NaN → false).

After reading an element supporting missing value semantics, it SHOULD behave according to the specified semantics if the runtime data model allows for it. If no semantics are specified in the representation, the default semantics from the element specification SHOULD be used. When the encoded semantics value is unknown to the implementation, reading the element SHOULD NOT produce an error, and the implementation may freely choose the semantics.

For element specifications supporting missing value semantics, the semantics value most closely matching the source data model SHOULD be written. If the source data model instead behaves very differently from all currently defined ones, please reach out to get a new value for the behavior defined here.

Examples#

Pandas’ string arrays support both "NA" and "NaN" semantics, while R’s character array only supports "NA" semantics.

Therefore an R implementation SHOULD write a character array with "na-value": "NA", and when reading a "nullable-string-array", it MAY choose to read it into a character array, ignoring the "na-value" attribute (since character arrays can not be made to behave with "NaN" semantics).

AwkwardArrays#

Warning

Experimental

Support for ragged arrays via awkward array is considered experimental under the 0.9.0 release series. Please direct feedback on it’s implementation to scverse/anndata.

Ragged arrays are supported in anndata through the Awkward Array library. For storage on disk, we break down the awkward array into it’s constituent arrays using ak.to_buffers then writing these arrays using anndata’s methods.

>>> store["varm/transcript"].visititems(print)
node1-mask <HDF5 dataset "node1-mask": shape (5019,), type "|u1">
node10-data <HDF5 dataset "node10-data": shape (250541,), type "<i8">
node11-mask <HDF5 dataset "node11-mask": shape (5019,), type "|u1">
node12-offsets <HDF5 dataset "node12-offsets": shape (40146,), type "<i8">
node13-mask <HDF5 dataset "node13-mask": shape (250541,), type "|i1">
node14-data <HDF5 dataset "node14-data": shape (250541,), type "<i8">
node16-offsets <HDF5 dataset "node16-offsets": shape (40146,), type "<i8">
node17-data <HDF5 dataset "node17-data": shape (602175,), type "|u1">
node2-offsets <HDF5 dataset "node2-offsets": shape (40146,), type "<i8">
node3-data <HDF5 dataset "node3-data": shape (600915,), type "|u1">
node4-mask <HDF5 dataset "node4-mask": shape (5019,), type "|u1">
node5-offsets <HDF5 dataset "node5-offsets": shape (40146,), type "<i8">
node6-data <HDF5 dataset "node6-data": shape (59335,), type "|u1">
node7-mask <HDF5 dataset "node7-mask": shape (5019,), type "|u1">
node8-offsets <HDF5 dataset "node8-offsets": shape (40146,), type "<i8">
node9-mask <HDF5 dataset "node9-mask": shape (250541,), type "|i1">

>>> store["varm/transcript"].visititems(print)
node1-mask <zarr.core.Array '/varm/transcript/node1-mask' (5019,) uint8 read-only>
node10-data <zarr.core.Array '/varm/transcript/node10-data' (250541,) int64 read-only>
node11-mask <zarr.core.Array '/varm/transcript/node11-mask' (5019,) uint8 read-only>
node12-offsets <zarr.core.Array '/varm/transcript/node12-offsets' (40146,) int64 read-only>
node13-mask <zarr.core.Array '/varm/transcript/node13-mask' (250541,) int8 read-only>
node14-data <zarr.core.Array '/varm/transcript/node14-data' (250541,) int64 read-only>
node16-offsets <zarr.core.Array '/varm/transcript/node16-offsets' (40146,) int64 read-only>
node17-data <zarr.core.Array '/varm/transcript/node17-data' (602175,) uint8 read-only>
node2-offsets <zarr.core.Array '/varm/transcript/node2-offsets' (40146,) int64 read-only>
node3-data <zarr.core.Array '/varm/transcript/node3-data' (600915,) uint8 read-only>
node4-mask <zarr.core.Array '/varm/transcript/node4-mask' (5019,) uint8 read-only>
node5-offsets <zarr.core.Array '/varm/transcript/node5-offsets' (40146,) int64 read-only>
node6-data <zarr.core.Array '/varm/transcript/node6-data' (59335,) uint8 read-only>
node7-mask <zarr.core.Array '/varm/transcript/node7-mask' (5019,) uint8 read-only>
node8-offsets <zarr.core.Array '/varm/transcript/node8-offsets' (40146,) int64 read-only>
node9-mask <zarr.core.Array '/varm/transcript/node9-mask' (250541,) int8 read-only>

The length of the array is saved to it’s own "length" attribute, while metadata for the array structure is serialized and saved to the “form” attribute.

>>> dict(store["varm/transcript"].attrs)
{'encoding-type': 'awkward-array',
 'encoding-version': '0.1.0',
 'form': '{"class": "RecordArray", "fields": ["tx_id", "seq_name", '
         '"exon_seq_start", "exon_seq_end", "ensembl_id"], "contents": '
         '[{"class": "BitMaskedArray", "mask": "u8", "valid_when": true, '
         '"lsb_order": true, "content": {"class": "ListOffsetArray", '
         '"offsets": "i64", "content": {"class": "NumpyArray", "primitive": '
         '"uint8", "inner_shape": [], "parameters": {"__array__": "char"}, '
         '"form_key": "node3"}, "parameters": {"__array__": "string"}, '
         '"form_key": "node2"}, "parameters": {}, "form_key": "node1"}, '
        ...
 'length': 40145}

These can be read back as awkward arrays using the ak.from_buffers function:

>>> import awkward as ak
>>> from anndata.io import read_elem
>>> awkward_group = store["varm/transcript"]
>>> ak.from_buffers(
...     awkward_group.attrs["form"],
...     awkward_group.attrs["length"],
...     {k: read_elem(v) for k, v in awkward_group.items()}
... )
>>> transcript_models[:5]
[{tx_id: 'ENST00000450305', seq_name: '1', exon_seq_start: [...], ...},
 {tx_id: 'ENST00000488147', seq_name: '1', exon_seq_start: [...], ...},
 {tx_id: 'ENST00000473358', seq_name: '1', exon_seq_start: [...], ...},
 {tx_id: 'ENST00000477740', seq_name: '1', exon_seq_start: [...], ...},
 {tx_id: 'ENST00000495576', seq_name: '1', exon_seq_start: [...], ...}]
-----------------------------------------------------------------------
type: 5 * {
    tx_id: ?string,
    seq_name: ?string,
    exon_seq_start: option[var * ?int64],
    exon_seq_end: option[var * ?int64],
    ensembl_id: ?string
}
>>> transcript_models[0]
{tx_id: 'ENST00000450305',
 seq_name: '1',
 exon_seq_start: [12010, 12179, 12613, 12975, 13221, 13453],
 exon_seq_end: [12057, 12227, 12697, 13052, 13374, 13670],
 ensembl_id: 'ENSG00000223972'}
------------------------------------------------------------
type: {
    tx_id: ?string,
    seq_name: ?string,
    exon_seq_start: option[var * ?int64],
    exon_seq_end: option[var * ?int64],
    ensembl_id: ?string
}