to_table

class to_table(df, file_path, name='df_feat', sep=None, random_state=None)[source]

Write an AAanalysis output table to a delimited text file with a metadata sidecar.

Two artifacts are written: the table itself (.csv or .tsv, without the row index) and a JSON sidecar named after it (<stem>.meta.json). The sidecar documents every exported column with the meaning, dtype, and allowed values from the shipped data schemas, and records the AAanalysis version and input fingerprint, so a table can still be read correctly long after the session that produced it.

Added in version 1.2.0.

Parameters:
  • df (pd.DataFrame) – Table to export, such as the CPP feature table (df_feat), a sequence table (df_seq), or the feature matrix X wrapped in a DataFrame. Its required columns must match the schema selected by name.

  • file_path (str) – Path of the table file to write. Its extension selects the format and must be .csv or .tsv; the directory must already exist.

  • name (str, default='df_feat') – Name of the data schema describing df, used to document the exported columns. One of the keys of the shipped schemas (see Data Schemas).

  • sep (str, optional) – Column separator. None derives it from the file extension (',' for .csv, a tab for .tsv). A .csv file also accepts ';'.

  • random_state (int, optional) – Seed under which df was computed, recorded in the provenance part of the sidecar. None means no seed was in effect.

Returns:

dict_metadata – The JSON-serializable sidecar record, with the following keys:

  • schema_version: version of the sidecar layout itself.

  • name, description: the schema df was documented against.

  • file_name, file_name_metadata, sep, n_rows, n_cols: how the table was written.

  • columns: per column, its meaning, contract dtype, written pandas dtype (dtype_pandas), and where defined its allowed values or range.

  • columns_undocumented: columns outside the schema contract (empty for a table that matches its schema).

  • categories, parts: the AAontology category and sequence part vocabularies the table’s values are drawn from.

  • provenance: the record of get_provenance(), including the package version and a fingerprint of df.

Return type:

dict

Notes

  • The row index is not written, so the exported table has no unnamed first column. Rows keep their order; compare a filtered frame after reset_index(drop=True).

  • Load the table back with pandas. Plain inference already restores the AAanalysis output tables exactly:

    df_feat = pd.read_csv("df_feat.csv")
    

    To restore the written dtypes regardless of inference, pass them from the sidecar:

    dict_metadata = json.load(open("df_feat.meta.json"))
    dtypes = {c: r["dtype_pandas"] for c, r in dict_metadata["columns"].items()}
    df_feat = pd.read_csv("df_feat.csv", dtype=dtypes)
    
  • Only delimited text is written. Columns holding non-scalar objects (lists, arrays, nested frames) are stringified by pandas and do not round-trip.

See also

Examples

AAanalysis output tables can be written to a delimited text file together with a JSON metadata sidecar using the to_table() function. The sidecar documents every exported column, so a table stays interpretable long after the session that produced it. Only df and file_path are required:

import json
import tempfile
from pathlib import Path

import pandas as pd
import aaanalysis as aa

aa.options["verbose"] = False
folder_out = Path(tempfile.mkdtemp())

# Load an example CPP feature table
df_feat = aa.load_features()
aa.display_df(df_feat, n_rows=10, show_shape=True)

# Export it together with its metadata sidecar
file_path = str(folder_out / "df_feat.csv")
dict_metadata = aa.to_table(df=df_feat, file_path=file_path)
DataFrame shape: (150, 15)
  feature category subcategory scale_name scale_description abs_auc abs_mean_dif mean_dif std_test std_ref p_val_mann_whitney p_val_fdr_bh positions feat_importance feat_importance_std
1 TMD_C_JMD_C-Seg...3,4)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.244000 0.103666 0.103666 0.106692 0.110506 0.000000 0.000000 31,32,33,34,35 0.970400 1.438918
2 TMD_C_JMD_C-Seg...3,4)-FINA910104 Conformation α-helix (C-cap) α-helix termination Helix terminati...n et al., 1991) 0.243000 0.085064 0.085064 0.098774 0.096946 0.000000 0.000000 31,32,33,34,35 0.000000 0.000000
3 TMD_C_JMD_C-Seg...6,9)-LEVM760105 Shape Side chain length Side chain length Radius of gyrat... (Levitt, 1976) 0.233000 0.137044 0.137044 0.161683 0.176964 0.000000 0.000001 32,33 1.554800 2.109848
4 TMD_C_JMD_C-Seg...3,4)-HUTJ700102 Energy Entropy Entropy Absolute entrop...Hutchens, 1970) 0.229000 0.098224 0.098224 0.106865 0.124608 0.000000 0.000001 31,32,33,34,35 3.111200 3.109955
5 TMD_C_JMD_C-Seg...6,9)-RADA880106 ASA/Volume Volume Accessible surface area (ASA) Accessible surf...olfenden, 1988) 0.223000 0.095071 0.095071 0.114758 0.132829 0.000000 0.000002 32,33 0.000000 0.000000
6 TMD_C_JMD_C-Seg...2,3)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.222000 0.058671 0.058671 0.064895 0.069547 0.000000 0.000001 27,28,29,30,31,32,33 0.000000 0.000000
7 TMD_C_JMD_C-Seg...4,5)-FAUJ880109 Energy Isoelectric point Number hydrogen bond donors Number of hydro...e et al., 1988) 0.215000 0.146661 0.146661 0.174609 0.188034 0.000000 0.000004 33,34,35,36 1.032400 1.510722
8 TMD_C_JMD_C-Seg...3,4)-JANJ780101 ASA/Volume Accessible surface area (ASA) ASA (folded protein) Average accessi...n et al., 1978) 0.215000 0.124317 0.124317 0.166309 0.153364 0.000000 0.000004 31,32,33,34,35 1.080400 1.296094
9 TMD_C_JMD_C-Seg...,10)-WILM950103 Polarity Hydrophobicity (interface) Hydrophobicity (interface) Hydrophobicity ...e et al., 1995) 0.212000 0.141305 -0.141305 0.168603 0.217235 0.000000 0.000005 33,34 1.747200 2.150664
10 TMD_C_JMD_C-Seg...6,9)-AURR980110 Conformation α-helix α-helix (middle) Normalized posi...ora-Rose, 1998) 0.211000 0.125350 0.125350 0.160819 0.174121 0.000000 0.000005 32,33 1.788800 2.700803

The table is written without the row index, so it reads back with plain pandas and the round-trip is lossless:

df_feat_read = pd.read_csv(file_path)
aa.display_df(df_feat_read, n_rows=10, show_shape=True)
df_feat_read.equals(df_feat)
DataFrame shape: (150, 15)
  feature category subcategory scale_name scale_description abs_auc abs_mean_dif mean_dif std_test std_ref p_val_mann_whitney p_val_fdr_bh positions feat_importance feat_importance_std
1 TMD_C_JMD_C-Seg...3,4)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.244000 0.103666 0.103666 0.106692 0.110506 0.000000 0.000000 31,32,33,34,35 0.970400 1.438918
2 TMD_C_JMD_C-Seg...3,4)-FINA910104 Conformation α-helix (C-cap) α-helix termination Helix terminati...n et al., 1991) 0.243000 0.085064 0.085064 0.098774 0.096946 0.000000 0.000000 31,32,33,34,35 0.000000 0.000000
3 TMD_C_JMD_C-Seg...6,9)-LEVM760105 Shape Side chain length Side chain length Radius of gyrat... (Levitt, 1976) 0.233000 0.137044 0.137044 0.161683 0.176964 0.000000 0.000001 32,33 1.554800 2.109848
4 TMD_C_JMD_C-Seg...3,4)-HUTJ700102 Energy Entropy Entropy Absolute entrop...Hutchens, 1970) 0.229000 0.098224 0.098224 0.106865 0.124608 0.000000 0.000001 31,32,33,34,35 3.111200 3.109955
5 TMD_C_JMD_C-Seg...6,9)-RADA880106 ASA/Volume Volume Accessible surface area (ASA) Accessible surf...olfenden, 1988) 0.223000 0.095071 0.095071 0.114758 0.132829 0.000000 0.000002 32,33 0.000000 0.000000
6 TMD_C_JMD_C-Seg...2,3)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.222000 0.058671 0.058671 0.064895 0.069547 0.000000 0.000001 27,28,29,30,31,32,33 0.000000 0.000000
7 TMD_C_JMD_C-Seg...4,5)-FAUJ880109 Energy Isoelectric point Number hydrogen bond donors Number of hydro...e et al., 1988) 0.215000 0.146661 0.146661 0.174609 0.188034 0.000000 0.000004 33,34,35,36 1.032400 1.510722
8 TMD_C_JMD_C-Seg...3,4)-JANJ780101 ASA/Volume Accessible surface area (ASA) ASA (folded protein) Average accessi...n et al., 1978) 0.215000 0.124317 0.124317 0.166309 0.153364 0.000000 0.000004 31,32,33,34,35 1.080400 1.296094
9 TMD_C_JMD_C-Seg...,10)-WILM950103 Polarity Hydrophobicity (interface) Hydrophobicity (interface) Hydrophobicity ...e et al., 1995) 0.212000 0.141305 -0.141305 0.168603 0.217235 0.000000 0.000005 33,34 1.747200 2.150664
10 TMD_C_JMD_C-Seg...6,9)-AURR980110 Conformation α-helix α-helix (middle) Normalized posi...ora-Rose, 1998) 0.211000 0.125350 0.125350 0.160819 0.174121 0.000000 0.000005 32,33 1.788800 2.700803
True

to_table() returns the sidecar record it wrote next to the table as <stem>.meta.json. Its top level states the version of the sidecar layout, the schema the table was documented against, and how the file was written:

{k: v for k, v in dict_metadata.items()
 if k not in ["columns", "columns_undocumented", "categories", "parts", "provenance"]}
{'schema_version': '1.0',
 'name': 'df_feat',
 'description': 'CPP feature table (the primary downstream contract; see also the simple DICT_DF_FEAT). One row per feature. Columns 1-13 are the canonical lower bound (LIST_COLS_FEAT); optional/dynamic columns follow in stable order.',
 'file_name': 'df_feat.csv',
 'file_name_metadata': 'df_feat.meta.json',
 'sep': ',',
 'n_rows': 150,
 'n_cols': 15}

The columns entry is the part that makes the export self-describing: every exported column carries its meaning, its contract dtype, the pandas dtype it was written with, and, where the contract defines them, its allowed values or numeric range:

df_columns = pd.DataFrame(dict_metadata["columns"]).T
aa.display_df(df_columns, n_rows=10, show_shape=True)
DataFrame shape: (15, 10)
  dtype required nullable unique description example validation dtype_pandas allowed_values range
feature str True False True Opaque PART-SPL... parse by hand. TMD_C_JMD_C-Seg...3,4)-KLEP840101 feature_id str nan nan
category str True False False AAontology cate...eature's scale. Energy nan str ['ASA/Volume', 'Composition', 'Conformation', 'Energy', 'Others', 'Polarity', 'Shape', 'Structure-Activity', 'Structure', 'Embeddings', 'PTMs', 'Functional sites'] nan
subcategory str True False False AAontology subcategory. Charge nan str nan nan
scale_name str True False False Human-readable scale name. Charge nan str nan nan
scale_description str True False False One-sentence sc...le description. Net charge (Kle... et al., 1984). nan str nan nan
abs_auc float True False False Absolute adjust...king statistic. 0.244000 nan float64 nan [-0.5, 0.5]
abs_mean_dif float True False False Absolute mean d...eference group. 0.104000 nan float64 nan [0, 1]
mean_dif float True False False Signed mean dif... the direction. 0.104000 nan float64 nan [-1, 1]
std_test float True False False Standard deviat...the test group. 0.107000 nan float64 nan [0, None]
std_ref float True False False Standard deviat...eference group. 0.110000 nan float64 nan [0, None]

The provenance entry is the record of get_provenance(): it identifies the exact environment the exported table was produced in, and fingerprints the table itself, so a later run can confirm it starts from the same data:

# The values identify one specific run, so only the recorded keys are shown here
list(dict_metadata["provenance"])
['aaanalysis_version',
 'python_version',
 'dependencies',
 'git_commit',
 'random_state',
 'deterministic',
 'input_hash']

Use name to select the data schema the columns are documented against. It defaults to 'df_feat'; any other output table, such as a sequence table, is exported under its own schema. Columns outside the selected contract are still described and listed in columns_undocumented:

df_seq = aa.load_dataset(name="DOM_GSEC", n=5)
dict_metadata_seq = aa.to_table(df=df_seq, file_path=str(folder_out / "df_seq.csv"),
                                name="df_seq")
print(dict_metadata_seq["columns_undocumented"])
aa.display_df(pd.DataFrame(dict_metadata_seq["columns"]).T, n_rows=10, show_shape=True)
['gene']
DataFrame shape: (9, 9)
  dtype required nullable unique description example dtype_pandas validation range
entry str True False True Unique protein/...nce identifier. P05067 str nan nan
gene unknown False True False Column outside ... by AAanalysis. nan str nan nan
sequence str False False False Amino acid sequ...e-letter code). MLPGLALLLL... str amino_acid_sequence nan
label int False False False Class/group lab...bels_* helpers. 1 int64 nan nan
tmd_start int False False False 1-based start p...-based format). 37 int64 nan [1, None]
tmd_stop int False False False 1-based stop po...-based format). 59 int64 nan [1, None]
jmd_n str False False False N-terminal juxt...-based format). NSPFYYDWHS str nan nan
tmd str False False False Transmembrane d...e-TMD formats). LQVGGLICAGVL str nan nan
jmd_c str False False False C-terminal juxt...-based format). KCKCKFGQKS str nan nan

sep sets the column separator and random_state records the seed the exported table was computed under. The separator defaults to the one implied by the file extension (a comma for .csv, a tab for .tsv), and a .csv file also accepts ';':

file_path_tsv = str(folder_out / "df_feat.tsv")
dict_metadata_tsv = aa.to_table(df=df_feat, file_path=file_path_tsv, name="df_feat",
                                sep="\t", random_state=42)
df_feat_tsv = pd.read_csv(file_path_tsv, sep="\t")
aa.display_df(df_feat_tsv, n_rows=10, show_shape=True)
dict_metadata_tsv["provenance"]["random_state"]
DataFrame shape: (150, 15)
  feature category subcategory scale_name scale_description abs_auc abs_mean_dif mean_dif std_test std_ref p_val_mann_whitney p_val_fdr_bh positions feat_importance feat_importance_std
1 TMD_C_JMD_C-Seg...3,4)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.244000 0.103666 0.103666 0.106692 0.110506 0.000000 0.000000 31,32,33,34,35 0.970400 1.438918
2 TMD_C_JMD_C-Seg...3,4)-FINA910104 Conformation α-helix (C-cap) α-helix termination Helix terminati...n et al., 1991) 0.243000 0.085064 0.085064 0.098774 0.096946 0.000000 0.000000 31,32,33,34,35 0.000000 0.000000
3 TMD_C_JMD_C-Seg...6,9)-LEVM760105 Shape Side chain length Side chain length Radius of gyrat... (Levitt, 1976) 0.233000 0.137044 0.137044 0.161683 0.176964 0.000000 0.000001 32,33 1.554800 2.109848
4 TMD_C_JMD_C-Seg...3,4)-HUTJ700102 Energy Entropy Entropy Absolute entrop...Hutchens, 1970) 0.229000 0.098224 0.098224 0.106865 0.124608 0.000000 0.000001 31,32,33,34,35 3.111200 3.109955
5 TMD_C_JMD_C-Seg...6,9)-RADA880106 ASA/Volume Volume Accessible surface area (ASA) Accessible surf...olfenden, 1988) 0.223000 0.095071 0.095071 0.114758 0.132829 0.000000 0.000002 32,33 0.000000 0.000000
6 TMD_C_JMD_C-Seg...2,3)-KLEP840101 Energy Charge Charge Net charge (Kle...n et al., 1984) 0.222000 0.058671 0.058671 0.064895 0.069547 0.000000 0.000001 27,28,29,30,31,32,33 0.000000 0.000000
7 TMD_C_JMD_C-Seg...4,5)-FAUJ880109 Energy Isoelectric point Number hydrogen bond donors Number of hydro...e et al., 1988) 0.215000 0.146661 0.146661 0.174609 0.188034 0.000000 0.000004 33,34,35,36 1.032400 1.510722
8 TMD_C_JMD_C-Seg...3,4)-JANJ780101 ASA/Volume Accessible surface area (ASA) ASA (folded protein) Average accessi...n et al., 1978) 0.215000 0.124317 0.124317 0.166309 0.153364 0.000000 0.000004 31,32,33,34,35 1.080400 1.296094
9 TMD_C_JMD_C-Seg...,10)-WILM950103 Polarity Hydrophobicity (interface) Hydrophobicity (interface) Hydrophobicity ...e et al., 1995) 0.212000 0.141305 -0.141305 0.168603 0.217235 0.000000 0.000005 33,34 1.747200 2.150664
10 TMD_C_JMD_C-Seg...6,9)-AURR980110 Conformation α-helix α-helix (middle) Normalized posi...ora-Rose, 1998) 0.211000 0.125350 0.125350 0.160819 0.174121 0.000000 0.000005 32,33 1.788800 2.700803
42