NumericalFeature.feature_matrix

static NumericalFeature.feature_matrix(features, dict_num_parts, df_parts, df_scales=None, n_jobs=1)[source]

Create the numerical-mode feature matrix X for given feature ids and per-part tensors.

Numerical analog of SequenceFeature.feature_matrix(): for each sample and each feature id, reconstructs the feature value from the pre-sliced per-residue tensors in dict_num_parts (PLM embeddings, structure, annotations, …) instead of an amino-acid-to-scale lookup, so per-residue context is preserved. The result is the numerical input X for a downstream model or for NumericalFeature.filter_correlation(). This is the missing step that turns CPP.run_num()-selected features back into a model matrix.

Added in version 1.1.0.

Parameters:

  • features (array-like, shape (n_features,) or pd.DataFrame) – Ids of features ('PART-SPLIT-SCALE') for which the matrix of feature values should be created. Alternatively, a df_feat DataFrame (e.g. from CPP.run_num()), in which case its 'feature' column is used.

  • dict_num_parts (dict[str, np.ndarray]) – Per-part NaN-padded numerical tensors, as produced by NumericalFeature.get_parts() and consumed by CPP.run_num(). Each value has shape (n_samples, L_part_max, D), row-aligned across parts; must cover every part referenced in features.

  • df_parts (pd.DataFrame, shape (n_samples, n_parts)) – The string df_parts returned alongside dict_num_parts by NumericalFeature.get_parts() (row-aligned with it). Supplies each part’s real residue length via the exact same helper CPP.run_num() uses internally (non-gap character count), so every split lands on the residues run_num selected. Its columns must cover every part in dict_num_parts.

  • df_scales (pd.DataFrame, shape (n_letters, n_scales), optional) – DataFrame whose columns name the D dimensions of dict_num_parts (the same df_scales used to construct the CPP for CPP.run_num()); its column order defines the SCALE -> D-index mapping. Its row (amino acid) values are unused in numerical mode. Default from load_scales() unless specified in options['df_scales'] — pass your own when the D axis is a custom (e.g. embedding) space, since the default AA-scale set will not match a custom D.

  • n_jobs (int, None, or -1, default=1) – Number of CPU cores (>=1) used for multiprocessing. If None, the number is optimized automatically. If -1, the number is set to all available cores. Overridden by options['n_jobs'] when set.

Returns:

X – Feature matrix containing feature values for samples. Column i corresponds to feature i in features; values are byte-identical to those CPP.run_num() computed for the same feature ids — both take per-part lengths from the same df_parts, so they select identical residues for every split.

Return type:

array-like, shape (n_samples, n_features)

Raises:

ValueError – If dict_num_parts is empty or has inconsistent / non-3D tensors, if df_parts is missing a part column or its row count / real lengths do not match dict_num_parts, if the D axis does not match len(df_scales.columns), if a feature id is malformed or references a part/scale absent from dict_num_parts / df_scales, or if a feature’s split selects only padded (all-NaN) residues (producing a NaN value).

Notes

  • Mapping ``positions`` back to ``dict_num_parts`` — do not index with it. The 'positions' column of a CPP.run_num() df_feat is a display numbering in TMD-JMD coordinate space: positions are offset by the JMD length and use the start / tmd_len / jmd_n_len / jmd_c_len shown at run time (e.g. a jmd_n_len=10 TMD is numbered 21..30, decoupled from the actual per-sample TMD length). These numbers do not directly index the (L, D) per-part array. feature_matrix therefore does not read positions; it reconstructs each value the same way CPP.run_num() does — by re-applying the SPLIT encoded in the feature id to the part’s 0-based residue axis (arange(L_part), where L_part is the sample’s real residue count taken from df_parts), selecting the SCALE column along D, and averaging the selected residues (nanmean, rounded to 5 decimals). Because the split is applied to the residue axis and not to the display numbering, X lines up with run_num by construction.

  • Real per-part lengths come from ``df_parts``, via the very same length rule (non-gap character count of df_parts) that CPP.run_num() applies internally — not inferred from the tensor’s NaN padding. Passing the df_parts that NumericalFeature.get_parts() returns alongside dict_num_parts therefore makes X identical to run_num in every case, including when a real residue is all-NaN across D (an unresolved structure position or a masked embedding): its length is still counted from the string, exactly as run_num counts it.

  • A feature whose split selects only padded (all-NaN) residues yields a NaN value; this raises a ValueError rather than returning a silently-NaN column.

  • Unlike SequenceFeature.feature_matrix() there is no accept_gaps option: a numeric tensor carries no gap characters, and NaN padding is ignored via nanmean automatically.

See also

Examples

NumericalFeature.feature_matrix turns :meth:CPP.run_num-selected features back into a model matrix X. It reconstructs each feature value from the per-residue tensors in dict_num_parts (the numerical analog of :meth:SequenceFeature.feature_matrix), preserving the per-residue context that per-AA-averaged sequence features discard.

We first build a small per-residue dict_num (here a synthetic [0, 1]-normalized embedding), slice it into parts with :meth:NumericalFeature.get_parts, and run :meth:CPP.run_num to obtain a df_feat of selected features.

import numpy as np
import pandas as pd
import aaanalysis as aa
import aaanalysis.utils as ut
aa.options["verbose"] = False

# Tiny example: 8 proteins, a D=8 per-residue embedding, two classes
n_samples, D = 8, 8
rng = np.random.default_rng(0)
seqs = ["ACDEFGHIKLMNPQRSTVWY" * 3] * n_samples          # length 60
df_seq = pd.DataFrame({"entry": [f"P{i}" for i in range(n_samples)], "sequence": seqs,
                       "tmd_start": 11, "tmd_stop": 50})
labels = [1, 1, 1, 1, 0, 0, 0, 0]
dict_num = {e: rng.random((60, D)) for e in df_seq["entry"]}   # already [0, 1]-normalized

# df_scales NAMES the D dimensions (its amino-acid values are unused in numerical mode);
# df_cat assigns each dimension a category for the CPP redundancy filter.
dim_names = [f"dim{i}" for i in range(D)]
df_scales = pd.DataFrame(np.zeros((20, D)), index=list("ACDEFGHIKLMNPQRSTVWY"), columns=dim_names)
df_cat = pd.DataFrame({ut.COL_SCALE_ID: dim_names, ut.COL_CAT: ["Embedding"] * D,
                       ut.COL_SUBCAT: [f"block{i // 4}" for i in range(D)],
                       ut.COL_SCALE_NAME: dim_names, ut.COL_SCALE_DES: dim_names})

nf = aa.NumericalFeature()
df_parts, dict_num_parts = nf.get_parts(df_seq=df_seq, dict_num=dict_num)
cpp = aa.CPP(df_parts=df_parts, df_scales=df_scales, df_cat=df_cat, verbose=False)
df_feat = cpp.run_num(dict_num_parts=dict_num_parts, labels=labels, n_filter=15, n_jobs=1)
aa.display_df(df_feat, n_rows=10, show_shape=True)

DataFrame shape: (15, 13)
  feature category subcategory scale_name scale_description abs_auc abs_mean_dif mean_dif std_test std_ref p_val_mann_whitney p_val_fdr_bh positions
1 JMD_N_TMD_N-Pat...ern(C,2,6)-dim3 Embedding block0 dim3 dim3 0.500000 0.558000 -0.558000 0.094000 0.169000 0.020921 1.000000 15,19
2 JMD_N_TMD_N-Pat...(C,2,6,10)-dim3 Embedding block0 dim3 dim3 0.500000 0.416000 -0.416000 0.128000 0.110000 0.020921 0.067909 11,15,19
3 JMD_N_TMD_N-Pat...n(N,10,14)-dim6 Embedding block1 dim6 dim6 0.500000 0.407000 0.407000 0.159000 0.108000 0.020921 0.066813 10,14
4 TMD-Pattern(C,9,12)-dim2 Embedding block0 dim2 dim2 0.500000 0.405000 0.405000 0.103000 0.108000 0.020921 0.072042 19,22
5 TMD_C_JMD_C-Pat...rn(N,9,12)-dim2 Embedding block0 dim2 dim2 0.500000 0.405000 0.405000 0.103000 0.108000 0.020921 0.075317 29,32
6 JMD_N_TMD_N-Pat...ern(N,2,6)-dim3 Embedding block0 dim3 dim3 0.500000 0.380000 -0.380000 0.084000 0.138000 0.020921 0.109011 2,6
7 TMD_C_JMD_C-Seg...ent(10,12)-dim0 Embedding block0 dim0 dim0 0.500000 0.378000 0.378000 0.102000 0.158000 0.020921 0.104872 36
8 TMD_C_JMD_C-Pat...ern(N,4,7)-dim3 Embedding block0 dim3 dim3 0.500000 0.372000 -0.372000 0.142000 0.098000 0.020921 0.101035 24,27
9 TMD-Pattern(C,11,15)-dim7 Embedding block1 dim7 dim7 0.500000 0.366000 0.366000 0.115000 0.108000 0.020921 0.099817 16,20
10 TMD_C_JMD_C-Pat...rn(N,6,10)-dim7 Embedding block1 dim7 dim7 0.500000 0.366000 0.366000 0.115000 0.108000 0.020921 0.092054 26,30

The required inputs are features (the 'feature' ids), dict_num_parts (the per-part tensors from :meth:get_parts), and df_parts (the string parts returned alongside them, which supply each part’s real residue length exactly as :meth:CPP.run_num derives it); df_scales names the D dimensions so the SCALE token of each feature id maps to the right column of the tensor. X has shape (n_samples, n_features) and its values are byte-identical to those :meth:CPP.run_num computed for the same feature ids:

features = df_feat["feature"].to_list()
X = nf.feature_matrix(features=features, dict_num_parts=dict_num_parts, df_parts=df_parts, df_scales=df_scales)
print(f"n samples: {X.shape[0]}")
print(f"n features: {X.shape[1]}")
print(f"Shape of X: {X.shape}")
aa.display_df(pd.DataFrame(X, index=df_parts.index, columns=features), n_rows=10, show_shape=True)

n samples: 8
n features: 15
Shape of X: (8, 15)
DataFrame shape: (8, 15)
  JMD_N_TMD_N-Pattern(C,2,6)-dim3 JMD_N_TMD_N-Pattern(C,2,6,10)-dim3 JMD_N_TMD_N-Pattern(N,10,14)-dim6 TMD-Pattern(C,9,12)-dim2 TMD_C_JMD_C-Pattern(N,9,12)-dim2 JMD_N_TMD_N-Pattern(N,2,6)-dim3 TMD_C_JMD_C-Segment(10,12)-dim0 TMD_C_JMD_C-Pattern(N,4,7)-dim3 TMD-Pattern(C,11,15)-dim7 TMD_C_JMD_C-Pattern(N,6,10)-dim7 TMD_C_JMD_C-Segment(4,11)-dim2 TMD_C_JMD_C-Segment(5,14)-dim2 TMD_C_JMD_C-Segment(5,15)-dim2 TMD_C_JMD_C-Pattern(C,8,11)-dim2 TMD_C_JMD_C-Pattern(N,5,9)-dim2
entry                              
P0 0.367470 0.570470 0.781660 0.737920 0.737920 0.170320 0.683140 0.093180 0.815300 0.815300 0.560270 0.560270 0.560270 0.727150 0.676630
P1 0.113730 0.226170 0.800460 0.561060 0.561060 0.158470 0.617600 0.359630 0.953920 0.953920 0.828670 0.828670 0.828670 0.620380 0.538700
P2 0.175700 0.368770 0.460770 0.555350 0.555350 0.330540 0.884600 0.485440 0.759480 0.759480 0.580690 0.580690 0.580690 0.758620 0.760320
P3 0.215880 0.299710 0.487840 0.451090 0.451090 0.104720 0.662400 0.286620 0.632170 0.632170 0.614800 0.614800 0.614800 0.403140 0.532370
P4 0.958880 0.961280 0.297240 0.107260 0.107260 0.561060 0.247740 0.678580 0.526940 0.526940 0.259530 0.259530 0.259530 0.265300 0.090950
P5 0.524310 0.664460 0.210840 0.105990 0.105990 0.356340 0.503890 0.761880 0.528160 0.528160 0.320200 0.320200 0.320200 0.235030 0.404810
P6 0.724700 0.744480 0.338570 0.112930 0.112930 0.729340 0.119130 0.754070 0.368070 0.368070 0.283820 0.283820 0.283820 0.218810 0.149100
P7 0.896130 0.757080 0.055480 0.358330 0.358330 0.639150 0.463780 0.518070 0.274560 0.274560 0.281440 0.281440 0.281440 0.368220 0.455140

Instead of a list of feature ids, the df_feat DataFrame can be passed directly as features — its 'feature' column is used automatically, so features=df_feat is equivalent to features=list(df_feat['feature']):

X_from_df_feat = nf.feature_matrix(features=df_feat, dict_num_parts=dict_num_parts, df_parts=df_parts, df_scales=df_scales)
print(f"Shape of X: {X_from_df_feat.shape}")
print(f"Identical to list-of-ids result: {np.array_equal(X, X_from_df_feat)}")

Shape of X: (8, 15)
Identical to list-of-ids result: True

df_scales names the D dimensions of dict_num_parts and must be the same df_scales used to construct the :class:CPP for :meth:run_num; its column order defines the SCALE → dimension mapping. When omitted it defaults to the bundled amino-acid scales, which only fits when D matches that default — for a custom embedding space, always pass your own df_scales (a wrong D raises a clear error):

# Selecting a single embedding dimension by name still reproduces the run_num values
one_dim_feats = [f for f in features if f.endswith("dim0")] or features[:1]
X_one = nf.feature_matrix(features=one_dim_feats, dict_num_parts=dict_num_parts, df_parts=df_parts, df_scales=df_scales)
print(f"features: {one_dim_feats}")
print(f"Shape of X: {X_one.shape}")

features: ['TMD_C_JMD_C-Segment(10,12)-dim0']
Shape of X: (8, 1)

Multiprocessing is controlled by n_jobs (set to the maximum when n_jobs=None, all cores when n_jobs=-1). It is only worthwhile for large feature sets (>~1000 features per core) because of process-management overhead; the result is identical regardless of n_jobs:

X_serial = nf.feature_matrix(features=features, dict_num_parts=dict_num_parts,
                             df_parts=df_parts, df_scales=df_scales, n_jobs=1)
print(f"Identical to default (n_jobs=1) result: {np.array_equal(X, X_serial)}")

Identical to default (n_jobs=1) result: True

Mapping ``positions`` back to the tensor. The 'positions' column of df_feat is a display numbering in TMD-JMD coordinate space (JMD-offset, e.g. 21..30 for a TMD), not an index into the (L, D) per-part array. feature_matrix therefore never reads positions; it reconstructs each value by re-applying the SPLIT in the feature id to the part’s 0-based residue axis — exactly as :meth:CPP.run_num does — so X lines up with run_num by construction:

sample_feat = features[0]
sample_pos = df_feat.loc[df_feat["feature"] == sample_feat, "positions"].iloc[0]
print(f"feature: {sample_feat}")
print(f"df_feat 'positions' (display numbering, NOT tensor indices): {sample_pos}")

feature: JMD_N_TMD_N-Pattern(C,2,6)-dim3
df_feat 'positions' (display numbering, NOT tensor indices): 15,19