Numerical Analysis and Machine Learning

1. Chemical data as a matrix

For n molecules or reactions and p features, define $X\in R^{n\times p}$ and a target vector $y\in R^n$. Columns may contain physicochemical descriptors, fingerprint bits, Sterimol parameters, buried volumes, or field features.

2. NumPy arrays and linear algebra

Use ndarray rather than numpy.matrix. The * operator is elementwise multiplication and @ is matrix multiplication.

import numpy as np

print(X.shape)
gram = X.T @ X
beta, residuals, rank, singular = np.linalg.lstsq(X, y, rcond=None)

Use np.asarray, inspect shape and dtype, and reject NaN or infinite values before fitting. Broadcasting can silently produce a plausible but unintended array, so test dimensions explicitly.

3. Least squares and conditioning

Avoid explicitly forming an inverse of $X^{T}X$. Use least-squares, QR, SVD, or tested estimators. Large condition numbers indicate scale differences or collinearity.

The formal normal-equation solution is numerically less stable than QR, SVD, or a tested least-squares solver. Inspect rank and singular values; a large condition number signals scaling differences or collinearity.

4. Ridge, Lasso, and Elastic Net

Ridge applies an L2 penalty and shrinks correlated coefficients smoothly. Lasso applies an L1 penalty and can produce sparse coefficients, but may select one of several correlated descriptors unstably. Elastic Net combines both.

from sklearn.linear_model import ElasticNetCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

model = make_pipeline(
    StandardScaler(),
    ElasticNetCV(l1_ratio=[0.1, 0.5, 0.9, 1.0], cv=5),
)
model.fit(X_train, y_train)

Standardize features before comparing penalties unless their scales are intentionally meaningful. Select both alpha and the Elastic Net mixing ratio inside cross-validation.

5. Splits, preprocessing, and metrics

Put scaling, imputation, and feature selection inside a Pipeline so that each validation fold learns preprocessing only from its training partition. Random splitting can overestimate performance when related scaffolds or catalyst series occur in both sets; design group splits around the intended extrapolation task.

Report MAE, RMSE, R2, baselines, sample counts, uncertainty, and applicability domain.

from sklearn.model_selection import GroupKFold, cross_validate

cv = GroupKFold(n_splits=5)
scores = cross_validate(
    model, X, y, groups=scaffold_ids, cv=cv,
    scoring={"mae": "neg_mean_absolute_error", "r2": "r2"},
)

6. Principal component analysis

PCA projects centered data onto orthogonal directions of maximum variance. scikit-learn centers but does not scale inputs, so descriptors with different units usually require standardization.

from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(X)
pca = PCA(n_components=0.95)
scores = pca.fit_transform(X_scaled)
loadings = pca.components_.T

Scores locate samples; loadings describe feature directions. A component's overall sign may be reversed without changing the solution.

Report explained variance together with scores and loadings. Component signs are arbitrary; interpret groups of correlated loadings rather than assigning mechanism from a single coefficient.

7. SHAP

SHAP decomposes a model output into a baseline and feature contributions.

Beeswarm plots summarize a dataset; waterfall plots explain one prediction; scatter plots show feature-value relationships. Attribution depends on the model, background distribution, and treatment of correlated features.

For correlated chemical descriptors, attribution can be distributed among substitutes. State the explainer, background data, output scale, and correlation assumptions.

import shap

explainer = shap.Explainer(fitted_model, X_background)
shap_values = explainer(X_test)
shap.plots.beeswarm(shap_values)
shap.plots.waterfall(shap_values[0])

8. Practical workflow

1. Define the prediction unit and target before calculating features.
2. Freeze structure standardization and descriptor definitions.
3. Reserve a chemically meaningful external or grouped test set.
4. Place every learned preprocessing step inside a pipeline.
5. Tune hyperparameters with nested or clearly separated validation.
6. Report baselines, uncertainty, and applicability domain.
7. Use coefficients, PCA, and SHAP to form hypotheses, then test them with independent chemistry.

9. References

• NumPy User Guide
• scikit-learn linear models
• scikit-learn PCA
• SHAP documentation
• scikit-learn pipelines
• Cross-validation guidance