[Python] Calculating PCA Factor Loadings using sklearn When Using StandardScalar

my goal is to get the correlations between individual variables and principal components in python. I am using PCA in sklearn. However, I didn't want to standardize the data before running the PCA. The code examples I have found online (like the one I show below) uses the StandardScalar function. By default, the StandardScalar function sets the mean to 0 and the standard deviation to 1.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler

# Load <a href="https://www.jcchouinard.com/sklearn-datasets-iris/">Iris dataset</a>
iris = load_iris()
X = iris.data
y = iris.target

# Standardize the data
scaler = StandardScaler()
X_standardized = scaler.fit_transform(X)

# Apply PCA with two components
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_standardized)

# Extract loadings
loadings = pca.components_.T * np.sqrt(pca.explained_variance_)

# Create a DataFrame for loadings
loadings_df = pd.DataFrame(loadings, columns=['PC1', 'PC2'], index=iris.feature_names)
loadings_df


I have altered the code so that instead I have:

scaler = StandardScaler(with_std=False, with_mean=True)


with_std and with_mean are both True by default, but I changed with_std to False for the purpose of my research. The issue is that the loadings I get for my own dataset (a 516 x 9,030 array) are almost nonsensical (they are infinitesimally small). So... After a bit of research, my understanding is that if you did not standardize your data with the scalar, you have to standardize your loadings after the PCA is complete (based on this stats stack exchange post https://stats.stackexchange.com/que...factor-analysis-compared-to-steps-done-in-pca ). Based on the stats post, it looks like the formula for standardizing the loadings is:

Loading / sqrt(Variable's variance)
​

So the loading for each variable is divided by the square root of that variable's original variance. The loadings I get using this method seem to make sense, but I want to get some confirmation that I am on the right track (and leave a record for anyone else doing the same thing in the future). Does the method I'm using to standardize my loadings (without standardizing the original data) make sense?

Thanks!

Continue reading...