Unlocking PCA with Biplots

Unlocking PCA with Biplots

Principal Component Analysis (PCA) is a statistical technique used to simplify complex datasets by reducing their dimensionality while retaining as much variance as possible. One effective way to visualize the results of PCA is through a Biplot. A Biplot combines a score plot and a loading plot to provide a comprehensive view of both observations and variables in the principal component space. This article will guide you through the essentials of PCA, explain Biplots, and provide practical code examples for creating Biplots in R and Python.

Understanding Principal Component Analysis (PCA)

PCA is a method used to transform a dataset into a new coordinate system where the greatest variance lies along the first principal component, the second greatest variance along the second principal component, and so on. This process helps in reducing the number of dimensions while retaining the most important aspects of the data.

Here’s a step-by-step breakdown of PCA:

  1. Standardization: Scale the data to ensure each feature contributes equally.
  2. Covariance Matrix Computation: Compute the covariance matrix to understand the relationships between features.
  3. Eigenvalue and Eigenvector Calculation: Find the eigenvalues and eigenvectors of the covariance matrix to identify the principal components.
  4. Projection: Project the original data onto the principal component axes to obtain the new coordinates.

What is a Biplot?

A Biplot provides a graphical representation that combines two plots:

  1. Score Plot: Displays the observations (samples) in the new principal component space.
  2. Loading Plot: Shows the variables (features) and their contributions to the principal components.

This combination helps in understanding the relationship between observations and variables in the PCA space.

How to Interpret Biplots

Interpreting a Biplot involves:

  1. Direction and Length of Arrows: In the loading plot, arrows represent variables. The direction indicates the influence on principal components, while the length indicates the strength of this influence.
  2. Proximity of Observations to Arrows: In the score plot, the proximity of observations to arrows shows how strongly variables influence those observations.
  3. Angle Between Arrows: The angle between arrows indicates the correlation between variables. A small angle indicates a high correlation, while a large angle suggests low or no correlation.

Practical Examples of Biplots

To illustrate how Biplots can be created and interpreted, we’ll use the Iris dataset and the Wine Quality dataset. We’ll provide code examples in both R and Python.

Example 1: Iris Dataset

The Iris dataset contains measurements of iris flowers and is commonly used to demonstrate PCA. Let’s walk through creating a Biplot for this dataset.

In R:

# Load necessary libraries
library(ggplot2)
library(ggfortify)

# Load the Iris dataset
data(iris)

# Perform PCA
pca_result <- prcomp(iris[, 1:4], scale. = TRUE)

# Create a Biplot
autoplot(pca_result, data = iris, colour = 'Species', loadings = TRUE, loadings.colour = 'blue') +
theme_minimal() +
ggtitle("PCA Biplot of Iris Dataset")

In Python:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris

# Load the Iris dataset
iris = load_iris()
X = iris.data
y = iris.target
feature_names = iris.feature_names
target_names = iris.target_names

# Standardize the data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Perform PCA
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

# Create a DataFrame for plotting
df_pca = pd.DataFrame(data=X_pca, columns=['PC1', 'PC2'])
df_pca['target'] = y

# Plot Biplot
plt.figure(figsize=(10, 8))
plt.scatter(df_pca['PC1'], df_pca['PC2'], c=df_pca['target'], cmap='viridis', edgecolor='k', s=50)

# Plot the loading vectors
for i, feature in enumerate(feature_names):
plt.arrow(0, 0, pca.components_[0, i], pca.components_[1, i],
head_width=0.1, head_length=0.1, fc='blue', ec='blue')
plt.text(pca.components_[0, i] * 1.2, pca.components_[1, i] * 1.2, feature, color='blue')

plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.title('PCA Biplot of Iris Dataset')
plt.grid(True)
plt.show()

Example 2: Wine Quality Dataset

The Wine Quality dataset contains attributes of different wines and their quality ratings. Here’s how to create a Biplot for this dataset.

In R:

# Load necessary libraries
library(ggplot2)
library(ggfortify)
library(readr)

# Load the Wine Quality dataset
wine_data <- read_csv("winequality-red.csv")

# Perform PCA
pca_result <- prcomp(wine_data[, -1], scale. = TRUE)

# Create a Biplot
autoplot(pca_result, data = wine_data, colour = 'quality', loadings = TRUE, loadings.colour = 'blue') +
theme_minimal() +
ggtitle("PCA Biplot of Wine Quality Dataset")

In Python:

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

# Load the Wine Quality dataset
wine_data = pd.read_csv("winequality-red.csv")
X = wine_data.drop('quality', axis=1)
y = wine_data['quality']

# Standardize the data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Perform PCA
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

# Create a DataFrame for plotting
df_pca = pd.DataFrame(data=X_pca, columns=['PC1', 'PC2'])
df_pca['quality'] = y

# Plot Biplot
plt.figure(figsize=(10, 8))
scatter = plt.scatter(df_pca['PC1'], df_pca['PC2'], c=df_pca['quality'], cmap='viridis', edgecolor='k', s=50)
plt.colorbar(scatter, label='Quality')

# Plot the loading vectors
for i, feature in enumerate(X.columns):
plt.arrow(0, 0, pca.components_[0, i], pca.components_[1, i],
head_width=0.1, head_length=0.1, fc='blue', ec='blue')
plt.text(pca.components_[0, i] * 1.2, pca.components_[1, i] * 1.2, feature, color='blue')

plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.title('PCA Biplot of Wine Quality Dataset')
plt.grid(True)
plt.show()

Conclusion

Biplots are an invaluable tool for visualizing PCA results, providing insights into the relationships between observations and variables in a reduced-dimensional space. By combining the score plot and loading plot, you gain a holistic view of how data points and features interact. Whether you’re analyzing flower species, wine quality, or any other dataset, mastering Biplots can significantly enhance your data interpretation capabilities.

 

Read also Enhancing Data Analysis with PCA and k-means