PCA for materials data · Part 3 of four · October 9, 2026

Mean, variance and covariance in materials measurements

We return to our specimens. Before we combine their measurements, we need to describe differences between specimens and relationships between measurement columns.

What does the mean tell us?

For S003, S005 and S006, the yield strengths are 292.4, 240.2 and 286.6 MPa. Their sum is 819.2 MPa. Dividing by three gives a mean of 273.07 MPa. Their differences from this mean are approximately +19.33, −32.87 and +13.53 MPa.

A mean summarizes a column, but does not tell us how different its entries are. Three specimens all having 273.07 MPa would have the same mean, but no differences between them.

Here \(x_i\) is the measurement for specimen \(i\), \(n\) is the number of specimens, and \(\bar x\) is the mean. The summation symbol means add the measurements. The mean has the same units as the measurement.

Mean grain size in our spreadsheet refers to averaging grain sizes within one specimen. Averaging that column across specimens is a second, different average.

Why do we subtract the mean?

Subtracting each column's mean is called centering. It lets us describe each specimen relative to the average specimen in the dataset. Centered values can be negative; a negative centered strength means below-average strength, not a physically negative strength.

How much do the values differ?

If we add the signed differences from the mean, positive and negative values cancel. To measure their size without this cancellation, we square each difference and add them. Sample variance divides that sum by one less than the number of specimens.

Why one less? Our 300 specimens are a sample of all the specimens we could prepare. Dividing by \(n-1\) corrects a small tendency of a sample to underestimate the spread. With 300 specimens, dividing by 299 or by 300 makes almost no difference.

For the three yield strengths above, this variance is approximately 818.57 MPa². Its square root, the standard deviation, is approximately 28.61 MPa. Variance has squared measurement units; standard deviation has the original units.

Which measurements change together?

Suppose a specimen is above average in yield strength and also above average in tensile strength. Both centered values are positive, so their product is positive. If it is above average in one and below average in the other, the product is negative. Covariance averages these products over all specimens:

Here \(x_i\) and \(y_i\) are two measurements on the same specimen. Their column means are \(\bar x\) and \(\bar y\). A positive covariance means the two measurements tend to rise together; a negative covariance means one tends to fall when the other rises. Covariance has the product of the two measurement units, MPa \(\times\) MPa here, so its size depends on the units we choose.

Correlation removes the units. It divides the covariance by the two standard deviations. The result is dimensionless and always lies between \(-1\) and \(+1\).

Across all 300 specimens, the correlation between yield strength and tensile strength is 0.967. They tend to increase together. Yield strength and elongation have correlation −0.875: higher yield strength tends to accompany lower elongation in this synthetic dataset.

Grain size and yield strength have correlation −0.958. Grain size and conductivity have correlation +0.551. Conductivity therefore contributes another relationship that we should examine, rather than leave out.

These are descriptions of this dataset, not proof of a physical cause. A correlation close to +1 indicates a strong positive linear relationship; one close to −1 indicates a strong negative linear relationship. A value near zero indicates little linear relationship, but does not rule out other relationships.

Why should we consider units?

S003's yield strength can be written as 292.4 MPa or 292,400,000 Pa. It is the same strength, but its numerical value changes by a million. Its numerical variance changes by a million squared. A PCA calculation based directly on these numbers can therefore change when we change units.

For this example, we subtract each column's mean and divide by its standard deviation. This is called standardization. Each resulting column is dimensionless, with mean zero and sample variance one.

Across all 300 specimens, mean yield strength is 244.56 MPa and its standard deviation is 31.96 MPa. For S003, the standardized value is (292.4 − 244.56)/31.96, approximately 1.497. For S005, it is approximately −0.136. These values describe their positions relative to the dataset's mean and standard deviation.

Standardizing is a choice, not a requirement of every PCA. Here it gives each feature equal initial variance. We must consider measurement reliability and our scientific question before making that choice for real data.

We now have a covariance matrix

We collect all the variances and covariances in a square table, with one row and one column for every feature. This is the covariance matrix. Our five features produce a 5 by 5 matrix. A diagonal entry is a feature's variance; an off-diagonal entry describes its covariance with another feature.

For standardized yield and tensile strengths, their two-feature portion is approximately:

The diagonal ones are their standardized variances. The matching off-diagonal entries show that the two measurements tend to change together. For standardized columns, the covariance equals the correlation, 0.967.

What are the eigenvectors of this small matrix?

In Part 2, we looked for arrows that a matrix lengthens or shortens without turning. Let us try two arrows with this matrix. The first arrow, (1, 1), means “both strengths one standard deviation above average”. The second, (1, −1), means “yield strength up, tensile strength down”.

Neither arrow turns. The arrow (1, 1) is an eigenvector with eigenvalue 1.967. The arrow (1, −1) is an eigenvector with eigenvalue 0.033. As in Part 2, the arrow length does not matter; (0.707, 0.707) points the same way and has length one.

What do these numbers mean for our specimens? Imagine plotting standardized tensile strength against standardized yield strength, one point per specimen. Draw a line through the center and measure how spread out the points are along it. Along the (1, 1) direction, at 45°, the variance is 1.967, the largest we can find. Along (1, −1), it is only 0.033. The two eigenvalues add up to 2, the total variance of two standardized columns. So the 45° direction holds 1.967 / 2 = 98.4% of the variance: the two strengths mostly change together.

In the next article, we turn that line ourselves to check this, and then use the complete 5 by 5 matrix.