Does anyone know of an example of a simple physical system where eigenvalues have a physical interpretation? The examples I know of are all in quantum mechanics, which is a bit abstract for me.
I think a system of springs is a good example. I think having a bunch of springs hooked together is a bit abstract so let's instead think of a molecule and model the bonds between the atoms as springs. If you were to squeeze this molecule together or try to pull it apart and then let go, it would vibrate in some complex way. By complex I mean that it wouldn't just bounce back along the direction that you compressed or stretched it.
However, if you write down the matrix of spring constants for the system and solve for the eigenvalues and eigenvectors of this system you can do something special. If you compress or stretch the molecule along the direction of the one of the eigenvectors then let go, the molecule will continue to vibrate along that same direction. The motion will not spread out to all other degrees of freedom. It will also vibrate with a frequency given by the eigenvalue of that eigenvector.
Additionally, any complex vibration of the system can be broken down into a combination of these independent vibrational modes. This is a simple fact because the eigenvectors form an orthogonal basis for the space.
Get a sheet of rubber. Grab it in both hands and stretch it. Inspect your sheet and find a line on the sheet that you could draw in with a marker and when you stretched the sheet the line would grow and shrink, but would not change what it was pointing at (probably a line from one of your hands to the other, in this simple example) That is an eigenvector of your sheet stretching transformation. The eigenvalue is how hard you're stretching the sheet.
Take a linear map from some space to itself, and ask:
What lines (through the origin) are mapped back to themselves? Those are the eigenvectors, and the amount by which they're elongated or shortened are the eigenvalues.
So, if we talk about 3d space, and we rotate things - the rotation axis is unchanged. That's an eigenvector (with eigenvalue 1).
If we mirror things - any vector in the mirror plane remains unchanged, that's an eigenvector (with eigenvalue 1), the vector perpendicular to the mirror is unchanged, but flipped, so that's an eigenvector (with eigenvalue -1).
If we dilate everything along the x axis by a factor of 2, say, then the x axis is an eigenvector (with eigenvalue 2), while the y and z axis and any vector in that plane is an eigenvector (with eigenvalue 1). Any other vector is "tilted", so not mapped to itself, so not an eigenvector.
It means the eigenvalues will only give you information about the system relatively to the center of that system.
Before describing any system, it's up to you (your "convention") to assert where is the zero-point of your world and in which directions the axes (x,y,z) are pointing.
For instance, in the real world you can choose your 3D coordinate system such that your mirror, as a physical system, keeps the origin untouched (0,0,0) -> (0,0,0). If you decide the origin is a point on the mirror, the equations will be linear: mirror(X) = AX. However if you setup the origin some point far from the mirror, like the center of your eyes, the equations are no longer linear, but affine: mirror(X) = AX+B. Looking at the values of the "AX" part of the system would reveal you the mirroring plane, but now shifted by an offset of "+B" -- the distance between the mirror and your eyes -- because your choice of coordinates was not leaving the origin intact.
It means it doesn't matter where it is: you can choose the origin, ie the point you measure from, it is arbitrary. Or another way of saying that is you can move the system to a different set of coordinates and it works in the same way.
... which means it's probably an imaginary physical system.
Maybe a good physical example is a piece of cloth that warps in 2D, and shrinks, when washed? Eigenvectors would describe the warping (skew, say) and eigenvalues the shrinkage relative to the original warp and weft.
Steve Brunton on YouTube has really good videos on eigenvectors & eigenvalues in context of matrix algebra (and then applied to simultaneous differential equations); https://youtube.com/watch?v=ZSGrJBS_qtc .
With more than 2 masses, you don't need to arrange the masses on a line, but you can have a 2d or 3d arrangement, with interconnecting springs. I am sorry I failed to find an example image.
The theory is explained, for example, around page 479 in this Thornton and Marion Classical Dynamics textbook. But you need to read about Lagrangian mechanics (chapter 7) before it makes sense.
> Does anyone know of an example of a simple physical system where eigenvalues have a physical interpretation?
Yep, vibration modes. Vibration frequencies represent their eigenvalues while the shape that the structural system exhibits when subjected to said vibration corresponds to it's eigenvector.
If a structural system is modelled as a linear elastic system it's possible to apply an eigendecomposition of that system and represent it in terms of linear combinations of it's vibration modes/eigenvector, and consequently we can get very accurate representations by using only a hand-full of these eigenvectors.
You know swing sets? We would start to swing back and forth just by moving our legs in a particular frwquencey, and without much effort we could move more and more? It turns out the frequency we moved our legs was the system's vibration frequency/eigenvalue for the vibration modes/eigenvector representing the we swinging back and forth.
> Does this relate to the normal modes or eigenmodes of a system?
Yes. The eigenvalues and eigenvectors of an undamped harmonic oscillator are respectively the vibration frequency and vibration mode.
One major class of structural analysis techniques is modal analysis, which determines the vibration modes and corresponding frequencies of specific structural systems subjected to particular boundary conditions.
Moment of inertia tensor [1], and principal axes of rotation [2].
Principle axes are the axes where a weightless body can rotate around without "wobbling". These axes are orthogonal to each other. If a rigid body has I_1 < I_2 < I_3 moments of inertia, then rotation around the first and third axes is stable and rotation around the second axes is unstable.
The vibration of a bell (say) could be modelled by a matrix, with a state vector to represent position, velocity, and acceleration, and the matrix modelling the differential equations describing their evolution over time. The eigenvectors represent a basis of the system, so that we can describe any potential state vector as a sum of eigenvectors. If we do so, then each step of the system can be modelled by multiplying each of these eigenvectors by its corresponding eigenvalue. If an eigenvalue happens to be complex, then we can describe it in phasor form as the product of an amplitude and a angle. The amplitude tells us how it will decay (or amplify) over time. The angle tells us the frequency of oscillation, and thus the note that the bell will typically sound.
Theory of Vibration with Applications by William Thompson and Marie Dillon Dahleh.
Say you have two cars linked, with some spring constant;
| --^^-- [c1] --^^-- [c2] --^^--|
where '^^' is a spring and '|' is a wall.
The motion of these cars can be written using the spring forces in the system or, alternately, as the harmonic motion of the undamped system with some natural frequency.
Setting this up as two simultaneous equations (one for each car) and solving for the roots give you the eigenvalues. The natural frequency is the square root of the eigenvalue. In other words, the eigenvalues help you define the natural frequencies which can be used to characterize the motion of the cars in the more complicated spring-mass system.
Things that vibrate have natural modes of vibration. A particular vibrational pattern can be decomposed into a time-varying linear combination of these modes. The modes of vibration are eigenfunctions and the frequencies at which they vibrate are the square root of the corresponding eigenvalues.
You can look up a vibrating drum head (circular membrane) for a simple example.
That's more of a further packing than an unpacking. Although that totally should be an expression for things that go on for too long: "Can you pack this for me please"
If i recall correctly you can represent a harmonic oscillator as a linear differential equation with a 2x2 matrix. The imaginary part of the eigenvalues of this matrix correspond to the angular frequency of the oscillator.
I like this example because it gives a physical meaning to both eigenvalues and imaginary numbers. It also shows the connection between the sine and cosine and the complex powers of e comes from (since you can show that all three solve the differential equation).
Take an airplane... it’s dynamics are described by a series of differential equations. We want to know if it’s stable! If the wife values of the dynamics are real and greater than 1 it’s unstable. If the eigenvalues are complex and have a modulus greater than 1 it will oscillate instability. If one is equal to one, it will cause everyone to vomit.
The speed of sound is the eigenvalue of a particular matrix (the "flux Jacobian") in the Euler equations, the 5-component system of partial differential equations that describe gas dynamics.
There are a couple of other comments that have mentioned oscillation modes, vibrations, etc. The first 7 pages of this series on sound synthesis might help give an idea of where these might come from:
The third page in particular shows a plot of "amplitude" versus "frequency" to show the "harmonic spectrum of a sawtooth wave". The "frequencies" correspond to the modes of vibration (i.e., sine waves of different frequency), which are the "eigenvectors" in this case. The "amplitudes" are the relative contribution of those vibrations to the overall sound, and these correspond to "eigenvalues".
The article is talking purely about constructing sounds via synthesis, so there's not necessarily a linear system associated with it, but there is a connection. Wave equations represented by linear partial differential equations can often be analyzed as a linear system that has these "modes of vibration" (i.e., series of orthogonal sinusoids at different frequencies). If you were to, for example, model a plucked string (like a guitar), you can model the solution as a weighted sum of eigenvectors (in this case, "modes of vibration" or sinusoids of different frequencies). The "weights" would be the eigenvalues, which determine the spectrum and ultimately the timbre of the sound produced.
That might seem more involved, because it's an infinite-dimensional linear system (i.e., the vectors are functions on a interval, rather than finite lists of numbers). It turns out, though, that the finite-dimensional discretization of an infinite-dimensional linear system (i.e., a partial-differential equation approximated by a finite-dimensional linear system) will sometimes have eigenvectors / eigenvalues that have similar features as the infinite-dimensional case. For example, there are certain finite-difference operators that can be written in matrix form whose eigenvectors will work out to be sampled sinusoids.
I'm not totally sure of the history, but I think a lot of the interest in eigenvectors / eigenvalues as a topic in matrix theory originated from this are (i.e., numerical solutions for partial-differential equations that were used to model physical systems).
Most are introduced to the interplay between physics and linear algebra through the study of the mass - spring system where the type (real or complex), sign and amount of the eigenvalues determine its behavior and stability. For example, complex eigenvalues with positive real part indicate an unstable, or chaotic, in terms of amplitude convergence oscillation.
Not quite. For a Markov probability matrix, 1 is always an eigenvalue, and all other eigenvalues are less than or equal to 1. For each eigenvalue that is equal to 1 you get a long term stable state probability. These distributions contain disjoint subsets of the states, and the system will converge to one of those subsets, depending on the initial state. The eigenvalues that are strictly less than 1 do not add any information to the long term state of the system.
See Stochastic Processes and Their Applications, V4 (1976) pages 253-259. I wrote it while still in grad school.
The values associated with each vertex on the _dominant eigenvector_ (the eigenvector associated with the dominant eigenvalue) are the long-term stable state probabilities. That's from a single eigenvector, not "the eigenvectors".
I read an interview with Australian wire music composer Alan Lamb that a stringed instrument with multiple overtones vibrating on the string can be analyzed by breaking down the vibration into eigenvalues, but I've never found any reference material that explain that. I'm wondering if he was referring to FFT.
If you discretize the string into a bunch of tiny masses, linked together by a bunch of tiny springs, you can build a mass Matrix (diagonal) M and a Stiffness matrix K (element ij = stiffness of spring that links mass I and mass j).
I can't remember the next part exactly, you can look it up in a textbook, but you multiply the matrices KMK, or similar, and the eigenvalues of this are the natural frequencies of the string. The eigenvectors represent the mode shapes, ie the displacement of each mass element.
The same technique is used in Finite Element Analysis to find the modes and modeshapes of complex structures (a car frame, a bridge, etc)
Think of a fun house mirror that for the sake of this example make you look twice as tall but 20% skinnier. This can be modeled by a two-by-two matrix with eigenvalues of 2 and 0.8. (Indeed, it will have them on the diagonals which makes it easier to study.)
That's not true. The page rank is read from the eigenvector, and is the value associated with the given vertex (ie web page). There are as many page rank values as there are web pages, but only one eigenvector from which to read: the dominant eigenvector of the transition matrix, which is the one with the largest eigenvalue. So, only a single eigenvalue for the entire pagerank computation.
I like this explanation, which puts the concept of operator at center and has some nice computational relations, and is related to the physical world by the notion of an observable, with an eigenfunction being a special kind of operator-function relationship. At the end, there's a nice non-QM application (Fourier transforms).
> An observable is any dynamical variable that can be measured... in classical mechanics... observables are represented by functions* (such as position as a function of time), in quantum mechanics they are represented by mathematical operators... We shall not in general distinguish between the observable and the operator that represents that observable (such as the position of a particle along the x-axis)"
> "An operator is a symbol for an instruction to carry out some action, an opeeation, on a function...in certain cases, the outcome of an operation is the
same function, multiplied by a constant" [that constant being the eigenvalue]
> "An important point is that a general function can be expanded in terms of all the eigenfunctions of an operator, a so-called complete set of functions... then a general function can be expressed as the linear combination [a sum over a complete set of functions, each function having its own coefficient]"
Finally, we get to a practical real-world example (non-QM):
> "...for instance, the straight line g = ax can be recreated over a certain range by superimposing an infinite number of sine functions, each of which is an eigenfunction of the [differentiation] operator, d2/dx2. Alternatively, the same function may be constructed from an infinite number of exponential functions, which are eigenfunctions of d/dx."
Extending this general concept, we can go into the famous Fourier Transform, used widely for all kinds of classical problems in converting waveforms into frequency peaks, such as in musical analysis, electrical engineering, etc. The wiki page on Fourier transforms has a very brief mention of this view, i.e. The Fourier transform decomposes a function into eigenfunctions for the group of translations.
Here's what looks like a deep dive into this approach to the Fourier Transform (2008):
So my actual favorite first example is to do this with Fibonacci numbers as a linear recurrence relation, but that's not really a "physical" interpretation. Let me give you my favorite physical one:
The essence of special relativity is that acceleration is a bit weirder than you think. In particular when you accelerate by amount a in some direction x, even after accounting for the usual Doppler shifts you will find that clocks separated from you by that coordinate, appear to tick at the rate 1 + a x/c² seconds per second, where c² is a fundamental constant. Clocks ahead of you tick faster, clocks behind you tick slower (and indeed appear to slow down and approach a ‘wall of death,’ more technically called an ‘event horizon,’ at a distance c²/a. (This effect is called the ‘relativity of simultaneity,’ and it is in some sense the only real prediction of special relativity, as the rest of this comment will show—the other effects of ‘time dilation’ and ‘length contraction’ are second-order and can be derived from this first-order effect.)
This means that the transformation equations for moving into a neighboring reference frame are not the ones that Galileo and Newton proposed,
t' = t
x' = x – v t
but slightly modified to (to first order in v, so only considering small velocity changes)
t' = t – (v/c²) x
x' = x – v t
where w = c t is a measure of time in units of distance using this fundamental constant. How do we generalize and get the full solution? We can do it by looking in the eigenvector basis. Consider new coordinates p = x – c t and q = x + c t, given any (x, t) you can find a unique (p, q) which describes it and if you want to get back those values you would say x = (p + q)/2, t = (q – p)/(2 c). But feed these magical coordinates that come from eigenvectors into the above transform and it "diagonalizes",
p' = (1 + v/c) p
q' = (1 – v/c) q
and therefore if you want to make a big change in "velocity" c φ (here instead φ turns out to be "rapidity") out of N smaller changes, you can repeat this transform N times with little boosts by v/c = φ/N, and you will stitch together the full Lorentz transform out of little first-order Lorentz transforms:
p' = (1 + φ/N)^N p = e^φ p
q' = (1 – φ/N)^N q = e^{-φ} q
Transforming back and using the hyperbolic sine and cosine, sinh(x) = (e^x – e^{-x})/2, cosh(x) = (e^x + e^{-x})/2, the full formula is
w' = w cosh(φ) – x sinh(φ)
x' = x cosh(φ) – w sinh(φ)
where w = c t is a simple time-in-units-of-meters coordinate. Usually we denote cosh(φ) = γ, sinh(φ) = γ β, which gives this the more familiar form you'll find in textbooks, and the identity cosh²x = 1 + sin²x gives a formula γ = 1/√(1 – β²) for the latter... but this ‘rapidity form’ is in some ways more elegant. Anyway, point stands, from the "first-order" transform you can derive the "full" transform just by building any large velocity change out of an infinite number of infinitesimal velocity changes, and this is the source of the factor γ which describes time dilation and length contraction.
Okay, now for physical interpretation. You asked what physical meaning these eigenvalues and eigenvectors of the Lorentz transformation have, and the answer is this: the eigenvalues (1, 1) and (1, -1) of the Lorentz matrix represent light rays, the p/q description we came up with above was a description of spacetime in terms of light-ray coordinates where we identify an event at a particular place and time with the light rays that it casts, announcing that the event has happened, in the +x and -x directions. On the negative side, these are also the last light rays that were able to touch the event before it happened, so represent "everything it could have possibly known about" -- there is a space between these two "light cones" which is its "relativistic present," the things that anything which was there at the event cannot know about until the future.
The eigenvalues, exp(φ) = sinh(φ) + cosh(φ) = γ + γ β = √[(1 + β)/(1 – β)] and exp(-φ) = √[(1 – β)/(1 + β)], are the Relativistic Doppler shifts of those light rays. Indeed one can read them as e.g. exp(-φ) = 1/γ * 1/(1 + β) , here 1/(1 + β) is the standard Doppler shift formula from nonrelativistic physics and 1/γ is the decrease in frequency due to time dilation.
