If you're into eigenfunctions, pick up any textbook on quantum mechanics. The hamiltonian is a linear operator whose eigenfunctions are the stationary states of the system ("stationary" because an isolated system in a stationary state will never leave that state) and whose eigenvalues are the observable values of the energy of the system. In general, there is a correspondence between observable quantities and Hermitian operators on wavefunctions: "measurement" is the application of a Hermitian operator to the wavefunction, and the values you may observe are the eigenvalues of the operator. So, for instance, energy is quantized in some systems because their hamiltonian has discrete eigenvalues.

Small correction: eigenfunctions are analogous to eigenvectors, not eigenvalues. In fact they _are_ eigenvectors, in the sense that they are vectors in a vector space of functions (or some restricted set of functions, e.g. continuous functions or periodic functions).

Maybe I'm missing what's interesting about this, but a function like f(z) = 5z + 2 would output a wave with changed amplitude and phase when z = sin(x). That doesn't seem that interesting to me, so f(z) must have some other interesting properties?

I think what the comment is missing is that the class of linear f(z)'s is much larger than you might expect. Partly because it just is, and partly because we like it that way.

In the land of analog signal processing: any combination of capacitors, inductors, and resistors [https://soundcertified.com/wp-content/uploads/2020/04/speake...] is linear.

In the land of math abstractions of signal processing: differentiation, integration, finite-impulse-response (FIR) filters, IIR filters, frequency-domain equalization, etc. All linear. Remember, linearity is with respect to the full time history, so f(z) = z(t) - z(t-1) + z(t-2) is still linear.

So we already know the eigenvectors of that whole arbitrary pile of componentry! Given any box of the above components, we can exactly characterize its response to any input -- for all time -- by knowing a list of that system's eigenvalues -- one for each of the already-known eigenvectors.

That's the "frequency response" -- the eigenvectors are sinusoids, and the frequency response is the eigenvalue corresponding to each eigenvector (sinusoid). And of course the Fourier transform takes you back and forth from the time domain to the eigen-domain.

We liked this analytical framework so much that when we could fabricate nonlinear devices (transistors) easily, we purposefully arranged things so these devices were only used in a linear part of their response curve. Hence, amplifiers: f(z) = 11 z.

And then the musicians introduced distortion and f*cked it all up -- our system isn't in a linear regime, our old eigenvectors are meaningless, and we can't predict what will come out. Pure chaos.

Heavy Metal ruined the clean perfection of our fancy math devices, you say? That's pretty metal.

Yea, that makes sense. Basically exactly how you can use an eigenbasis in linear algebra to diagonalize a matrix.

It's even worse than you describe it!

f needs to be linear, but the function in your example is not linear.

However, there are quite interesting linear functions. Example: f(x(t)) = x(t-2) + 4dx/dt - \int_0^t 2x(s) ds

5z + 2 is linear?

affine, not linear. describes a line that doesn't go through the origin. that pesky shift breaks linearity

5(2z) + 2 != 2(5z + 2)

Good point. But what makes studying these functions interesting? Like what key theorems govern this class of functions?

All of linear algebra is based on linear functions. Of course that doesn't mean that you can't study affine functions with it, but it adds a layer of extra complication.

For example, linear functions over finite dimensional vector spaces can be represented with matrices which means that everything you can compute about matrices you can also compute about linear functions.

This is confusing terminology. Functions of the form ax+b are often called “linear”, but they're only linear in the general sense if b=0.

Honestly I forgot the details, but basical the ENTIRE field of DSP stands on this fact.

Basicaly there exist some functions into which you can feed in sound waves and the output is guaranteed to still be a sound wave.

If you'd feed in a sound wave and if the function would corrupt it you would not be able to do any digital signal processing, since the output must be a wave.

Sound(wave) in -> Sound(wave) out, guaranteed to always be true.

> there exist some functions into which you can feed in sound waves and the output is guaranteed to still be a sound wave.

That in and of itself does not seem like a particularly insightful observation. It's just obvious that such functions exist. I can think of three of them off the top of my head: time delay, wave addition, and multiplication by a scalar. There must be something more to it than that.

In math, the obvious things aren't always true and the true things are often not obvious.

Trivially, the identity f(x) = x satisfies the guarantee as well. What amounts to insightful observation is the definition and classification of these functions. In exploring their existence in various forms, we can begin to understand what properties these functions share.

So the interesting part is not that this class of function _exists_, because of course it does! Your intuition has led you to three possible candidates. But if we limit ourselves to only the functions that satisfy the condition _wave-in implies wave-out_ what do they look like as a whole? What do these guarantees buy us if we _know_ the result will be a wave? For example, f(g(x)) is also guaranteed to be _wave-in-wave-out_. Again, maybe obvious, but it's a building block we can use once we've proved it true.

Ok, but what is being asked is what interesting properties does such a function have. You still haven’t answered that question.

Let me try. Let’s say you have some operator F = crazyweirdblackbox. Like, it exponentiates and does a bunch more complicated things. You can apply this operator to some input function and transform it into some output function. The programming analogy would be some piece of code that works on a lambda and spits out another lambda. The finite-dimensional linear algebra equivalent is a matrix. You would ideally want F to be invertible (or undoable) for all inputs. Just in case.

If g(x) is an eigenfunction of F, then h(x) = F(g(x)) is actually just a rescaled of g(x): h(x) = ag(x), for some constant a. No matter how complicated and hard-to-compute F is, it boils down to just one number a, when acting on some special function g(x).

So what? This only applies to special g(x), and not any choice for g. Let’s say that I have some special function y(x) that isn’t necessarily g(x). But I have a whole bunch of eigenfunctions of F called g_k(x). If I knew that F was linear (kind of a prerequisite for computing eigenfunctions anyway), then I can decompose some output function y(x) into a weighted sum of g_k: y(x) = sum([w_k g_k(x) for k in range(infinity)]).

Some abuse of Python list comprehension there.

So instead of evaluating F, which might be very hard computationally or numerically, we can instead do a for-loop over potentially easier functions g_k. And store some weights w_k that essentially describe how F transformed an input function into y. Easy-peasy.

And maybe I don’t want to evaluate the entire sum. So I could choose to evaluate only the “important” terms in the sum to approximate y(x).

The entirety of signal processing, much of quantum mechanics, much of electromagnetism, and many other partial differential equations can fit into this framework. And we use PDEs to describe F when we really have no idea how to even to write it down. But we can still compute eigenfunctions and therefore make progress in evaluating F since we know the effect of F on certain special functions.

Does that help?

It's been 15 years since I was in my EE degree, and I learned all this stuff to the point of intuition. Since then I've been a C-monkey at megacorp and not doing any real math beyond simple geometry.

Thanks for tickling the old neurons.

Yes, exactly.

This comment was downvoted into oblivion (I vouched it back from the dead), but I have no idea why. You hit the nail on the head.

In fact functions are just infinite-dimensional vectors. Almost all of the theory goes through unchanged. This is the basic idea of functional analysis.

Notably, one very important part which does not go through is that for mappings between infinite dimensional spaces linearity does not imply continuity. (E.g. a series of functions bounded by a constant can have arbitrarily large derivatives)

A large part of functional analysis is dealing with that fact and its implication for PDEs.

Or rather size-of-their-domain-dimensional?

If you want, but you can do better. I believe, for instance, that at least continuous functions on the reals have a countable basis. Might even be as strong as measurable, not sure about that. That's how, for instance, fourier transforms work.

The fourier Transform essentially means that every L^2 (the square of the function has a finite integral) is completely "described" by an l^2 series (a series of numbers whose sum of squares converges), which is about the greatest piece of magic in mathematics. One very important piece here is that the term "function" is somewhat of a lie (since the result couldn't be true if it weren't).

The result for measurable functions (not almost functions) shouldn't be true (I think). I am not even sure it is true for L^1 almost functions.

The GP kind of implied functions R->R, but yes.

I’m not sure I understand but it seems to me you are just talking about eigenvalues in C.

That’s interesting but not particularly remarkable because eigenvalues are defined for linear transformations of any vector space over a field.