Kalman Filters were developed outside the context of “Machine Learning” and found practical applications much earlier.
If you define ML as an algorithm that is “trained by input data”, than any statistical model would be ML. So is the ambiguity around the terms ML and AI in general.
Where a KF is really going to kick the pants of a multi-layer perception/neural network is how computationally efficient it is. A KF only takes a couple of matrices of size N^2, where N is the number of variables you’re trying to predict. Compare this to a NN with hundreds/thousands of nodes. Also, the KF is “online learning” in that it “is trained as you go” rather than some other ML models that require upfront training, a KF is very useful for live update-and-predict use cases. And, again, it’s extremely computationally efficient, and can run easily on embedded systems. The book ad, here, suggests tracking: so tracking an airplane with an air traffic control radar would be an effective use for a KF. (Where NNets have found any other uses I’m sure you’re aware of.)
Another huge benefit of a KF is that, unlike NNets, KFs are “explainable”, and in fact extremely well understood by many professionals. This means that a KF can be better tuned to suit a purpose with less fear of unexpected results that may be more common in other ML models. Like, KF(S, x) will always return an explainable new state, where NN(x) may result in a surprise state and no amount of analysis can reveal why (and require training a new model, the “retrain and pray” solution).
Because of "resume driven engineering". Even simple problems that you can solve with PID controllers or Kalman Filters but everyone wants to throw ML at it instead, so they can put "ML experience" on their LinkedIn because that's what's hot right now as recruiters probably never heard of Kalman filters or PID controllers.
A lot of technical decisions aren't based on "what's the quickest, cheapest and easiest solution to the problem?" but "what solution is most likely to get me hired at a pay bump when I jump ship?"
The thing is, to train a NN to estimate your output from your input, you need input-output pairs. KFs are a way of measuring that output in the first place. So they are not even the same class of solutions.
ML Models are usually used in the context of prediction, Y = F(X,θ), where X = inputs, θ = weights, F = model. There's typically no explicit feedback look (only historical data), no time-variation (you can add using lags however), and no existing model structure in most cases (most are black boxes, some like linear regression have a linear model which are fairly loose).
Kalman Filters are used in the context of a very specific model-type for dynamic systems (a state-space model, see below) to update states (xₖ) using feedback data from sensors (yₖ). These state-space models can either be derived by fitting data, or they can be derived from first principles through physics equations.
xₖ₊₁ = f(xₖ) + g(uₖ)
yₖ = h(xₖ)
The feedback loop is modeled explicitly, including any control actions (uₖ) that you took to affect the environment.
For instance, when driving a car, examples of states (x) are position/velocity/acceleration (which might not be directly measured with a sensor! But can be backed out from a mathematical model from quantities that are measured), sensor measurements (y) might be speedometer, accelerometer readings, and control actions (u) might be throttle position, brake pressure, steering angle. The Kalman filter has a model relating all this in time, and based on that model and sensor readings, it reconstructs/infers the likeliest states in the presence of even noisy measurements. This is why Kalman filters are known as "state estimation" algorithms.
ML models typically do not do this -- they only predict. Kalman filters predict and update.
If you can make certain assumptions about the system
(mainly that sources of noise follow gaussian distributions and are independent), then the Kalman filter gives the best possible estimate of the system state. And it can be computed cheaply, like on the Apollo guidance computer.
You basically need to know some kind of a model for the system to run KF. Whereas ML is all about working out the model automatically.
As for similarities, KF is a really efficient implementation of Bayesian inference. I think that any ML model that isn't fundamentally using Bayesian inference, is fundamentally flawed.
ML requires training, significant amounts of compute power, and large datasets.
The Apollo program used Kalman filters with limited compute resources.
Kalman filters are for predicting system states in the presence of uncertainty; ML is really searching for and matching patterns, under uncertainty not in its training set, it tends to to find the glitch in the matrix.
You could probably call a Kalman filter an ML model if you wish. One major difference is that you usually have to prescribe the prediction model. It doesn't learn to predict the next value like an LLM, instead it learns to find an optimal weighted sum of its prediction and the measured values. So it's a prediction-correction loop that requires an interpretable model, that has the side effect of allowing you to estimate system states. This is quite different than having an arbitrary hidden state with learned structure.
In other words it performs well for certain applications specifically because it allows you to bring in domain knowledge in the form of the process model and known uncertainties. Whereas deep learning models try to generalize the model and learn implicit structure from data.
Kalman filters are the *optimal* solution to the problem of controlling a linear system driven by additive white Gaussian noise and a quadratic cost function.
The model is updated sequentially (online learning).
It's Bayesian, and you specify what the model is. If you don't have a lot of data, and the model is a good match for reality that can be a good thing. Otherwise if you have data, ML is going to be more accurate because it can make a better model than your hand crafted model that you used in your kalman filter.
An ML model in theory can model anything with an input and an output. That's almost anything in the universe. You can replace a huge portion of engineering with it in theory.
Actually in theory you can replace everything with it. So what's the point of asking this question here? Ask it for everything.
Ask the question: "can we replace humans with ML?" That question is closest to being answered right now then ever before. We are on the cusp of an impending future where that answer could be: "yes."
Honestly, modern ML models are extraordinarily messy. They are inefficient, unreliable (when the goal is perfect reliability), often misused, mostly unexplainable, and very much a "throw the spaghetti at the wall to see what sticks" type of problem solving.
Kalman filters, and other similar digital filtering and prediction algorithms, are like scalpels compared to the broadsword of NNs and such. There are plenty of things that you can't or shouldn't use a kalman filter for, but for the tasks that it is suited for, you cannot do better with another solution. ML is mostly hand wavy bullshit, and DSP algorithms are like... doing real math, real engineering.
How is this different than any of the ML models?