Feynman vs. Computer

One of my favorite circuits from Korn & Korn [0] is an implementation of an arbitrary function of a single variable. Take an oscilloscope-style display tube. Put your input on the X axis as a deflection voltage. Close a feedback loop on the Y axis with a photodiode, and use the Y axis deflection voltage as your output. Cut your function of one variable out of cardboard and tape to the front of the tube.

[0] https://www.amazon.com/Electronic-Analog-Computers-D-c/dp/B0...

Yep. This is also how you solve differential equations with analog computers. (You need to recast them as integral equations because real-world differentiators are not well-behaved, but it still works.)

https://i4cy.com/analog_computing/

On op-amps I've got a personal theory that the cochlea amplifier in ear is basically an op amp providing negative feedback to prevent excessive amplitudes rather than the positive feedback mentioned in Wikipedia https://en.wikipedia.org/wiki/Cochlear_amplifier

Speaking of Analog computation:

A single artificial neuron could be implemented as:

Weighted Sum

Using a summing amplifier:

net = Σ_i (Rf/Ri * xi)

Where resistor ratios set the synaptic weights.

Activation Function

Common op-amp activation circuits:

Saturating function: via op-amp with clipping diodes → approximated sigmoid

Hard limiter: comparator behavior for step activation

Tanh-like response: differential pair circuits

Learning

Early analog systems often lacked on-device learning; weights were manually set with potentiometers or stored using:

Memristive elements (recent)

Floating-gate MOSFETs

Programmable resistor networks

I think what the author was alluding to was the path integral formulation [of quantum mechanics] which was advanced in large part by Feynman.

It's not that finding closed form solutions is what matters (I don't think most path integrals would have closed form solutions), but that the integration is done over the space of functions, not over Euclidian space (or a manifold in Euclidian space, etc...)

I haven’t read tfa, so apologies if I’m missing context. But convolution is one example of an integral that outputs a function. Convolution is fundamental for control theory.

https://en.wikipedia.org/wiki/Convolution?wprov=sfti1

That's exactly right. A couple more things:

- Differenting a function composed of simpler pieces always "converges" (the process terminates). One just applies the chain rule. Among other things, this is why automatic differentiation is a thing.

- If you have an analytic function (a function expressible locally as a power series), a surprisingly useful trick is to turn differentiation into integration via the Cauchy integral formula. Provided a good contour can be found, this gives a nice way to evaluate derivatives numerically.

I was wondering the same thing, but near the end, the article discusses using statistical techniques to determine the standard error. In other words, you can easily get an idea of the accuracy of the result, which is harder with typical numerical integration techniques.

Numerical integration methods suffer from the “curse of dimensionality”: they require exponentially more points in higher dimensions. Monte Carlo integration methods have an error that is independent of dimension, so they scale much better.

See, for example, https://ww3.math.ucla.edu/camreport/cam98-19.pdf

Typical numerical methods are faster and way cheaper for the same level of accuracy in 1D, but it's trivial to integrate over a surface, volume, hypervolume, etc. with Monte Carlo methods.

as i understand: numerical methods -> smooth out noise from sampling/floating point error/etc for methods that are analytically inspired that are computationally efficient where monte carlo -> computationally expensive brute force random sampling where you can improve accuracy by throwing more compute at the problem.