I generally agree with the point of the article ("Fourier transform is not magical").
However saying it is "just" curve fitting with sinusoids fails to mention that, among an infinite number of basis functions, there are some with useful properties, and sinusoids are one such: they are eigenvectors of shift-invariant linear systems (and hence are also eigenvectors of derivative operators).
The comparison to railroad infrastructure is interesting.
I think the author is wrong on this point however:
> Today’s tech just cannot do what will be required of it (AI shouldn’t be dispensing medication when it can’t even count to 7).
The failures of AI are thought provoking, and more so when considered together with other results where AI performs at near expert level on challenging benchmarks. However, perfect reasoning is hardly a requirement. Most humans are not particularly good at reasoning, and most jobs do not need it. Both humans and AI can use calculators and other tools. All that's needed is that the AI is more or less as good as a human, while requiring much less pay.
A good exercise to appreciate the current state of AI might be to ask AI to write an essay about this topic ("how much revenue is needed to justify current AI spend, and draw parallels to the dotcom boom and building the transcontinental railroad"). Try it with two different models, using the deep research mode. I expect the results would be humbling.
...
So, in summary: We likely need on the order of hundreds of billions to low-trillions of dollars annually in AI revenue to justify the present level of infrastructure and model investment. Current realized revenues are many orders of magnitude below that.
But that’s the cold math. History suggests that such math often overlooks strategic externalities, spillover effects, hype, and speculative capital flows.
I am sympathetic to the motivations and argument in the article, but the analogy with bridge building is flawed.
In engineering, if your assumptions are correct and you apply the formulas correctly, the bridge will not fall.
This is _not_ true of software, since it suffers from mathematical incompleteness. Computation is isomorphic to mathematics, and, just as there is no way to objectively estimate how long it will take to prove a theorem, there is no way _objectively_ estimate program properties, even simple things like "will this program ever print the string "xx". The proofs are variations of the Halting problem.
that is basically the point the author missed. The PE license is to grant an individual a license for them to certify theirs and others works meets established guidelines. Now over time this may become true that something like level 2,3,4 are vague guidelines now but in future they will become more concrete. At that point it might become necessary to certify that a companies system meets those guidelines for the algorithms used.
While the post uses DPO to illustrate RL and RLHF, in fact DPO is an alternative to RLHF that does not use RL. See the abstract of the DPO paper https://arxiv.org/abs/2305.18290, and Figure 1 in the paper: "DPO optimizes for human preferences while avoiding reinforcement learning".
The confusion is understandable. The definition of RL in the Sutton/Barto book extends over two chapters iirc, and after reading it I did not see how it differed from other learning methods. Studying some of the academic papers cleared things up.
I think there was some quote from Karpathy who said that RLHF isn't actually "true" RL. As an armchair person, even after trying to understand it RLHF always seemed so roundabout. You don't have some open ended environment, you already have a fixed set of preferences. Instead of directly optimizing the model against that like DPO does, RLHF goes out of its way to train value/reward networks encoding these preferences then optimizing against that. I assumed that maybe it was just done this way for performance or stability or some other math -heavy reason, it was good to see that my suspicion was not off-base.
http://scribblethink.org/Work/kcsest.pdf
https://news.ycombinator.com/item?id=13731975
https://www.amazon.ca/Limits-Software-People-Projects-Perspe...