Well, business english IS annoyingly verbose and full of empty phrases. It would be cool if we could dispense with vapid pleasantries.
I'm certainly not going to be the first to stake my job or my promotion on that particular hill. So I can fully understand why people will still turn "We need the database changes ASAP, you promised they'd be done. Get it done!" into half a page of drivel.
They don't have the same quality and kind of data. For example, Claude Code might have general conversation flow data for implementing feature X, but Cursor has users individual editing actions AND the chat flow. Which line did the user manually edit after the agent did it's thing? What's the commit message (if done manually)? Stuff like that is worth it's weight in gold.
It's really important to note that the programmatic vs. interactive framing is a little misleading. For example, if you're using some IDE integrations, that is still interactive, that is still the same usage pattern as using it in the Claude Code CLI. BUT with these new rules, it'll count against this special "programmatic" usage.
Considering what kind of fuckups happened up until now everytime Anthropic vibe codes a horrible anti-automation "fix" (like billing extra usage just because the string "hermes" exists somewhere in the prompt), this will get funny/horrible really fast. After all, you can very easily script tmux to use Claude Code "interactively".
What crazy shenanigans are gonna happen? Are people that type too fast, or people that copy paste too much, or people that output to markdown/json too often going to wake up to a 800$ extra usage bill or a banned account?
People do want the advantages of decentralization, but they don't want to pay the price for it. Even worse, centralized systems are great for most of the time, the pain generally happens only in a short span, but then very intensely (imagine a merger and a sudden price spike). Decentralization is a little bit of pain all the time, for a lot of happiness only in the rare case where the centralized alternative collapses.
Obviously, there's different options and variables and bla bla bla, but considering how consolidated and highly industrialized and standardized meat production is, this data is very likely close enough to true for the wast majority of beef burgers eaten by the people complaining about AI resource consumption: https://ourworldindata.org/environmental-impacts-of-food
The premise of your link is founded on the energy associated to with a single prompt. The source in your link for that energy claim links to a blog post that then links back to an earlier blog post from the original author of the link you provided (it's basically a circular reference).
Basically, there's a lot of words in your initial link, but they all hinge on the readers taken the stated energy assumption for a single (undefined) prompt at face value. If that initial assumption is wrong (at min, it's poorly defined in your link) all further conclusions are invalid.any a scientific publication have done this same trickery =].
They don't define what a query is when they are talking about AI power usage. If we want to get serious, we'd tie usage to tokens since we can actually track token usage.
>The source in your link for that energy claim links to a blog post that then links back to an earlier blog post from the original author of the link you provided (it's basically a circular reference).
Huh? The latter blog post does link to the former's blog, but not as a source for that claim. It cites an Altman blog, an estimate from EpochAI, an article in the MIT Technology Review (albeit one that estimates 3x higher), and a paper put out by Google. It's really surprisingly well cited and I don't know how you came away from it thinking it was a circular reference. The google study is in the subheading!
2) I click the link associated with the 0.3 Wh of energy claim in the section "The full cost of a prompt".
3) The link from 2) takes me to a blog post from Hannah Ritchie. In Hannah's post, I click a link associated with the following excerpt:
"Third, as a result, more recent estimates suggested that the assumptions I relied on (h/t to Andy Masley’s work on this) — that one standard query used 3 watt-hours (Wh) of electricity — were possibly an order of magnitude too high. In this case, I was happy to be conservative and overestimate the energy use."
4) This link takes me to the author of your original post, but earlier.
None of this quantifies cost per token, which is really the much more relevant metric than whatever a "cost per text based query" means => which I think is both quite broad and quite model dependent.
Well, I've got a small server rack and roof top solar, therefore data centers don't actually use water.
In other words, bringing up some anecdotal, hyper specific (how many meat eaters just "have a few cows"?) information says absolutely nothing about the truth of the matter, but a lot about what you believe constitutes an argument.
A third perspective here, but maybe small ownership of these things allows for best practices (i.e. small farmers are greener and care about passing arable land to the next generation, small server owners care more about total system ownership which necessitates alternative energy production and making use of hardware that would otherwise be trashed). I think you're both onto something, now kiss!
I think good faith would request that the source used for these kinds of questions is not one of the VC firms at the root of these questions.
Doubly so when they use such innocuous and authoritative titling as "Our World in Data" which implies some collectivist, community-based outlook that this website is indeed not.
To wit, this page is produced in part by the Global Change Data Lab which is a team of economists, and YCombinator.
Ourworldindata basically just uses data from published research papers and makes interactive graphs that are easy to understand. They also cite their source in every graph and every article. Trying to paint them as disingenious is pretty baseless, you would have to take it up with the authors of the source data and not owid.
It's literally funded and published by the VCs backing so many of the AI companies here. Literally the fountain of money that produced Sam Altman.
I'm going to have to ask for a little more rigour than "they use data".
Data can be wielded in all manner of manipulative fashion to suit a preferred narrative. And when it's wielded by engineers defending large AI (or any) companies, they should at least acknowledge the fact that the data reporting is "coming from inside the house", so to speak.
They got loans to buy inference hardware on the promise of potential AGI, or at least something approaching ASI, all leading to stupid amounts of profit for those investors.
We therefore cannot just look at inference costs directly, training is part of the pitch. Without the promises of continuous improvement and chasing the elusive AGI, money for investments for inference evaporates.
The constant improvements of SOTA are the main thing keeping the investment machine running. We can't really remove training costs from inference costs, because a bunch of the funding and loans for the inference hardware only exists because the promises the continuous training (tries to) provides.
I'm certainly not going to be the first to stake my job or my promotion on that particular hill. So I can fully understand why people will still turn "We need the database changes ASAP, you promised they'd be done. Get it done!" into half a page of drivel.
reply