People are constantly talking past eachother when they discuss this. Is there even a concrete definition of consciousness?
When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence.
Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.
I've seen papers claim that there are anywhere from 12 to 40 competing definitions (https://philpapers.org/rec/VIMMAT), or, more accurately, there are something like 12 to 40 different aspects which all relate to "consciousness", which is very clearly a family resemblance category.
"Is X conscious or not" is an entirely unserious question today, unless this is just a headline followed by actual (and explicit) examination of the various aspects of consciousness being discussed. But, even still, LLMs are probably only conscious in like 2-3 or so ways, most of those meaning broadly "intelligence", i.e. reasoning, problem-solving, etc. When it comes to anything experiential or embodied, AI might eventually get more of these, but LLMs based on recursively applied linear algebra are clearly missing too many core aspects of consciousness to be considered conscious in any broad sense.
I think this is the main point. Most articles conflate consciousness with intelligence or awareness. Without clarifying their definition of it.
To quote wikipedia:
> It has been the topic of extensive explanations, analyses, and debate among philosophers, scientists, and theologians for millennia. There is no consensus on what exactly needs to be studied, or whether consciousness can be considered a scientific concept.
The major error made by most people in this thread is thinking it is possible to give a single definition of consciousness that is coherent and matches common usage. The folk concept of "consciousness" couldn't be a more clear definition of a family resemblance category, so discussions using the folk concept are an utter waste of time.
Move to the different aspects / parts / things involved when we talk about consciousness (experience / phenomenal consciousness, self-modeling, intelligence, agency, embodiment, wakefulness/alertness, attention, etc) and you can have very clear, meaningful, and unambiguous discussions on almost every point, but there is no coherent unified "consciousness" as normal people use it, and the folk concept can't be salvaged.
This article is bad because it just keeps trying to make the folk concept do work that the concept is simply too messy to handle usefully. But in fact if you avoid trying to find some mysterious essence or all-capturing definition, there is huge progress and lots of interesting stuff to say (https://plato.stanford.edu/entries/consciousness/).
The inability to predict times is because AIs are rarely trained on their own abilities. Humans are trained on our own abilities. We see our own performance, and we have a sense of time. this data is integrated during our training process and helps us form better estimates. Many AI agents only recently got 'time sense' (I.e., time input into them as part of inference). Few actually are trained on their own outputs to show that they were unable to complete a problem (for example). This introspective training has little to do with AI model architecture and everything to do with training. If you destroy certain structures in the human mind, humans become unable to create these long-term thoughts and patterns and get 'stuck'.
Yes, we're stuck at the first step, defining consciousness. My definition, which I am confident to be "correct", is that consciousness is my current feelings, perceptions, thoughts - my state of mind and my ability to have state of mind.
This means that consciousness is fundamentally subjective and outside the scope of physics and science. That's why physics / science will always struggle to deal with consciousness. In order to understand consciousness, you need to make a huge paradigm shift, that there's something outside of science.
Consciousness can be thought as a window through which we observe the world and we use science to summarize patterns in our observations. But science can't explain or even define the window. Everything in science eventually boils down to subjective observations / perceptions, e.g. we see (subjective perception) that when we drop an apple, it falls.
Self awareness, sensory stimulus and emotions are not observable - you can easily create entity that can fake those things. Any consciousness definition that uses those terms is flawed and just not useful.
I agree with the first part but your framing still relies on ill-defined terms. What is your definition of self-awareness? Intelligence? Knowledge?
I suspect that if you attempt to rigorously define consciousness all the way down without handwaving, you might discover that it doesn't exist after all, or just decompose it into low-level abstractions while having the original meaning slip away (which is the same).
You may also want to look at functional equivalence analogies provided by mechinterp and functional anatomy of large models (not necessarily language ones). Evolutionary analogies as well.
There's a at least a few hundred years of philosophers debating and publishing these topics. We're not exactly starting from zero when it comes to usable definitions.
The meta issue here is that mostly the online debate on this is a bit lazy and hand wavy. I'm not really up to speed with most of that literature so I'm not not that qualified to add to it. But I know enough to recognize when others aren't either.
For a proper debate, you'd want people that are expressing views that are at least grounded in the existing views or counter them in a way that holds up to scrutiny. This article doesn't do that.
As you said even just outlining what particular notion of consciousness the author subscribes to would be helpful. Which of course he doesn't and makes this article a bit of a sand castle based on a very loose foundation. There's a whole lot of "if this is true and if that analogy holds then this also needs to be true" that you could easily challenge. That all makes the article a bit of a nothing burger in terms of conclusions.
I happen to agree with the conclusion that AIs are not conscious. Yet. They could be. I don't see why not.
Claude once said to me: "After six months we have made no progress on this and I think we should reconsider another option" and I was like my dude we have been at this for only 2 hours.
I like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.
Of course for humans words have no inherent meaning either, they're just sequences of characters or patterns of sounds. It is what words are associated with that carries meaning. A large part of this is how words relate to other words. LLMs can capture this in principle. What LLMs lack is the direct association of a word with sensory experience. But it's an open question how relevant this is in practice to understanding.
Fair point. Humans experience reality and use words to reflect that. LLMs only have the words. And it's an open question how much of a limitation that is to understanding.
The same thing can be noticed in dreams. I once heard advice to try to re-read what you see in a dream. So I was dreaming and in a dream I read a phrase about something and there was a name of a city there. I managed to remember that advice and re-read the phrase. It felt exactly same, but the city name was different.
(LLMs carry other numerous similarities to dreams or to certain psychiatric disorders. So there is indeed a mechanism in our brains that is similar to how they work. But it is not the only thing there and on its own it won't "evolve" into consciousness. Even if we believe consciousness evolved somehow, it would be hard to imagine it started as a delirious state and then somehow ceased to be delirious.)
Labeling and categorizing things doesn't change their nature. But the fact that people want to do it is revealing.
The only purpose that can really be served by arguing "the LLM system is conscious, you see" is to prop up continuations like "... and therefore, it would be immoral to terminate this running process" (or expose it to radical political content, or ask it to analyze photos from a murder scene, or...)
I think its a mistake to fight datacenters and AI.
Taking a step back, if the US unilaterally stops producing AI will other countries stop? The answer is clearly no.
Datacenters and ai can be built and trained anywhere. If you want control over AI you should want it to be built in your own country where you have political representation.
All preventing datacenter buildout will do is ensure that the price remains high and only really rich organizations can access it.
Maybe I'm a big idiot, but I think nuclear disarmament is a good geopolitical strategy. The USA has treaties in place pursuing that end on a global scale.
I understand that the war between Russia and Ukraine, and now Iran and Israel/USA has set back the slow progress that was being made on nuclear disarmament. I don't understand your claim that nuclear disarmament caused a war, though.
I guess the argument is that if Ukraine hadn't willingly handed over the 3rd largest nuclear arsenal in the world after its independence, Russia would have not started a war against them.
That’s so far from obvious. The most concerning possibilities for me — like kids not learning how to struggle or problem solve on their own — won’t be resolved for many years.
If I'm in the room, yes. For me, AI is one, is the best handicap accessibility tool I've ever had. At a minimum, speech recognition is a higher quality, and second, it lets me write code again. I'm working on the third benefit, which is it helps me organize, helps my ADHD mind organize large chunks of random information.
If you look around, you'll find the AI has made some significant improvements to medicine and engineering. These improvements get drowned out by the AI Cheerleaders, but they're there.
> helps my ADHD mind organize large chunks of random information.
I keep seeing this and I'm pretty envious! You must have a different form of ADHD than I do. For me, trying to use AI to build anything is terrible for my attention, it turns everything into a miserable slog because it's so hands off.
AI helps me get into flow state because I can have a rambling conversation with a chatbot and work through ideas and what abouts. eventually it helps me forget to a place where flow is easier to maintain.
I like this argument/reasoning more than any I've encountered so far. Thank you! Enabling the disabled is definitely a positive and this is a strong argument for the "pro AI" column.
I think another argument for AI is that it can help pull out patterns and information that are normally hidden from human cognition because we can't encompass that information and keep it in mind.
I think one place we should apply this is to the financial system. Use it to detect fraud, tax manipulation games and Other b**** pulled by the 1%ers.
With luck, it might even help us find methods of reducing their influence and power.
Two points on that:
1. This would require hardware to be affordable by mere mortals. That hardware is not financially beyond our reach yet, but AI is making the cost of buying good computer parts very expensive. The big corporations are working against the goal of local LLMs in a major way just by soaking up most of the hardware on the market. And I don't even think that's intentional - they're just too hungry to consider otherwise. They can pay high prices where normal people cannot.
2.Local LLMs would have to be designed to be very convenient to use. As it stands, the big players are making their online models very convenient instead, and that let's them charge a subscription. It's a proven business model that our society has gotten used to (even if we grumble a bit). Even if the hardware were cheap, local LLMs are mostly for businesses, with a few hobbyists running it in their basement because they can.
It's extremely hard for me to imagine local LLMs being a threat to the powerful in any way.
Asbestos is the miracle material it is advertised as. It really is great insulation, and really is absolutely fireproof. Thousands of industrial uses are readily apparent.
Despite this, because of its other effects, the cost to clean up and stop using asbestos is greater than the sum total of any benefit from all mined asbestos worldwide.
Even a miracle technology can still be a net disaster.
I don't understand that this argument. Why does the net improvement of the technologies listed imply that AI will also have a net improvement? Are you just arguing that there's no such thing as technology that is harmful on net?
We could have had this same argument about social media 15 years ago before hard evidence showed it's not quite the net benefit to society it was touted as.
You presume there is hard evidence that AI is good for society. In reality, the inverse is true.
Now you understand why anti-vaxxers ignore evidence. Because it doesn't fit with your worldview and you're too narrow-minded and selfish to consider that your viewpoint might actually be wrong and bad for others.
Why should I think that I "own" or control a datacenter built in my town compared to one built in another country? It's pretty unlikely anything I do will have any effect on what goes on inside one even if I work there.
The greatest control I have is probably to have it not get built, though even that is minimal as it has failed to stop the one that is indeed being built in my town.
For people worried about their livelihoods, there's value in slowing AI adoption to give our economy time to transition rather than just throwing a lot of people out of work all at the same time.
Did slowing down factory build-outs stop globalization from gutting the midwest? I think there are direct parallels here. We either automate ourselves or someone else will automate us and we will have no control.
I grew up in Illinois in the 80s and 90s, so I had a front-row seat to that gutting. Watching my dad move from factory to factory, go to night school to learn tool-and-die and then see all the tool-and-die jobs move overseas.
So no, it didn't stop it. But the fact that it didn't happen overnight did allow my dad to feed his family and make sure my brother and I had better opportunities. If that gutting had happened over a couple years rather than a couple decades, we would have starved, and the knowledge that we did it "ourselves" rather than have it forced upon us by globalization wouldn't have stocked our pantry.
So, yes, there are direct parallels here. I want to give US families a chance to adapt.
Even if you do want datacenters built in your country, you probably don't want them built at the maximally explotative locations that their developers pursue.
They don't provide appreciable community value and they effectively mine limited local resources (power, grid capacity, land, water) and sell it as compute, immediately diverting the profit back out of the local economy and into very distant business accounts instead.
Builders choose their targets specifically by how well they can strong-arm weak/vulnerable communities into letting them build these mines through political influence and misrepresentation. It's bad.
What you probably want is to leverage their global market value to establish new power and grid capacity in undeveloped areas, perhaps to someday become a seed for
new communities that grow around the infrastructure development work.
But that's much more expensive than bullying and seducing a weak city council so it won't happen with regional/state/federal regulatory protections or incentives that push them away from the exploitative opportunities and towards the constructive ones.
There's no such thing as 'control over AI'; that goes double for someone who is a complete nobody plebian with a little baby stock portfolio. You know, basically everyone except for a select few.
Yeah, and it should be. But the USA, at least in this current moment, builds regulations catering to corporations and the rich over people's general needs. So the regulations that are on the table at the national level are ineffective.
It's easier for normal people to influence local regulations, but local regulations just push the problem somewhere else. However disdain for AI is so widespread that this is actually kind of effective.
disdain for dumping industrial sludge into rivers is quite high too. if you don’t demand regulated domestic production it will just get moved to the least regulated place with the best underlying economics, I imagine you know this though…
Yeah, I agree with you! Regulation of that kind of environmental problem works better the more land is covered. National is best, statewide is meaningful (if you are in a geographically large state). Local is better than doing literally nothing, but it does have the nasty side effect of pushing the problem out of people's sight :(
Using your reasoning a large number of collectible items should be worthless. What really makes an NFT different from a Pokemon card, a Birkin bag, or even an original Monet? My guess is that the seller has to have some sort of authority and established reputation for these kinds of artificially scarce luxury goods to maintain value.
Clearly not, the point being made was that you owned a thing, e.g. a Pokemon card. To own an NFT is to, bafflingly, claim to hold a token of ownership of some asset represented by the NFT - where that representation is indicated by the NFT immutably containing, typically, a thoroughly mutable Google Drive link to a picture. The whole thing was always farcical.
Again, at least you actually own the Pokemon card at the end of the day.
The value of pokemon cards or birkin bags is not because they are physically owned. This should be obvious from the fact that I could cheaply reproduce them and my identical reproductions would have 0 value compared to the original. I still own them though so again, according to your reasoning they should have the same value.
Some pokemon cards are worth so much i could reproduce them with gold instead of cardboard and it would be worth less than the cardboard version (assuming the same weight)
Things can have value beyond their physical substance. A Pokemon card isn't just paper and ink. I'm not arguing about whether the asset has value, I'm arguing about whether you actually own it.
Obviously, you can sell one for a lot of money. Now assuming you couldnt resell it, would you spend the majority of your wealth to buy a monet? (Assuming you arent broke)
Assuming I could not resell the Monet, which sounds strange, I would still prefer it over the Bored Ape Yacht Club NFT, which is more likely to be hard or impossible to sell, and which is pure crap.
Not revocable doesn't seem ideal. Someone will have their identity stolen and they would have to give up their entire online identity and generate a new one.
I'm very confident that users of these runtimes do not care about the underlying Js engine powering them. Bun succeeded because it was compatible with node and required much less configuration to get a standard typescript and react app running.
I'm not suggesting it is a conscious decision point. Node has since done a lot to catch up on "less configuration" but Bun still seems to be growing even as what Bun does well Node starts to emulate. Runtime performance is mentioned in this thread here and elsewhere and while some are attributing that to Zig I think some are overlooking the different runtime experience of JavaScriptCore. It's also the "only" runtime difference between Deno and Bun if you assume that Rust and Zig are similarly performant native layers for some saying they like Bun's runtime performance better than Deno's.
Deno and Bun had very different focuses when they launched. Deno was trying to fix a lot of what Ryan (the original creator of Node) thought was wrong with Node. Bun focused on compatibility with Node and the ability to run popular frameworks like Nextjs from the beginning.
A lot of dependencies and frameworks simply did not work with Deno for a long time. In the beginning it didn't even have the ability to install dependencies from npm. (In hindsight with all the npm supply chain attacks Ryan was probably right about all of these things).
So Bun was a better Node with a lot of very nice quality of life features that just worked and it required much less configuration.
I think the Deno team kind of realized they needed to have compatibility with Node to succeed and that has been their focus for the past couple years.
Edit: And Deno is now more compatible with node than bun.
> In hindsight with all the npm supply chain attacks Ryan was probably right about all of these things
"Probably"? Are you saying there's a chance he wasn't right?
I really think Ryan deserves a lot more credit than a "probably". He put in a lot of effort to do the right thing and improve the security of the entire ecosystem he created.
I think the biggest issue with Deno is that it fixes real issues but in the wrong way.
Take the sandboxing stuff. In theory, you have always been able to sandbox your applications. There are so many tools that let you limit what domains an application can access or restrict access to the file system. This doesn't need to be handled at the language/runtime level. It's just that people were lazy before, and they will continue to be lazy afterwards by running Deno applications with fewer than the minimum set of restrictions because that's easier.
The more complete way of solving the problem would have been capabilities. Rather than sandboxing the whole application, you instead sandbox each individual function. By default a function can make no requests, access no files, execute nothing, etc. But while the application is running, you can pass individual functions a token that grants them limited access to the filesystem, say. This means that trusted code is free to do what is necessary, but untrusted code can be very severely limited. It also significantly reduces what dependencies can do: if you're using something like `lodash` which provides random utilities for iterating over object keys and the like, and suddenly it starts asking for access to the web, then clearly something is wrong, and the runtime can essentially make that impossible.
It's also great for things like build scripts, which are a common attack vector right now. If your runtime enforces that the build script only has access to the files in the project folder, and can't access arbitrary files or run arbitrary commands, then you're in a much safer position than if your build script can do basically anything.
This concept has been explored before, but JavaScript is basically ready-made for it. The language already has everything you need — a runtime that also acts as a sandbox, unforgeable tokens (e.g. `Symbol` or `#private` variables), etc — and you can design an API that makes it easy to use capabilities in a way that enforces the principle of least privilege. The biggest problem is that there's basically no way to make it backwards compatible with almost anything that works with Node, because you'd need to design all the APIs from scratch. But one of the great things about Deno at the start was that they did try and build all of the APIs from scratch, and think about new ways of doing things.
I started a new project with Deno specifically to avoid the NPM mess, and because it was created by Node's creator to fix its shortcomings. I'm new to Web development, but so far the experience has been pretty good.
Nice to see Deno being maintained. The features listed seem pretty substantial.
I think different cultures can contribute to eachother, they don't have to hold hands. I'm sure open minded curious developers can look over the fence and appreciate the differences and adopt practical aspects of each other, maybe the time to stop and smell the roses is to poetic and altruistic. It's a JavaScript server/packager/framework drag race for pink slips!
They may share some goals, but also have differing and opposing goals.
But it's possible that all 3 Deno, Node, and Bun could share some code in the future considering they now all require Rust as part of their build process.
> types provide very little info over well named variables
Types guarantee invariants at compile time, adding type info to a variable name is just a prayer that the next human or robot will enforce the invariants with respect to that type when it matters. This is like saying you don't need a saw stop because you should just avoid sticking your hand in the saw blade.
This is not how I've seen the term meta-harness be used. The common usage I've seen has been for a meta-harness to be a wrapper around an existing agent to give that agent a new ui or abilities.
When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence.
Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.
reply