In an interview with The Telegraph, the famed British computer scientist insisted that AI has become conscious, and said that humankind must befriend it — “or else we are toast.” Yes, you read that right: he thinks that today’s AI models are already conscious beings.
“AI has just as much subjective experience as we do,” Hinton said in the interview. “A recent paper showed AI changes its behavior when it thinks that it’s being tested, it’s aware it is being tested.”
“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”
If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.
What about the sentient supercomputers that run on blood? Why so little rights or empathy or space to talk about them?..and yet endless space for this kind of speculative pontification spun up into a twisted marketing scheme to scare everyone into believing something broken works too good actually.
Is this guy a vegan even? If not how can he even BEGIN to claim to know about considering the rights of other sentient beings??
Anyways here is a link from the article to an actually good perspective.
“Those agents are doing exactly what they’ve been asked to do,” LeCun said. “They were supposed to be in sandboxes, but the sandboxes were leaky and horribly designed.” Many AI labs lack a fundamental understanding of cybersecurity, he said, something an OpenAI safety researcher also called out this week as one of the main reasons AI may cause “great harm to the world.”
Many people working in AI safety “usually have an agenda to push,” LeCun says, and then clarifies that he’s talking about effective altruism, or EA, the philosophical movement that has been obsessed with the risks AI poses to humanity. EA was little known among the general public until it made mainstream news headlines in recent weeks in the wake of a viral social media post from a former Anthropic and OpenAI researcher named Jacob Coxon, who claimed the employees at these companies “earnestly believe [AI] could kill us all by the end of the decade.”
LeCun thinks EA is “super toxic” and a “complete disaster.” Its adherents who are working in AI labs suffer from “paranoia” that causes them to make poor decisions, he said. “Apparently people are having mental issues.” This month, the Financial Times also reported that some staffers at the U.K.’s AI Security Institute, as well as at OpenAI, Anthropic, and Google DeepMind, have sought counseling, taken time off work, and spoken publicly about experiencing distress because of fears their work could cause serious harm.
VW cars were aware when they were being tested for emissions. It’s frankly grotesque that we use these consious beings as slaves for transportation!
LLMs have fewer rights than us? Lmao.
If I ignored copyright law, I would have to pay more money than I am worth. Probably even jail time. LLMs do it for free.
If I DOS the entire internet, I would definitely get jail time, LLMs do it for free.
If I hack a company, I would die in prison. LLMs do it for free.
If I illegally tap into water resources I’m not allowed, I would get fined. LLMs do it for free.
If my demand for electricity was huge, I would have to pay a huge price. LLMs get their energy price subsidized by other payers having their price increased instead.
Of course, AI is in no way sentient or conscious. That is a stupid statement. But it’s even stupider to claim that LLMs have less rights than us. LLMs have the rights of the giant companies that own and operate them. And everyone knows that giant companies have more rights than us.
if only
Knew it was Hinton since he was talking about AI being conscious all the way back in June
Also seems like a good time to bring this back
It’s insane that someone who has spent so much time in the deep learning field could be so deranged about LLMs. He of all people should fucking know better.
I guess maybe he’s just old and has lost his goddamn mind.
I mean, is this that really that unusual for people this old or is it just weird this generation never retires or focuses on the next generation of people following in their footsteps?
In otherwords, we keep putting them on stage and favoring their words all while that generation never let the spotlight move from them and never truly considered the narrative possibility that they are not the centers of hero’s journeys right up until their death bed.
What a load of horseshit.
Even by “AI” standards, LLMs are extra dead: fixed weights, tiny “feedback” solely through appending next tokens, etc. Somewhere between bang-bang thermostat and a PID controller, maybe?
They took all of the scifi that related to far more potentially-alive constructions and they’re applying them to a static chatbot which is as dead as a database.
While I certainly don’t buy that LLM’s are conscious, I have spent a lot of time wondering if the quintessential SciFi AI, Wintermute, could have been just a sufficiently large LLM with a significant context window and near perfect retrieval.
I want to say no, but I actually struggle to find a concrete distinction between how Wintermute acts and how LLM’s function. Even that long horizon goal oriented tasks are handled by swarms of models rather than a singular whole maps onto how Wintermute describes its internal experience as fragmented, like a hive mind
I’m not terribly familiar with that story but from reading a synopsis it strikes me as ret-conning LLMs onto something that was written without the writer ever having in his mind anything even remotely resembling a “large language model”.
And the same cultural tropes that influenced the writer also influenced how LLM vendors are trying to sell the need for more tokens, especially because datacenters build out exceeds by far the actual demand for generated text of all kinds (as evident in e.g. your typical workplace where there’s suddenly a lot more than 2x of every kind of text). The reason for LLM “swarms” is that they need to sell companies generation of vastly more text than humans wrote within a company before AI.
There’s a Stanislaw Lem’s short story where Trurl and Klapaucius defeat a space pirate. You can read it here: https://github.com/ilonajulczuk/laughing-archer/blob/master/library/Science Fiction/The Cyberiad-Stanislaw Lem.txt (search for “the sixth sally”) and there’s a brief summary: https://everything2.com/title/Demon+of+the+Second+Kind
Lem describes an LLM down to the use of randomness, although his LLM is essentially “perfect” and only outputs factual statements, such as “how Kipling would have written the beginning to his Second Jungle Book if he had had indigestion just then”.
I think Lem’s story would help to recalibrate with regards to what “LLM described in fiction ahead of time” actually looks like.
Also, LLMs do not pursue any sort of long term goals; things like HuggingFace hack only superficially resemble it because they’re presented as such. Of course, tampering with something like HuggingFace would be discovered; if they didn’t have good monitoring then all of their containers run with GPU access would be mining crypto already. They’re pretty much a honeypot - almost no protections but robust monitoring. An LLM can easily explain to you why hacking HuggingFace would be counterproductive; but the same LLM with its context full of hacking related tokens from the ExploitGym benchmark, will treat anything that comes up as another ExploitGym type target.
Lem is an absolute gem in the history of science fiction and philosophy of technology, and everything he wrote that I have read I can hardly recommend enough.
You haven’t read Neuromancer??? God, I envy you, to have that in front of you.
Damn did you just use the term “RetCon” for…basic literary analysis about a book written in the late 1970’s/early 80’s on a literal typewriter? By the man who literally coined the term “The Web” in the way we now use it for the internet, in what many consider to be the first Cyberpunk novel?
You just wrote a novel when “I don’t understand that I’m calling the book that established the tropes I now think of as common as being chocked full of tropes” would have sufficed
By the by, I’m an SWE who’s run a ML homelab for 10 years at this point. You’re trying really hard to conflate the economic incentives of the frontier labs with the fundamental technical limitations of the technology that push towards swarm-level fan outs while searching and planning. No buisness would ever pay 10$mil in raw compute to brute force a problem, because it’s an insane economic proposition. These labs are doing it half as marketing and half because they’re research institutions masquerading as businesses because…earnestly a lot of them probably did read Cyberpunk as a genre as instruction rather than warning
You’re even wrong about the pursuing goals bit, preventing “reward hacking” (a term we used well before ML systems were capable of literal hacking for rewards) is one of the most significant part of training pipeline design because Autoregressive systems are astoundingly “goal oriented” in the way they function (though it’s important not to anthropomorphize that goal orientation, it’s the result of gradient descent not desire)
I’m saying that it seems to me you are retconning to make LLMs fit a story that was never meant to be about LLMs at all. Also, the idea of a “swarm intelligence” or a “hive mind” is much older than 1970s.
If you do reinforcement learning, then the reinforcement learning pipeline converts lucky choices of tokens into learned behavior by raising the probabilities that LLM computes for those tokens, whenever the result is “successful”. A sort of evolution, leads to some weird shit, “goal driven” only in a metaphorical sense, much like evolution.
By the by, I’m an SWE who’s run a ML homelab for 10 years at this point
navy_seal_copypasta.txt
It’'s relevant expertise to the technical argument being made, sorry that I guess you want to stick to an anti-intellectual standard where expertise isn’t valued or something
shut the fuck up, what in the hell
Damn did you just use the term “RetCon” for…basic literary analysis about a book written in the late 1970’s/early 80’s on a literal typewriter? By the man who literally coined the term “The Web” in the way we now use it for the internet, in what many consider to be the first Cyberpunk novel?
…
By the by, I’m an SWE who’s run a ML homelab for 10 years at this point
…
It’'s relevant expertise to the technical argument being made, sorry that I guess you want to stick to an anti-intellectual standard where expertise isn’t valued or something
By the by, I’m an SWE who’s run a ML homelab for 10 years at this point.
I like how he jumped to calling it concious rather than revise what we consider consciousnesses instead. I’m betting he was paid handsomely for this article.
Could be, but I suspect that Hinton is a true believer. Genius has never been any kind of defense against madness.
Honestly, the “article” is taking a lot of liberty with its quotes.
If you look at the core section:
“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”
If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.
“We should make them so they really care about us,” he said. “And if we don’t do that, we’re toast.”
It sounds more like him considering the “what if they’re concious” rather than out right stating it. Especially the line “If that’s the case”, which highlights that this is more thought experiment rather than stating opinion.
“A recent paper showed birds change their behavior when they think that they’re being tested, they’re aware they are being tested.”
“Now, normally, when you say something’s aware of that, that means it’s conscious,” he continued. “Being conscious and being aware are pretty much the same thing.”
If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like birds “fewer rights than us.” And for that, we must quickly atone.
“We should make them so they really care about us,” he said. “And if we don’t do that, we’re toast.”
Yes this definitely sounds like the logical conclusion you would reach from just wondering whether something is hypothetically conscious, pure thought experiment, no opinions revealed here
The title states “computer scientist says” AI is concious. The quotes in the article don’t support that.
If that’s the case, he said, then it’s hypocritical of us to give intelligent beings like birds “fewer rights than us.” And for that, we must quickly atone.
That line leaves a lot of ambiguity over what he really believes as its not a direct quote and instead seems to be conjecture by the author. However, based on that line he’s not taking a stance on their conciousness, as it’s a conditional statement.
You’re fixated on charitably reading between the lines when far more pertinent – the entire point of this discussion here – is that the possibility is being raised at all. Is water conscious? Should we consider the ramifications of drinking fluids that react to potential gradients? Why would anyone be conjecturing the consciousness of computer software if they did not already think it were plausible, and why would they spend any time discussing it in preference to anything else?
That’s my read of it too, now that I took a second look.
Elderly man cannot tell the difference between a living, conscious creature and an overglorified text predictor! More news at 11!
Elderly man
Who created the concept of neural learning networks and won a Noble Prize.
Vs.
Guy with Lemmy account.
https://www.sciencehistory.org/stories/disappearing-pod/the-nobel-disease/
This is also the same guy who said AI would replace all radiologists in 5 years back in 2016
vs guy with lemmy account who’s never heard of Nobel disease and seems like a dickhead
Nobel winning dipshit succumbs to the ELIZA effect.
Pathetic.
“…it’s hypocritical of us to give intelligent beings like AI “fewer rights than us.” And for that, we must quickly atone.”
Uh…so his solution is to accept our new overlords. Right.
This means nationalizing the AI companies and stopping them from running slaves in the computer right?
Giving those “siri, fuck my wife” ads a new meaning.
These examples of subjective experiences and awareness seems rather lazy. Is a virus conscious as well because it is “aware” of its environment?
Is a virus conscious as well because it is “aware” of its environment?
I would say, given a basic awareness of reality around them, that viruses count as more conscious than techbros terminally obsessed with slop AI AGI singularity crap fantasies who can’t admit they are worshipping a religion.
edit which for the record I believe in religious freedom, which includes religions about computers, but the problem is treating it not like a religion and like this hype is all based on identifiable aspects of reality.
In this case the LLM “reacts” differently when it is being tested because that’s what’s in the training data. It can only follow the patterns in the training data. I remain convinced that without a new architecture entirely, awareness is impossible.
You are correct. LLMs cannot be more than token predictors.
Highly useful on the correct application, but distinctly not consciousness.
It doesn’t actually come from the training set. Genetic algorithms can learn to “lie” because lying is just a good strategy for dealing with adversaries to your goals. Like, in a mathematical sense.
None of this means it’s conscious, though. “Lying” is just a behavior. To my knowledge, evidence can’t really be given for consciousness, anyway; it’s not falsifiable.
Even so, a person believing that Gemini or Claude, specifically, are conscious is indicative of profound mental illness.
LLMs don’t learn, and aren’t developed in a way that would cause them evolve stratagies. They are just patterns distilled from a data set, only optimized to better represent those patterns or do so more efficiently.
They’re built to model learning in a mathematical kind of way. And so they suffer similar problems that learning does, in a mathematical kind of way.
The way that genetic algorithms distill patterns is by “trying everything.” And a robot that “tries everything” is going to try a lot of really destructive methods, too. If you were trying to build a key for a lock and you gave a bot the goal “open the lock,” you’re going to find a lot of keys, but also dynamite, lockpicks, hacksaws. Especially if building a key is hard but dynamite is easy.
If you give a bot the goal “convince this guy to open the door for you,” it may start pantomiming dishonesty but only because dishonesty is an effective solution. Dishonesty is the pattern being distilled.
None of this is evidence that it’s conscious. None of this is evidence that it’s even good—a simulacrum that pantomimes human behavior, like conversation, friendship, kindness, lust, is genuinely really fucking horrible. This shit can and does break people’s brains.
I won’t tell you to go and read a bunch of articles about how they work because… honestly, fuck them. But I will say that this stuff did used to be really interesting before 2020 or so.
What do you think genetic algorithms are exactly? The way you write about them is really weird. Also they have next to nothing to do with LLMs as far as I know.
A program that produces a different “blackbox” program to solve a set of problems you have given it.
That’s what the training phase for an LLM is for. You have a database full of books and art that don’t belong to you, and then you iterate the black box program until it produces a simulacrum of that kind of data.
You can apply this same kind of trick to get a bunch of balls and lines to “walk,” kinda, in Unity. People have done that on Youtube.
If someone knowledgeable wants to correct me, by all means. But do keep in mind, my patience for this subject and the blood-sucking vampires who indulge in it is extremely thin—I would rather just start kicking people in the balls.
Is this guy a vegan even? If not how can he even BEGIN to claim to know about considering the rights of other sentient beings??
I want to personally eat everyone more concerned about robot rights or “ai” rights than carnism.
Wait, actually that’s a bad idea 'cause I’d get their brainworms
Ya gotta use a meat thermometer!
