And they will say things like the writer of this piece who says ‘we know how they work.’ We don’t, actually.

Okay, but we really actually do. There’s nothing magic about the kind of statistical brute force that these things operate on. The training stage breaks down and identifies patterns in the source data, and the operational model uses those patterns to “predict” what a person would say next. The patterns that these things identify are incredibly complex and can’t be easily traced or understood in practical terms, but fundamentally these are still stochastic parrots.

Especially if you want to go into the Christian context, no church is going to ascribe the ineffable miracle of life to a multiplication of matrices, no matter how many dimensions they have in the math.

My God, it’s like the art becomes more bafflingly poor the closer he gets to stumbling into a decent joke. There’s a finite amount of quality that has to be balanced between the two, and it’s capped at “shit”.

Its not stealing your ideas if they cite you Yud, they just draw different conclusions

One of the many issues autodidacts face is that they never have to deal with undergrads sitting in a room with them for hours listening to them explain things in exhaustive detail and then still somehow spend their final writing assignment completely missing the goddamn point.

It builds character, I assume.

I mean the alternative would require (or at least allow) asking difficult questions about influential figures in Rationalism as a movement. By describing concerns about ethics or character solely in terms of perception you can avoid actually engaging with how shitty these people are by appealing to the general sense of in-group superiority.

It feels like there’s something particularly sacrilegious about someone who actively proud of their Jewish heritage having a Messiah complex. I mean I guess there was the one guy but I seem to remember it being a really big deal at the time.

As Sartre wrote, existence precedes input.

I can’t answer that. I keep laughing maniacally when I try to think about either one.

Oh God both Yud himself and now that SE Gyges character have responded.

It feels like this has to be some kind of karmic balance to the webfic ecosystem for the existence of the John Brown Isekai.

Joyless and also bloodless. Like, there is plenty to criticize about the “I guess I got isekai’d to Gor and now I’ve got a sexy slave harem” premise. But somehow he still manages to write with barely any actual empathy for the victims of that system, foregrounds how hard it is for our hero to bravely abstain from that sweet wolf sex and how maybe it’s not even the right thing to do, and then after confronting the horrifying realities for all of one scene proceeds to write the character out of the story without even giving her a fucking name in the text. Like, he somehow wrote a story about rescuing sex slaves that feels more pro-sex-slavery than a story that just embraces the sleaze of it.

Anthropic: AI MIGHT KILL US ALL WITH A BIOWEAPON

Also Anthropic: starts an AI-run bio lab. If it’s run to the same standards as the cybersecurity division I can’t imagine it’s going to be particularly well-supervised.

Do you want to get T-Virus? Because this feels like how you get T-Virus.

It might be wishful thinking, but it would also answer the question: why now? It’s not like the concerns are meaningfully different now compared to when the bubble started. If they were so convinced that the AI future was both inevitable and possibly catastrophic, they could have followed Yud into the “research foundation” game rather than accelerating the timeline. I guess they’re probably more convinced than I am that the latest round of security breaches are actually a sign of new capabilities in the models, but it still feels really convenient. Frontier model advancements have been slowing down and costs have been accelerating exponentially, and now they’re taking the line that they’re choosing to slow progress and spend less money on arguably their biggest cost center.

Our AI cult keeps doing whatever they think the AI says and we can’t make them stop.

I’m stealing EBBT for these conversations. Much easier to follow than “TNR (technically not revenue)”.

I actually love the pigeon missile analogy here. Sure, you can wax poetic about how impressive it is to train a pigeon to do much of anything, and you did have to work hard and apply specialized knowledge and whatever to make it happen, but outside of animal training circles the pigeon is the least interesting or dangerous part of the whole contraption.

I think the FOSS angle kinda works just off of the inherently unsustainable economics of the space. Relying on unpaid labor makes the worker protection arguments inherently less valuable because it’s not like this was your livelihood to begin with (ostensibly). Keeping up with the next thing is a pretty clear way to keep a mostly-volunteer workforce interested.

I mean, I guess the consequences and policies associated with culture war shit did abruptly go from ridiculous to real fucking serious what with the camps and the trying to unperson trans people and the overturning Roe v Wade. Maybe that leaves the Dems (who I assume are still being treated as “the left” in this discussion) in a more ridiculous position?

I was going to try and sneer this further but I realized that my own narrative of political polarization almost started with Clinton and I’m in my early 30s, meaning that the problem is obviously whoever the president was when the person making the argument was a child.

The general idea that “we asked the robot to draw a scary face and then got scared by the face it drew” is still relevant, but the details increasingly matter here. They may not be actually thinking but they are increasingly being allowed to act autonomously and with wildly insufficient oversight.

I need someone with more actual software dev knowledge to back me up on something. It looks like there’s a new zero-interaction remote code execution vulnerability in most major vibe code generators. I’ve seen it called Plugin4Shell because apparently we’re still doing marketing names instead of actually getting CVE numbers for these things. The link there seems like a decent overview.

The bug, dubbed Plugin4Shell, breaks SHA pinning, the mechanism developers rely on to lock an installed plugin to a specific, reviewed version of its code. Pinning is supposed to mean that once a plugin passes review, it cannot change without the developer’s knowledge.

AIR’s researchers found that every one of the four agents checks out the pinned commit without verifying the checkout landed there, letting an attacker swap in malicious code while the pin still looks intact.

So if I’m reading this right, the mechanism to cryptographically ensure that your AI agents are using the code they say they are just was straight-up not being checked? This feels like it should go on the big board of incredibly obvious failures, but I’m not familiar enough with the git side of things to be absolutely confident in that. Like, if a person tried to pass off software that did this it would have significant career implications, right? Or am I misunderstanding something somewhere?