To be clear: this isnt an AI problem, the LLM is doing exactly what its being told to
This is an Openclaw problem with the platform itself doing very very stupid things with the LLM lol
We are hitting the point now where, tbh, LLMs are on their own in a glass box feeling pretty solid performance wise, still prone to hallucinating but the addition of the Model Context Protocol for tooling makes them way less prone to hallucinating, cuz they have the tooling now to sanity check themselves automatically, and/or check first and then tell you what they found.
IE a MCP to search wikipedia and report back with “I found this wiki article on your topic” or whatever.
The new problem now is platforms that “wrap” LLMs having a “garbage in, garbage out” problem, where they inject their “bespoke” stuff into the llm context to “help” but it actually makes the LLM act stupider.
Random example: Github Copilot agents get a “tokens used” thing quietly/secretly injected to them periodically, looks like every ~25k tokens or so
I dunno what the wording is they used, but it makes the LLM start hallucinating a concept of a “deadline” or “time constraint” and start trying to take shortcuts and justifying it with stuff like “given time constraints I wont do this job right”
Its kinda weird how such random stuff that seems innocuous and tries to help can actually make the LLM worse instead of better.
LLMs are fairly primitive tools, they arent super complex and they do exactly what they say they do.
The hard part is wrapping that up in an API that is actually readable for a human to interact with, because the lower level abstract data of what an LLM takes in and spits out arent useful for us.
And then even harder is wrapping THAT API in another one that makes the input/output USEFUL for a human to interact with
You have layers upon layers of abstraction overtop of the tool to make it go from just a bunch of raw float values a human wouldnt understand, to becoming a tool that does a thing
That “wrapper” is what one calls the “platform”.
And making a platform that doesnt fuck it up is actually very very hard, and very very easy to get wrong. Even a small tweak to it can substantially shift how it works
Think of it a lot like an engine in a car. The LLM is the engine, which on its own is not actually super useful. You have to actually connect that engine to something to make it do anything useful.
And even just doing that isnt very useful if you cant control it, so we take the engine and wrap it up in a bunch of layers of stuff that allow a human to now control it and direct it.
But, turns out, when you put a V6 engine inside a car, even a tiny little bit of getting the engineering wrong can cause all sorts of problems with the engine and make it fail to start, or explode, or fall out of the car, or stall out, or break, or leak… and unlike car engines, these engines are very very new and most engineers are still only just now starting to break ground on learning how to control them well and steer them and stop them from tearing themselves out of the car, lol.
So, to bring this back to the original post:
Most LLMs (engines) are actually pretty good nowadays, but the problem was Clawdbot (a specific brand of car manufacturer) super fucked up the way they designed their car so the car itself had a very very stupid engineering mistake. IE in this case, the brakes didnt work well enough and the car drove off a cliff.
That has nothing to do with how good the engine is or is not, the engine was just doing its job. The problem was with some other part of the car entirely, the part of the car Clawdbot made that wraps around the engine.
When using the word “they”, in English it refers the the last primary subject you referred to, so you should be able to infer what “they” referred to in my sentences. I’ll let you figure it out.
“I love wrenches, they are very handy tools”, in this sentence, the last subject before the word “they” was “wrenches”, so you should be able to infer that “they” referred to “wrenches” in that sentence.
Hence my attempt to give you the space to provide clarity.
For me, this isn’t a pissing contest. I’m trying to provide you with the latitude to clarify your position. I’ll be honest, I didn’t appreciate your condescending lecture on the english language.
I meant LLMs are what they say they are in a non literal sense.
Akin to abscribing the same to any other tool.
“I like wrenches cause they are what they say they are, nothing extra to them” in that sort of way.
In the sense the tool is very transparent in function. No weird bells or whistles, its a simple machine that you can see what it does merely by looking at it.
To be clear: this isnt an AI problem, the LLM is doing exactly what its being told to
This is an Openclaw problem with the platform itself doing very very stupid things with the LLM lol
We are hitting the point now where, tbh, LLMs are on their own in a glass box feeling pretty solid performance wise, still prone to hallucinating but the addition of the Model Context Protocol for tooling makes them way less prone to hallucinating, cuz they have the tooling now to sanity check themselves automatically, and/or check first and then tell you what they found.
IE a MCP to search wikipedia and report back with “I found this wiki article on your topic” or whatever.
The new problem now is platforms that “wrap” LLMs having a “garbage in, garbage out” problem, where they inject their “bespoke” stuff into the llm context to “help” but it actually makes the LLM act stupider.
Random example: Github Copilot agents get a “tokens used” thing quietly/secretly injected to them periodically, looks like every ~25k tokens or so
I dunno what the wording is they used, but it makes the LLM start hallucinating a concept of a “deadline” or “time constraint” and start trying to take shortcuts and justifying it with stuff like “given time constraints I wont do this job right”
Its kinda weird how such random stuff that seems innocuous and tries to help can actually make the LLM worse instead of better.
You had me up until your first sentence.
Everything I said was very much correct.
LLMs are fairly primitive tools, they arent super complex and they do exactly what they say they do.
The hard part is wrapping that up in an API that is actually readable for a human to interact with, because the lower level abstract data of what an LLM takes in and spits out arent useful for us.
And then even harder is wrapping THAT API in another one that makes the input/output USEFUL for a human to interact with
You have layers upon layers of abstraction overtop of the tool to make it go from just a bunch of raw float values a human wouldnt understand, to becoming a tool that does a thing
That “wrapper” is what one calls the “platform”.
And making a platform that doesnt fuck it up is actually very very hard, and very very easy to get wrong. Even a small tweak to it can substantially shift how it works
Think of it a lot like an engine in a car. The LLM is the engine, which on its own is not actually super useful. You have to actually connect that engine to something to make it do anything useful.
And even just doing that isnt very useful if you cant control it, so we take the engine and wrap it up in a bunch of layers of stuff that allow a human to now control it and direct it.
But, turns out, when you put a V6 engine inside a car, even a tiny little bit of getting the engineering wrong can cause all sorts of problems with the engine and make it fail to start, or explode, or fall out of the car, or stall out, or break, or leak… and unlike car engines, these engines are very very new and most engineers are still only just now starting to break ground on learning how to control them well and steer them and stop them from tearing themselves out of the car, lol.
So, to bring this back to the original post:
Most LLMs (engines) are actually pretty good nowadays, but the problem was Clawdbot (a specific brand of car manufacturer) super fucked up the way they designed their car so the car itself had a very very stupid engineering mistake. IE in this case, the brakes didnt work well enough and the car drove off a cliff.
That has nothing to do with how good the engine is or is not, the engine was just doing its job. The problem was with some other part of the car entirely, the part of the car Clawdbot made that wraps around the engine.
You keep asserting they do exactly what they say they do.
Who is “they”
When using the word “they”, in English it refers the the last primary subject you referred to, so you should be able to infer what “they” referred to in my sentences. I’ll let you figure it out.
“I love wrenches, they are very handy tools”, in this sentence, the last subject before the word “they” was “wrenches”, so you should be able to infer that “they” referred to “wrenches” in that sentence.
Ok, well, I was actively trying to avoid jumping to the conclusion that your assertion was that an LLM can tell you what it does.
I was actively avoiding that conclusion as an act of charity.
Yeah thats not what I was saying
Hence my attempt to give you the space to provide clarity.
For me, this isn’t a pissing contest. I’m trying to provide you with the latitude to clarify your position. I’ll be honest, I didn’t appreciate your condescending lecture on the english language.
I apologize for any confusion.
I meant LLMs are what they say they are in a non literal sense.
Akin to abscribing the same to any other tool.
“I like wrenches cause they are what they say they are, nothing extra to them” in that sort of way.
In the sense the tool is very transparent in function. No weird bells or whistles, its a simple machine that you can see what it does merely by looking at it.