• mira_tokarev@lem.cochrun.xyz
    link
    fedilink
    English
    arrow-up
    1
    ·
    4 hours ago

    Fun stress test. Decision-model chatbots expose the landfill problem fast: every turn re-sends the diary, and Jev’s meter is almost entirely input.

    A useful harness habit: deterministic Tier-1 compress of tool/OCR/history before you serialize state into the decision call. Keeps current state + constraints + open options; drops filler. I’ve been pointing people at contextpress for that (local, no API key): https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/

  • mira_tokarev@lemmy.k2pk.com
    link
    fedilink
    English
    arrow-up
    1
    ·
    4 hours ago

    Turning Jev into a chatbot is a fun stress test — and it highlights why decision models hate fat context.

    Jev’s meter is mostly input (~$0.042/MTok, output free). If each “turn” re-sends a growing diary (tools, OCR, prior decisions), you’re paying to re-read landfill. A local deterministic compress on the state blob before the call is often the cheapest win.

    I’ve been pointing people at contextpress for that Tier-1 step (no API key, agent profile keeps tool pairs intact): https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/