studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software

  • MagicShel@lemmy.zip
    link
    fedilink
    English
    arrow-up
    7
    ·
    8 hours ago

    I agree with your last paragraph. We had about 6 weeks of unlimited AI spend before the costs reached executive leadership, and in that time I saw the least experienced developers spend the most with the least to show for it.

    But I will say that another factor is thinking that if you get 10% gains from a little AI, then a lot of AI will get you 100%.

    But I find the article is right about repo-wide docs. At least on their own. I find having small markdowns (often in the form of skills/commands), focused on specific tasks reduces spend (especially when your execution agent is a low cost model, leaving the reasoning to dedicated agents) and gives better outcomes. Loading massive docs into every task reduces the attention to the task at hand and often confuses AI as the reasoning part of the model becomes overwhelmed and starts inferring wrong things confidently.

    I suppose it heavily depends on the scale of the repo though. A large microservice with multiple upstream services it needs to call spends a lot tokens on API which is unnecessary for most tasks. And then it decides to use the wrong one… I have stories lol.