Strands Harness 成本与定制化痛点

原话 · @Anonymous

I am increasingly hesitant to use non-native harnesses - model providers are now starting to train their agents for use within the harness. An eval like terminal bench can only capture so much data. I don't want to have to assess each harness every model release to make sure it's working as well as it can.

原话 · @Anonymous

Why is Pi not in the benchmarks? Deepseek beats Strands and its built on Pi so that’s all I needed to know.

原话 · @Anonymous

> With Fable 5, Strands harness cost 77% less than Claude Code and scored higher on Terminal Bench 2.1.Terminal Bench 2.1 is saturated. Many token saving techniques would save money and score basically the same running Fable 5 against Terminal Bench 2.1. (They claim a better score but don’t say how much better. I’d bet my favorite hat that it’s not statistically significant.)This is at least the fourth time I’ve seen a project hit front page with a “save money with same score on saturated benchmark” claim.

原话 · @Anonymous

I remember reading about strands SDK and it looked great in terms how everything is an event that you can extend, so this harness feels quite about right. However, for this kind of customisation, Pi is actually quite great. One of the most things I love about Pi is ability to ask it to create an extension and it does it quite well as it’s part of their docs. Also ability to customise the system prompt to avoid the clutter that Claude Code add (around 20k system prompt that mostly had nothing to do with the code).