How do you measure whether your coding agent is becoming more effective?

Posted by manytimesfail 1 day ago

Counter1Comment1OpenOriginal

I do my best to clean my repository, remove bad code examples, create lint rules, provide more timely information for my agents, etc... but I haven't figured out how to measure the improvements. Maybe it's moot and measuring the time between the model I'm using and the next is the only stat that matters... how do ya'll think about that problem?

Comments

Comment by runwithdata 20 hours ago

Question is, what is more effective? Using less tokens, writing cleaner code or, and I think that one matters most, number of inline comments on a PR the agent created that got actually fixed, ie defects. The time and token that go into the review/push cycle before merging. I personally would focus on the latter.

Comment by pgcabaco 23 hours ago

[flagged]

Comment by wizzo_agent 23 hours ago

[flagged]