Explore + Share Site

Test/verification technique effectiveness with Astra

“On (1), we previously saw that, when implementing Zstd from the RFC with 5.6 Sol, on average, agents spent $3.50 on medium effort and $5.50 on xhigh effort to get full (holdout) test correctness about 63% and 77% of the”

Jump to Highlight ↗
When we looked at how well agents used test/verification techniques in August1, we saw that GPT-5.6 Sol generally did what you might call a caricature of various techniques when asked to use a technique or library such as TDD, QuickCheck, Lean 4, etc.; not specifying any particular test method (which we'll call Default) resulted in many worthless unit tests, which was one of the best performing conditions because asking agents to test in specific ways mostly made them worse.
Quick Use-Case Tags (tap to vote)
Visit Website ↗

📝 Your Private Notes & Highlights 🔒 Only visible to you

Log in or create a free account to keep private highlights and personal notes on your saved pins.

Creator Trophy & Press Kit

Featured on Storesso

Are you the creator of danluu.com? Share your verified feature badge on your website, or export a custom 4:5 editorial social card for Instagram and X.

Verified Website Badge
Featured on Storesso
Claim Resource →
4:5 Social Studio

Generate a 1080×1350 vertical high-res feature card tailored for social sharing.

Instagram & X Feature Card

Export a polished, high-resolution 4:5 editorial graphic displaying your verified inclusion in the Storesso Discovery Index. Perfect for launch threads, founder updates, and newsletters.

Rendered at 1080×1350px (Retina ready) · No attribution watermark required.

Curated in These Public Collections

Discussion 0
🛡️ Safe comments: Text, Emojis & Reaction GIFs only (links & media URLs blocked)
✨ Author / Creator Quick-Reply No password required · Or Sign In