retry makes too few attempts and hides the error

Done

/home/devdi/devswarm/evals/tasks/retry_attempts/repo · 2026-11-09 16:20:06 UTC · took 34.2s

  1. Plan (done)
  2. Code (done)
  3. Test (done)
  4. Review (done)
  5. Finish (done)
Cost
$0.0043
Tests
Passed
Iterations
1
Re-plans
0
LLM calls
12
Prompt read from cache
79%

Usage by agent

Cost per agent
planner: $0.0008 planner planner, Cost: $0.0008 $0.0008 coder: $0.0017 coder coder, Cost: $0.0017 $0.0017 tester: $0.0012 tester tester, Cost: $0.0012 $0.0012 reviewer: $0.0006 reviewer reviewer, Cost: $0.0006 $0.0006
Tokens per agent
  • Cache read
  • Cache write
  • Input
  • Output
planner: 7,512 planner planner, Cache read: 5,533 planner, Cache write: 608 planner, Output: 1,363 7,512 coder: 19.4K coder coder, Cache read: 13.8K coder, Cache write: 3,484 coder, Output: 2,168 19.4K tester: 21.7K tester tester, Cache read: 17.0K tester, Cache write: 3,520 tester, Output: 1,229 21.7K reviewer: 4,740 reviewer reviewer, Cache read: 1,727 reviewer, Cache write: 2,467 reviewer, Output: 542 4,740
View as table
AgentCostCache readCache writeInputOutput
planner$0.00085,53360881,363
coder$0.001713,7673,484102,168
tester$0.001216,9553,520101,229
reviewer$0.00061,7272,4674542

Timeline

  1. planner iteration 1 Finished 3 calls · 7,512 tokens · $0.0008 · 8.9s

    Loading…

  2. coder iteration 1 Finished 4 calls · 19.4K tokens · $0.0017 · 11.2s

    Loading…

  3. tester iteration 1 Finished 4 calls · 21.7K tokens · $0.0012 · 9.6s

    Loading…

  4. verify iteration 1 Passed 0.5s
    ........                                                                 [100%]
    8 passed in 0.02s
    
  5. reviewer iteration 1 Finished 1 calls · 4,740 tokens · $0.0006 · 3.4s

    Loading…

Final diff

diff --git a/retrying.py b/retrying.py
index a49ffae..0aeba93 100644
--- a/retrying.py
+++ b/retrying.py
@@ -1,7 +1,8 @@
 def retry(fn, attempts: int = 3):
-    for _ in range(attempts - 1):
+    last_exc = None
+    for _ in range(max(attempts, 1)):
         try:
             return fn()
-        except Exception:
-            pass
-    return None
+        except Exception as exc:
+            last_exc = exc
+    raise last_exc
diff --git a/test_retrying.py b/test_retrying.py
index 409b641..18e3c26 100644
--- a/test_retrying.py
+++ b/test_retrying.py
@@ -1,5 +1,95 @@
+import pytest
+
 from retrying import retry
 
 
 def test_success_first_time():
     assert retry(lambda: 5) == 5
+
+
+def test_succeeds_after_failures():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        if len(calls) < 3:
+            raise ValueError("not yet")
+        return "ok"
+
+    assert retry(fn, attempts=3) == "ok"
+    assert len(calls) == 3
+
+
+def test_always_failing_calls_attempts_times_and_raises():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        raise RuntimeError(f"fail {len(calls)}")
+
+    with pytest.raises(RuntimeError, match="fail 3"):
+        retry(fn, attempts=3)
+    assert len(calls) == 3
+
+
+def test_fails_first_then_succeeds_with_two_attempts():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        if len(calls) == 1:
+            raise ValueError("first")
+        return 42
+
+    assert retry(fn, attempts=2) == 42
+    assert len(calls) == 2
+
+
+def test_nonpositive_attempts_still_calls_once():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        raise KeyError("x")
+
+    with pytest.raises(KeyError):
+        retry(fn, attempts=0)
+    assert len(calls) == 1
+
+
+def test_default_attempts_is_three():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        raise ValueError("boom")
+
+    with pytest.raises(ValueError):
+        retry(fn)
+    assert len(calls) == 3
+
+
+def test_raises_exact_exception_from_last_attempt():
+    excs = []
+
+    def fn():
+        exc = ValueError(f"attempt {len(excs)}")
+        excs.append(exc)
+        raise exc
+
+    with pytest.raises(ValueError) as info:
+        retry(fn, attempts=4)
+    assert info.value is excs[-1]
+    assert len(excs) == 4
+
+
+def test_base_exception_is_not_retried():
+    calls = []
+
+    def fn():
+        calls.append(1)
+        raise KeyboardInterrupt()
+
+    with pytest.raises(KeyboardInterrupt):
+        retry(fn, attempts=3)
+    assert len(calls) == 1