DevSwarm

All runs / fbbc4395e365

is_valid_email rejects real addresses

Done

/home/devdi/devswarm/evals/tasks/bad_email_regex/repo · 2026-11-09 16:06:58 UTC · took 47.0s

  1. Plan (done)
  2. Code (done)
  3. Test (done)
  4. Review (done)
  5. Finish (done)
Cost
$0.0053
Tests
Passed
Iterations
1
Re-plans
0
LLM calls
14
Prompt read from cache
82%

Usage by agent

Cost per agent
planner: $0.0010 planner planner, Cost: $0.0010 $0.0010 coder: $0.0016 coder coder, Cost: $0.0016 $0.0016 tester: $0.0019 tester tester, Cost: $0.0019 $0.0019 reviewer: $0.0008 reviewer reviewer, Cost: $0.0008 $0.0008
Tokens per agent
  • Cache read
  • Cache write
  • Input
  • Output
planner: 8,287 planner planner, Cache read: 5,817 planner, Cache write: 842 planner, Output: 1,620 8,287 coder: 20.2K coder coder, Cache read: 14.5K coder, Cache write: 3,673 coder, Output: 2,079 20.2K tester: 36.2K tester tester, Cache read: 29.7K tester, Cache write: 4,478 tester, Output: 2,016 36.2K reviewer: 5,117 reviewer reviewer, Cache read: 1,727 reviewer, Cache write: 2,459 reviewer, Output: 927 5,117
View as table
AgentCostCache readCache writeInputOutput
planner$0.00105,81784281,620
coder$0.001614,4673,673102,079
tester$0.001929,7334,478142,016
reviewer$0.00081,7272,4594927

Timeline

  1. planner iteration 1 Finished 3 calls · 8,287 tokens · $0.0010 · 11.3s

    Loading…

  2. coder iteration 1 Finished 4 calls · 20.2K tokens · $0.0016 · 12.7s

    Loading…

  3. tester iteration 1 Finished 6 calls · 36.2K tokens · $0.0019 · 16.8s

    Loading…

  4. verify iteration 1 Passed 0.5s 
    ......................                                                   [100%]
22 passed in 0.03s
  5. reviewer iteration 1 Finished 1 calls · 5,117 tokens · $0.0008 · 5.0s

    Loading…

Final diff

diff --git a/test_validate.py b/test_validate.py
index 722913d..2e2a555 100644
--- a/test_validate.py
+++ b/test_validate.py
@@ -1,3 +1,5 @@
+import pytest
+
 from validate import is_valid_email
 
 
@@ -7,3 +9,53 @@ def test_simple_address():
 
 def test_missing_at():
     assert not is_valid_email("userexample.com")
+
+
+@pytest.mark.parametrize(
+    "address",
+    [
+        "first.last@example.com",
+        "user+tag@example.com",
+        "USER@EXAMPLE.COM",
+        "a@mail.example.co.uk",
+    ],
+)
+def test_valid_real_addresses(address):
+    assert is_valid_email(address)
+
+
+@pytest.mark.parametrize(
+    "address",
+    [
+        "a@.com",
+        "a@example.c",
+        "a b@example.com",
+        "@example.com",
+        "a@example",
+        "user@example.com\n",
+    ],
+)
+def test_invalid_addresses(address):
+    assert not is_valid_email(address)
+
+
+@pytest.mark.parametrize(
+    "address",
+    [
+        "a@-x.com",
+        "a@x-.com",
+        "a@x..com",
+        "a@x.com.",
+        "a@@x.com",
+        "a@ex_ample.com",
+        "a@x.c0m",
+        "a@x.com\r",
+        "",
+    ],
+)
+def test_invalid_domain_edge_cases(address):
+    assert not is_valid_email(address)
+
+
+def test_hyphen_inside_domain_label_is_valid():
+    assert is_valid_email("a@x-y.com")
diff --git a/validate.py b/validate.py
index 1141430..992666e 100644
--- a/validate.py
+++ b/validate.py
@@ -1,7 +1,12 @@
 import re
 
-_EMAIL = re.compile(r"^[a-z0-9]+@[a-z0-9]+\.[a-z]{2,}$")
+_EMAIL = re.compile(
+    r"[A-Za-z0-9._%+-]+"
+    r"@"
+    r"(?:[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?\.)+"
+    r"[A-Za-z]{2,}"
+)
 
 
 def is_valid_email(address: str) -> bool:
-    return _EMAIL.match(address) is not None
+    return _EMAIL.fullmatch(address) is not None