is_valid_email rejects real addresses

Done

/home/devdi/devswarm/evals/tasks/bad_email_regex/repo · 2026-11-09 16:07:46 UTC · took 36.0s

  1. Plan (done)
  2. Code (done)
  3. Test (done)
  4. Review (done)
  5. Finish (done)
Cost
$0.0042
Tests
Passed
Iterations
1
Re-plans
0
LLM calls
11
Prompt read from cache
81%

Usage by agent

Cost per agent
planner: $0.0010 planner planner, Cost: $0.0010 $0.0010 coder: $0.0015 coder coder, Cost: $0.0015 $0.0015 tester: $0.0017 tester tester, Cost: $0.0017 $0.0017
Tokens per agent
  • Cache read
  • Cache write
  • Input
  • Output
planner: 8,329 planner planner, Cache read: 5,813 planner, Cache write: 840 planner, Output: 1,668 8,329 coder: 20.3K coder coder, Cache read: 15.0K coder, Cache write: 3,487 coder, Output: 1,779 20.3K tester: 24.9K tester tester, Cache read: 18.3K tester, Cache write: 4,779 tester, Output: 1,838 24.9K
View as table
AgentCostCache readCache writeInputOutput
planner$0.00105,81384081,668
coder$0.001514,9953,487101,779
tester$0.001718,2834,779101,838

Timeline

  1. planner iteration 1 Finished 3 calls · 8,329 tokens · $0.0010 · 10.7s

    Loading…

  2. coder iteration 1 Finished 4 calls · 20.3K tokens · $0.0015 · 11.0s

    Loading…

  3. tester iteration 1 Finished 4 calls · 24.9K tokens · $0.0017 · 13.2s

    Loading…

  4. verify iteration 1 Passed 0.5s
    ................                                                         [100%]
    16 passed in 0.02s
    

Final diff

diff --git a/test_validate.py b/test_validate.py
index 722913d..801b73c 100644
--- a/test_validate.py
+++ b/test_validate.py
@@ -7,3 +7,67 @@ def test_simple_address():
 
 def test_missing_at():
     assert not is_valid_email("userexample.com")
+
+
+def test_dot_and_plus_in_local_part():
+    assert is_valid_email("first.last@example.com")
+    assert is_valid_email("user+tag@example.com")
+
+
+def test_uppercase():
+    assert is_valid_email("USER@EXAMPLE.COM")
+
+
+def test_multi_part_domain():
+    assert is_valid_email("a@mail.example.co.uk")
+
+
+def test_no_dot_in_domain():
+    assert not is_valid_email("user@example")
+    assert not is_valid_email("a@example")
+
+
+def test_space():
+    assert not is_valid_email("a b@example.com")
+    assert not is_valid_email("user name@example.com")
+
+
+def test_empty_local_part():
+    assert not is_valid_email("@example.com")
+
+
+def test_empty_domain_label():
+    assert not is_valid_email("a@.com")
+
+
+def test_one_letter_tld():
+    assert not is_valid_email("a@example.c")
+
+
+def test_trailing_newline():
+    assert not is_valid_email("user@example.com\n")
+
+
+def test_leading_newline_and_tab():
+    assert not is_valid_email("\nuser@example.com")
+    assert not is_valid_email("user@example.com\t")
+
+
+def test_hyphen_in_domain_label():
+    assert is_valid_email("a@x-y.com")
+    assert not is_valid_email("a@-x.com")
+    assert not is_valid_email("a@x-.com")
+
+
+def test_invalid_domain_characters():
+    assert not is_valid_email("user@exa_mple.com")
+    assert not is_valid_email("user@example.com.")
+    assert not is_valid_email("user@example.c0m")
+
+
+def test_multiple_at_signs():
+    assert not is_valid_email("a@b@c.com")
+
+
+def test_non_ascii_rejected():
+    assert not is_valid_email("ü@example.com")
diff --git a/validate.py b/validate.py
index 1141430..6994d90 100644
--- a/validate.py
+++ b/validate.py
@@ -1,7 +1,9 @@
 import re
 
-_EMAIL = re.compile(r"^[a-z0-9]+@[a-z0-9]+\.[a-z]{2,}$")
+_EMAIL = re.compile(
+    r"[A-Za-z0-9._%+-]+@(?:[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?\.)+[A-Za-z]{2,}"
+)
 
 
 def is_valid_email(address: str) -> bool:
-    return _EMAIL.match(address) is not None
+    return _EMAIL.fullmatch(address) is not None