Exunit Code Review

ExUnit Code Review

Quick Reference

Mock Boundary Philosophy

Mock at external boundaries:

• HTTP clients, external APIs, third-party services
• Slow resources: file system, email, job queues
• Non-deterministic: DateTime.utc_now(), :rand

DO NOT mock internal code:

• Contexts, schemas, GenServers
• Internal modules, PubSub
• Anything you wrote

Review Checklist

Test Structure

• [ ] Tests are async: true unless sharing database state
• [ ] Describe-blocks group related tests
• [ ] Setup extracts common test data
• [ ] Tests have clear arrange/act/assert structure

Mocking

• [ ] Mox used for external boundaries (HTTP, APIs)
• [ ] Behaviors defined for mockable interfaces
• [ ] No mocking of internal modules
• [ ] verify_on_exit! in setup for strict mocking

Test Adapters

• [ ] Bypass for HTTP endpoint mocking
• [ ] Swoosh.TestAdapter for email testing
• [ ] Oban.Testing for background job assertions

Database

• [ ] Ecto.Adapters.SQL.Sandbox for isolation
• [ ] Async tests don't share database state
• [ ] Fixtures/factories used consistently

Valid Patterns (Do NOT Flag)

• Mock in unit test, real in integration - Different test levels have different needs
• Not mocking database in integration tests - Database is internal
• Simple inline test data - Not everything needs factories
• Testing private functions via public API - Correct approach

Context-Sensitive Rules

Gates (sequence)

Complete in order. Do not emit a finding until the prior step passes for that issue.

1. Evidence from the file — Open the test module (or helper) and tie the claim to concrete lines.

• Pass when: Each prospective finding includes [FILE:LINE] and a one-line factual description of what is on that line (or an adjacent line you name), not a generic style complaint.

2. ExUnit false-positive veto — Check this skill’s Valid Patterns and Context-Sensitive Rules for the case.

• Pass when: You can state “not covered by Do NOT Flag / Flag ONLY IF” in one sentence, or you drop the finding.

3. Cross-protocol verification — Apply review-verification-protocol (e.g. read full function/block, search usages before “unused” claims) to that same finding.

• Pass when: At least one protocol check relevant to the claim type is satisfied and would appear in your rationale if challenged.

Before Submitting Findings

Use [FILE:LINE] ISSUE_TITLE per finding after Gates (sequence) and the linked protocol are satisfied.

ExUnit Patterns

Async Tests

Default to Async

# GOOD - isolated test
defmodule MyApp.CalculatorTest do
  use ExUnit.Case, async: true

  test "adds numbers" do
    assert Calculator.add(1, 2) == 3
  end
end

When to Disable Async

# Sharing database with other tests
defmodule MyApp.UserTest do
  use MyApp.DataCase  # Sets async: false if needed

  test "creates user" do
    assert {:ok, _} = Accounts.create_user(%{email: "test@example.com"})
  end
end

Describe Blocks

Group Related Tests

defmodule MyApp.UserTest do
  use MyApp.DataCase

  describe "create_user/1" do
    test "with valid attrs creates user" do
      assert {:ok, user} = Accounts.create_user(valid_attrs())
      assert user.email == "test@example.com"
    end

    test "with invalid email returns error" do
      assert {:error, changeset} = Accounts.create_user(%{email: "invalid"})
      assert "is invalid" in errors_on(changeset).email
    end

    test "with duplicate email returns error" do
      Accounts.create_user(valid_attrs())
      assert {:error, changeset} = Accounts.create_user(valid_attrs())
      assert "has already been taken" in errors_on(changeset).email
    end
  end

  describe "authenticate_user/2" do
    # ...
  end
end

Setup

Shared Setup

defmodule MyApp.PostTest do
  use MyApp.DataCase

  setup do
    user = insert(:user)
    {:ok, user: user}
  end

  test "creates post for user", %{user: user} do
    assert {:ok, post} = Posts.create_post(user, %{title: "Test"})
    assert post.user_id == user.id
  end
end

Setup per Describe

describe "admin functions" do
  setup do
    admin = insert(:user, role: :admin)
    {:ok, admin: admin}
  end

  test "admin can delete", %{admin: admin} do
    # ...
  end
end

describe "user functions" do
  setup do
    user = insert(:user, role: :user)
    {:ok, user: user}
  end

  test "user cannot delete", %{user: user} do
    # ...
  end
end

Tags

Skip Tests

@tag :skip
test "not implemented yet" do
end

# Run: mix test --exclude skip

Slow Tests

@tag :slow
test "integration with external service" do
end

# Run: mix test --exclude slow
# Or: mix test --only slow

Custom Tags

@tag :integration
test "full workflow" do
end

# In test_helper.exs
ExUnit.configure(exclude: [:integration])

# Run: mix test --include integration

Assertions

Pattern Matching

# GOOD - precise matching
assert {:ok, %User{email: "test@example.com"}} = Accounts.create_user(attrs)

# GOOD - extract for further assertions
assert {:ok, user} = Accounts.create_user(attrs)
assert user.email == "test@example.com"
assert user.confirmed_at == nil

Refute

test "does not include deleted" do
  refute deleted_user in Accounts.list_active_users()
end

Assert Raise

test "raises on invalid input" do
  assert_raise ArgumentError, ~r/must be positive/, fn ->
    Calculator.sqrt(-1)
  end
end

Review Questions

1. Are tests async when possible? 2. Are describe blocks used to group related tests? 3. Is setup used to reduce duplication? 4. Are assertions using pattern matching effectively?

Integration Tests

What to Mock vs Real

Mock at External Boundaries Only

Example Integration Test

defmodule MyAppWeb.RegistrationFlowTest do
  use MyAppWeb.ConnCase
  use Swoosh.TestAssertions

  setup %{conn: conn} do
    bypass = Bypass.open()
    # Mock only external HTTP calls
    Application.put_env(:my_app, :verification_api_url, "http://localhost:#{bypass.port}")
    {:ok, conn: conn, bypass: bypass}
  end

  test "full registration flow", %{conn: conn, bypass: bypass} do
    # Mock external email verification API
    Bypass.expect_once(bypass, "POST", "/verify", fn conn ->
      Plug.Conn.resp(conn, 200, ~s({"valid": true}))
    end)

    # Test real controller -> context -> repo flow
    conn = post(conn, ~p"/register", %{
      user: %{email: "test@example.com", password: "password123"}
    })

    assert redirected_to(conn) == ~p"/welcome"

    # Verify real database state
    assert user = Repo.get_by(User, email: "test@example.com")
    assert user.confirmed_at == nil

    # Verify real email was "sent" via test adapter
    assert_email_sent(to: "test@example.com", subject: "Confirm your account")
  end
end

Ecto Sandbox

Configuration

# test/support/data_case.ex
defmodule MyApp.DataCase do
  use ExUnit.CaseTemplate

  using do
    quote do
      alias MyApp.Repo
      import Ecto
      import Ecto.Changeset
      import Ecto.Query
      import MyApp.DataCase
    end
  end

  setup tags do
    MyApp.DataCase.setup_sandbox(tags)
    :ok
  end

  def setup_sandbox(tags) do
    pid = Ecto.Adapters.SQL.Sandbox.start_owner!(MyApp.Repo, shared: not tags[:async])
    on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
  end
end

Async vs Shared Mode

# Async - each test gets its own transaction
defmodule MyApp.FastTest do
  use MyApp.DataCase, async: true  # Isolated, can run parallel
end

# Shared - tests share database connection
defmodule MyApp.SlowTest do
  use MyApp.DataCase, async: false  # Sequential, shared state
end

Allowing Processes

test "async process accesses database" do
  # Allow spawned process to use test's database connection
  Ecto.Adapters.SQL.Sandbox.allow(Repo, self(), some_pid)

  # Or use :shared mode
  Ecto.Adapters.SQL.Sandbox.mode(Repo, :shared)
end

Test Data

Fixtures vs Factories

# Fixture - simple helper functions
defmodule MyApp.TestHelpers do
  def user_fixture(attrs \\ %{}) do
    {:ok, user} =
      attrs
      |> Enum.into(%{
        email: "test#{System.unique_integer()}@example.com",
        password: "password123"
      })
      |> Accounts.create_user()

    user
  end
end

# Factory - with ex_machina
defmodule MyApp.Factory do
  use ExMachina.Ecto, repo: MyApp.Repo

  def user_factory do
    %MyApp.Accounts.User{
      email: sequence(:email, &"user#{&1}@example.com"),
      password_hash: Bcrypt.hash_pwd_salt("password123")
    }
  end
end

LiveView Integration Tests

Testing Async Assigns

defmodule MyAppWeb.DashboardLiveTest do
  use MyAppWeb.ConnCase
  import Phoenix.LiveViewTest

  test "loads data asynchronously", %{conn: conn} do
    {:ok, view, html} = live(conn, ~p"/dashboard")

    # Initial render shows loading state
    assert html =~ "Loading..."

    # Wait for async to complete
    assert render_async(view) =~ "Dashboard Data"
  end
end

Testing Events

test "handles form submission", %{conn: conn} do
  {:ok, view, _html} = live(conn, ~p"/posts/new")

  view
  |> form("#post-form", post: %{title: "Test", body: "Content"})
  |> render_submit()

  assert_redirect(view, ~p"/posts")
  assert Repo.get_by(Post, title: "Test")
end

Review Questions

1. Are only external boundaries mocked in integration tests? 2. Is Ecto sandbox properly configured? 3. Are async tests truly isolated? 4. Is test data created consistently (fixtures or factories)?

Mox Boundaries

Core Principle

Mock at external boundaries, not internal code.

Setting Up Mox

Define Behaviors

# lib/my_app/http_client.ex
defmodule MyApp.HTTPClient do
  @callback get(String.t()) :: {:ok, map()} | {:error, term()}
  @callback post(String.t(), map()) :: {:ok, map()} | {:error, term()}
end

# lib/my_app/http_client/hackney.ex
defmodule MyApp.HTTPClient.Hackney do
  @behaviour MyApp.HTTPClient

  @impl true
  def get(url), do: # real implementation

  @impl true
  def post(url, body), do: # real implementation
end

Configure Mock

# test/support/mocks.ex
Mox.defmock(MyApp.HTTPClientMock, for: MyApp.HTTPClient)

# config/test.exs
config :my_app, http_client: MyApp.HTTPClientMock

# config/prod.exs
config :my_app, http_client: MyApp.HTTPClient.Hackney

Inject Dependency

defmodule MyApp.ExternalService do
  @http_client Application.compile_env(:my_app, :http_client)

  def fetch_data(id) do
    @http_client.get("/api/data/#{id}")
  end
end

Writing Tests with Mox

Basic Expectation

defmodule MyApp.ExternalServiceTest do
  use ExUnit.Case, async: true

  import Mox

  setup :set_mox_from_context
  setup :verify_on_exit!

  test "fetches data successfully" do
    expect(MyApp.HTTPClientMock, :get, fn "/api/data/123" ->
      {:ok, %{"name" => "Test"}}
    end)

    assert {:ok, %{"name" => "Test"}} = ExternalService.fetch_data(123)
  end
end

Multiple Calls

test "retries on failure" do
  MyApp.HTTPClientMock
  |> expect(:get, fn _ -> {:error, :timeout} end)
  |> expect(:get, fn _ -> {:ok, %{}} end)

  assert {:ok, _} = ExternalService.fetch_with_retry(123)
end

Stub for Default Behavior

setup do
  stub(MyApp.HTTPClientMock, :get, fn _ -> {:ok, %{}} end)
  :ok
end

Allow for Async

test "async process uses mock" do
  parent = self()
  ref = make_ref()

  expect(MyApp.HTTPClientMock, :get, fn _ ->
    send(parent, {ref, :called})
    {:ok, %{}}
  end)

  # Start the task first to get its pid
  task = Task.async(fn ->
    ExternalService.fetch_data(123)
  end)

  # Parent grants permission to the child process BEFORE it uses the mock
  Mox.allow(MyApp.HTTPClientMock, parent, task.pid)
  Task.await(task)

  assert_receive {^ref, :called}
end

Anti-Patterns

DON'T Mock Internal Modules

# BAD - mocking your own context
defmock(MyApp.AccountsMock, for: MyApp.Accounts)

test "controller creates user" do
  expect(MyApp.AccountsMock, :create_user, fn _ -> {:ok, %User{}} end)
  # This tests nothing meaningful!
end

# GOOD - test the real context
test "controller creates user" do
  conn = post(conn, ~p"/users", %{user: valid_attrs()})
  assert %{"id" => _} = json_response(conn, 201)
  assert Repo.get_by(User, email: "test@example.com")
end

DON'T Mock Database

# BAD
defmock(MyApp.RepoMock, for: Ecto.Repo)

# GOOD - use Ecto.Adapters.SQL.Sandbox
use MyApp.DataCase

Review Questions

1. Are only external boundaries being mocked? 2. Are behaviors defined for mockable interfaces? 3. Is verify_on_exit! used in setup? 4. Are internal modules tested with real implementations?

Test Adapters

Bypass for HTTP

Setup

# mix.exs
{:bypass, "~> 2.1", only: :test}

Basic Usage

defmodule MyApp.APIClientTest do
  use ExUnit.Case, async: true

  setup do
    bypass = Bypass.open()
    {:ok, bypass: bypass}
  end

  test "fetches user data", %{bypass: bypass} do
    Bypass.expect_once(bypass, "GET", "/users/123", fn conn ->
      Plug.Conn.resp(conn, 200, ~s({"id": 123, "name": "Test"}))
    end)

    assert {:ok, user} = APIClient.get_user("http://localhost:#{bypass.port}", 123)
    assert user.name == "Test"
  end

  test "handles server error", %{bypass: bypass} do
    Bypass.expect_once(bypass, "GET", "/users/123", fn conn ->
      Plug.Conn.resp(conn, 500, "Internal Server Error")
    end)

    assert {:error, :server_error} = APIClient.get_user("http://localhost:#{bypass.port}", 123)
  end
end

Verify Request Body

test "sends correct payload", %{bypass: bypass} do
  Bypass.expect_once(bypass, "POST", "/webhooks", fn conn ->
    {:ok, body, conn} = Plug.Conn.read_body(conn)
    assert %{"event" => "user.created"} = Jason.decode!(body)
    Plug.Conn.resp(conn, 200, "OK")
  end)

  Webhooks.notify(:user_created, %{id: 1})
end

Swoosh for Email

Configuration

# config/test.exs
config :my_app, MyApp.Mailer, adapter: Swoosh.Adapters.Test

Testing Emails

defmodule MyApp.NotificationsTest do
  use ExUnit.Case, async: true
  use Swoosh.TestAssertions

  test "sends welcome email" do
    user = %{email: "test@example.com", name: "Test"}

    Notifications.send_welcome(user)

    assert_email_sent(
      to: "test@example.com",
      subject: "Welcome!"
    )
  end

  test "includes user name in body" do
    user = %{email: "test@example.com", name: "Alice"}

    Notifications.send_welcome(user)

    assert_email_sent(fn email ->
      assert email.html_body =~ "Hello, Alice"
    end)
  end
end

Oban for Jobs

Configuration

# config/test.exs
config :my_app, Oban, testing: :inline  # Jobs run immediately
# OR
config :my_app, Oban, testing: :manual  # Control when jobs run

Testing Job Enqueue

defmodule MyApp.WorkerTest do
  use MyApp.DataCase
  use Oban.Testing, repo: MyApp.Repo

  test "enqueues email job" do
    Notifications.schedule_email(user_id: 123)

    assert_enqueued(worker: MyApp.Workers.EmailWorker, args: %{user_id: 123})
  end

  test "job processes correctly" do
    assert :ok = perform_job(MyApp.Workers.EmailWorker, %{user_id: 123})
  end
end

Testing with :manual mode

# config/test.exs
config :my_app, Oban, testing: :manual

test "job side effects" do
  # Enqueue job
  Notifications.schedule_email(user_id: 123)

  # Job not yet run
  refute_email_sent()

  # Manually drain the queue
  Oban.drain_queue(queue: :mailers)

  # Now email sent
  assert_email_sent(to: "test@example.com")
end

DateTime Mocking

Simple Approach

# In code, accept optional time
def expires_at(duration, now \\ DateTime.utc_now()) do
  DateTime.add(now, duration, :second)
end

# In test
test "calculates expiry" do
  now = ~U[2024-01-01 12:00:00Z]
  assert Token.expires_at(3600, now) == ~U[2024-01-01 13:00:00Z]
end

With Mox

# Define behavior
defmodule MyApp.Clock do
  @callback utc_now() :: DateTime.t()
end

# Production implementation
defmodule MyApp.Clock.System do
  @behaviour MyApp.Clock
  def utc_now, do: DateTime.utc_now()
end

# Test mock
Mox.defmock(MyApp.ClockMock, for: MyApp.Clock)

# In test
expect(MyApp.ClockMock, :utc_now, fn -> ~U[2024-01-01 12:00:00Z] end)

Review Questions

1. Is Bypass used for HTTP endpoint mocking? 2. Is Swoosh.TestAdapter configured for email tests? 3. Is Oban.Testing used for job assertions? 4. Are time-dependent tests properly controlled?