--- since: 1.2.1 --- # DjangoPlay — `mailer` App > Doc's verified callers/config sources: `paystream/apps.py`, `paystream/app_settings/email.py`, `webapp/configs/email_flow_limits.json` (top-level shared config — see note below), `users/adapters/`, `users/views/ui/resend_verification.py`, `users/admin/signup_request.py`, and `utilities.constants.template_registry.TemplateRegistry`. ## 1. Summary `mailer` is DjangoPlay's **centralized transactional email subsystem** — every outbound email in the platform (signup, verification, password reset, support tickets, bug reports, unsubscribe) is designed to funnel through one of two entry points this app owns: `EmailEngine.send()` (the low-level rendering/sending engine) or `send_email_via_adapter()` (a thin wrapper that routes through `users.adapters.accounts.custom.CustomAccountAdapter.send_mail`, used by Celery tasks). It owns: - **One model**, `EmailDelivery` — a generic, domain-agnostic delivery-lifecycle tracker (queued/sent/failed) that other apps' email flows create rows in but that itself "does NOT know" about signup, password reset, support, or bugs (per its own docstring — it's pure infrastructure). - **The actual email-sending pipeline**: template resolution with fallback rules, inline CID image attachment, unsubscribe enforcement, per-flow rate limiting, and delivery-status tracking. - **Six Celery task flows** (`flows/`) — one per real-world email trigger — plus supporting link-builders (`links/`) and a two-tier rate-limiter (`throttling/`). This app is explicitly infra-only by design (its own module docstrings repeat this): `EmailEngine` "intentionally contains NO business logic for password reset, signup flow, onboarding, bug/support throttles — all business logic must stay in services," and `EmailDelivery` is deliberately decoupled from any specific flow. In practice this holds up well — but this pass also found a meaningful amount of **dead, broken, or silently-discarded code** sitting alongside the working pipeline (§7), including one bug that currently breaks a real admin action outright. --- ## 2. Architecture ### 2.1 The two send paths, and how they converge ``` Path A — Adapter-driven (allauth's own send_mail hook) ──────────────────────────────────────────────────────── CustomAccountAdapter.send_mail(template_prefix, email, context) [users app] │ (strips allauth's own subject/email_subject keys so EmailEngine owns them) ▼ EmailEngine.send(prefix, email, context, request=None, user=None) Path B — Celery task-driven (used by flows/*.py) ──────────────────────────────────────────────────────── send_email_via_adapter(template_prefix=..., to_email=..., context=..., user=..., request=...) [mailer.engine.base] │ calls django-allauth's get_adapter() → same CustomAccountAdapter instance ▼ adapter.send_mail(template_prefix, to_email, context) │ ▼ EmailEngine.send(...) ← SAME destination as Path A ``` Both paths converge on the exact same `EmailEngine.send()` call — there is genuinely one email-sending implementation in this app, not two competing ones. The two paths exist because allauth itself calls `adapter.send_mail()` directly for its own built-in flows (password reset key, email confirmation), while DjangoPlay's own Celery tasks need a callable that doesn't require an allauth-internal call context — `send_email_via_adapter` is that adapter-shaped wrapper for task code. ### 2.2 `EmailEngine.send()` — the actual pipeline, in order ``` 1. Normalize email (lowercase/strip), generate a short mail_id for log correlation 2. Resolve template prefix aliases (password_reset_key → PASSWORD_RESET_EMAIL, etc.) via TemplateResolver._normalize_prefix — allauth's prefixes get mapped to DjangoPlay's own TemplateRegistry constants 3. Hardcoded skip: email == "redstar@djangoplay.org" → return None, no email sent at all (see the users app doc §6.1 for the wider pattern this belongs to) 4. Resolve a "user" object from context (explicit param wins > context["member"].user_identity > context["employee"] > nothing) 5. Inject default context (site_name, verification/reset/unsubscribe expiry days from settings.LINK_EXPIRY_DAYS) + canonical EmailContextProvider design-token context (button colors, fonts — see the users app doc, adapters/context/email.py) 6. Hard invariant check: PASSWORD_RESET_EMAIL must have context["reset_url"] or raises RuntimeError — but then immediately below, if a user is present, calls PasswordResetContextProvider.inject_password_reset_context(), which REBUILDS reset_url from the DB token via reverse(), overwriting whatever was passed in (see §7.3 — the value passed by the calling task is effectively discarded) 7. Unsubscribe enforcement: UnsubscribeService.is_allowed(user, prefix) — if False, returns None silently (no exception, no error — a suppressed send looks identical to a successful no-op from the caller's perspective, since both return whatever msg.send() would return, and both non-error paths return None) 8. Inject unsubscribe_url if not already present + SupportContextProvider context (support email/phone/location/social links) 9. validate_mail_context(prefix, context) — checks required keys per template (using EmailEngine.py's OWN inline REQUIRED_MAIL_CONTEXT dict — NOT the one in mailer/constants/context_requirements.py, which is dead code, see §7.1) 10. TemplateResolver renders subject (.txt), body (.txt), and body (.html) — HTML has no fallback (hard error if missing); subject/text fall back to account/fallback/{prefix}{suffix} if account/email/{prefix}{suffix} is missing 11. Build EmailMultiAlternatives; from_email is DEFAULT_FROM_EMAIL only for PASSWORD_RESET_EMAIL (a hardcoded "no-reply" set), SUPPORT_EMAIL for everything else 12. Always attach the DjangoPlay logo (icon + text) as inline CID images. EMAIL_SIGNUP_SUCCESS additionally attaches a founder/profile signature photo. Contact icons (phone/email/location/linkedin/github) exist as a documented capability (InlineImageService.attach_contact_icons) but are NOT called from this pipeline — the code comment explains they were the historical cause of spam classification and were deliberately dropped in favor of plain text 13. msg.send() — network/SMTP errors (socket.gaierror, smtplib.SMTPException, OSError) are caught, logged as warnings, optionally surfaced via Django messages framework if a request is present, and return None (not re-raised). Any OTHER exception during send is logged as an error and re-raised. ``` ### 2.3 Rate limiting — two independent layers, deliberately different scopes ``` Layer 1: flow_throttle.allow_flow(flow, user_id, email, client_ip, prefer_user_identity) - Checks THREE limits in order: burst_ip → per_ip → (per_user OR per_email) - Config resolution: settings.EMAIL_FLOW_LIMITS[flow] → [...]["default"] → hardcoded DEFAULTS - settings.EMAIL_FLOW_LIMITS itself resolves (paystream/app_settings/email.py): env var EMAIL_FLOW_LIMITS (JSON) → webapp/configs/email_flow_limits.json → FALLBACK_EMAIL_FLOW_LIMITS - Called explicitly by PasswordResetService (users app) and resend_verification_for_email — i.e. it's opt-in per call site, not automatically applied to every send Layer 2: throttle.check_and_increment_email_limit(flow, max_total, user_id/email/client_ip, ttl_seconds) - The actual atomic counter Layer 1 calls into — Django cache-backend counters (cache.incr/cache.add/cache.touch), keyed by SHA-256(email) when identifying by email (not raw email — a deliberate privacy-conscious choice for cache key content) - Also called directly and independently by admin code (check_and_increment_email_limit(flow="signup_request_admin", ...) in users/admin/signup_request.py) — bypassing flow_throttle's IP/burst layering entirely and using only the raw counter with a hardcoded max_limit + 86400s TTL ``` **Not every send goes through rate limiting.** `EmailEngine.send()` itself has no throttle check built in — throttling is applied by whichever calling service chooses to call `allow_flow()` or `check_and_increment_email_limit()` before invoking a send. This is consistent with the app's "no business logic in the engine" design principle, but it does mean rate-limit coverage is only as complete as each individual call site remembered to add it — not structurally guaranteed by `mailer` itself. --- ## 3. Functionality — the Six Email Flows | Flow (`flows/`) | Trigger | Celery task? | `EmailDelivery` rows created | |---|---|---|---| | `member/signup.py` → `send_successful_signup_email_task` | New member created (manual signup, SSO signup) | Yes (`@shared_task`) | `EMAIL_SIGNUP_SUCCESS`; additionally `EMAIL_SSO_ONBOARDING` if the user's `sso_provider != "EMAIL"` | | `member/verification.py` → `send_verification_email_task` | Resend/initial email verification (async path) | Yes | `EMAIL_VERIFICATION_MANUAL` | | `member/verification.py` → `send_manual_verification_email_task` | Admin-triggered "resend verification" from the `SignUpRequest` admin page | Yes, but **called synchronously, not via `.delay()`, and with a broken call signature — see §7.5** | `EMAIL_VERIFICATION_MANUAL` (when it doesn't crash first) | | `password_reset.py` → `send_password_reset_email_task` | `PasswordResetService.send_reset_link` (users app) | Yes | `PASSWORD_RESET_EMAIL` | | `resend_verification.py` → `resend_verification_for_email` | User-facing "resend verification" form (`users/views/ui/resend_verification.py`) | **No — plain function, not a Celery task despite the name and its placement in `tasks.py`'s "Celery task registry"** (§7.4). Internally dispatches the real async `send_verification_email_task.delay(...)`. | Whatever `send_verification_email_task` creates once it runs | | `support.py` → `send_support_ticket_email_task` | New `helpdesk.SupportTicket` created | Yes, `max_retries=3` | `REQUEST_TO_SUPPORT_EMAIL` (to `SUPPORT_EMAIL`) + `CONFIRMATION_TO_USER_EMAIL` (to reporter) — both checked for an existing `SENT` delivery first (idempotency guard against duplicate sends on retry) | | `bug.py` → `send_bug_report_email_task` | New `helpdesk.BugReport` created | Yes, `max_retries=3` | Same two-email pattern as `support.py`, plus an `ISSUES_TRACKER_URL` context var built via `paystream.services.context_processors.site_context.build_subdomain_url("issues", "/")` | All six are re-exported through `mailer/tasks.py`, whose own docstring states it "exists ONLY to expose mailer-owned tasks to Celery autodiscovery" — worth noting this docstring is slightly inaccurate since one of the six (`resend_verification_for_email`) isn't actually a task (§7.4). ### 3.1 Support/bug ticket flows — retry-safe by design `support.py` and `bug.py` follow an identical, deliberately conservative pattern: before sending, they check whether an `EmailDelivery` row already exists with `status=SENT` for that specific ticket/bug + template combination, and skip entirely if so. Combined with `max_retries=3` on the Celery task itself, this means a task retry after a partial failure (e.g. admin notification sent, reporter confirmation failed) won't re-send the admin notification — each of the two emails in the pair is individually idempotency-checked, not just the task as a whole. ### 3.2 Signup flow — a documented three-step sequence, but only two steps actually run `send_successful_signup_email_task`'s docstring says it sends, in order: (1) signup success email, (2) SSO onboarding email (SSO users only), (3) "Verification emails are handled separately by verification service." The code matches this — but a large block of commented-out code at the bottom of the file (roughly 70 lines) shows an **earlier version** of this same task used to also send the verification/SSO-onboarding email itself as a "step 2," before that responsibility was split out into the now-separate `send_verification_email_task`. The dead code is harmless (never executes) but is worth removing — it currently reads as ambiguous about which system actually owns sending the verification email during signup. --- ## 4. Data Model `EmailDelivery` (`models/email_delivery.py`) — the only model in this app: - `task_id` (indexed) — the Celery task ID that created this row, when known. - `template_prefix` (indexed) — which `TemplateRegistry` constant this delivery is for. - `to_email` (indexed). - `status` — `QUEUED` (default) / `SENT` / `FAILED`. - `failure_reason` — free text, truncated to 5000 chars on `mark_failed()`. - `queued_at`, `sent_at`, `failed_at`. - `metadata` (JSONField) — flow-specific context (`member_id`, `user_id`, `ticket_id`, `bug_id`, `flow` label, `provider`, etc.) — this is how a generic model stays queryable per-flow without owning flow-specific columns. - `objects` = `ActiveManager` (excludes soft-deleted, inherited via `TimeStampedModel`/`AuditFieldsModel`), `all_objects` = plain manager. - `mark_sent()` / `mark_failed(reason)` — both use `update_fields=[...]` (never a bare `.save()`), and both log via the dedicated `mailer.events`/`mailer.errors` loggers (`constants/loggers.py`). **Not every task creates a delivery row before attempting to send.** `send_manual_verification_email_task` and `resend_verification_for_email` (the two verification-resend paths) never construct an `EmailDelivery` at all — only `send_verification_email_task` (the one they eventually call into for the actual send) does. So delivery tracking exists for the *outcome* of a resend, but not distinctly for each *resend attempt* — a resend that fails before reaching `send_verification_email_task` (e.g. because no active `SignUpRequest` exists) leaves no `EmailDelivery` trace at all, only a log line. `services/email_delivery_status_service.get_latest_email_delivery_status(email, template_prefix)` is the one read-side query helper — a simple "most recent delivery for this email+template" lookup, presumably used by a status-check UI or admin view (not traced further in this pass). --- ## 5. Configuration | Setting | Source | Purpose | |---|---|---| | `EMAIL_MODE` / `EMAIL_BACKEND` | `paystream/app_settings/email.py`, env `EMAIL_MODE` (`smtp` / `console` / `disabled`) | Lets local dev print emails to console or fully disable sending without touching SMTP credentials. | | `EMAIL_HOST`, `EMAIL_PORT`, `EMAIL_HOST_USER`, `EMAIL_HOST_PASSWORD`, `DEFAULT_FROM_EMAIL` | Same file, via `get_decrypted_value()` (the platform's encrypted-secrets pipeline — see the platform overview doc §6) | Standard SMTP config. | | `EMAIL_FLOW_LIMITS` | Env var `EMAIL_FLOW_LIMITS` (JSON) → **`webapp/configs/email_flow_limits.json`** → hardcoded `FALLBACK_EMAIL_FLOW_LIMITS` | Per-flow rate limits (§2.3). The checked-in JSON file currently defines limits for `default`, `resend_verification`, `signup_request`, `password_reset`, `support_request`, `bug_report` — each with `per_email`/`per_ip`/(sometimes) `burst_ip` max+window pairs. One entry has a literal `_comment` key documenting an ordering constraint ("signup_request limit must be greater than resend verification"), read and ignored by the loader (harmless, just an inline note for humans, not a schema field). | | `LINK_EXPIRY_DAYS` | Referenced by `EmailEngine.inject_defaults` and (per the users app doc) `SignupTokenManagerService`/`PasswordResetTokenManagerService` | Shared expiry-day source for verification/reset/unsubscribe links — the same setting both `mailer` and `users` read from, keeping token lifetime and the email copy that describes it in sync. | | `REPORT_BUG_ENABLED`, `REPORT_BUG_ENABLED_ON_URLS`, `REPORT_BUG_MAX_ATTACHMENT_SIZE_MB`, `REPORT_BUG_ALLOWED_FILE_TYPES` | `paystream/app_settings/email.py` | Not mailer-owned logic directly, but colocated in the same settings file since it governs the `helpdesk` bug-report flow that `mailer.flows.bug` sends email for. | --- ## 6. Cross-App Integration Points | App | How it touches `mailer` | |---|---| | **`users`** | The heaviest integration. `CustomAccountAdapter.send_mail` is `mailer`'s Path A entry point (§2.1). `PasswordResetContextProvider`, `EmailContextProvider`, `SupportContextProvider` (all in `users/adapters/context/`) are injected directly into every `EmailEngine.send()` call. `PasswordResetService` and `resend_verification` views call `mailer.throttling.flow_throttle.allow_flow` directly. `SignupTokenManagerService`/`PasswordResetTokenManagerService` own the tokens that `mailer.links.*` turn into URLs. | | **`teamcentral`** | Every flow that needs "the member" (signup, verification tasks) queries `teamcentral.models.MemberProfile` directly — `mailer` reads this model but doesn't own or modify it. | | **`helpdesk`** | `support.py`/`bug.py` read `SupportTicket`/`BugReport` (with their attachments) directly — `mailer` is the exclusive place these two models' notification emails are built and sent from. | | **`utilities`** | `utilities.constants.template_registry.TemplateRegistry` is the canonical source of every template-prefix constant used throughout this app (`T.PASSWORD_RESET_EMAIL`, `T.EMAIL_SIGNUP_SUCCESS`, etc. — note this is a *different* module from the broken, unused `mailer.constants.context_requirements`, §7.1). `utilities.admin.url_utils.get_site_base_url`/`get_admin_url` build every absolute URL this app embeds in emails. `utilities.commons.helpers.get_site_name()` is used for template context. | | **`paystream`** | `PaystreamConfig.ready()` calls `mailer.engine.asset_builder.build_logo_assets()` at Django startup (guarded by try/except so a failure there only logs a warning, never blocks boot) — this is how the resized logo/favicon assets `InlineImageService.attach_logo` depends on actually get generated. `paystream.services.context_processors.site_context.build_subdomain_url` is used by the bug-report flow to link to the `issues` subdomain. `paystream/app_settings/email.py` owns all of this app's settings, including the `webapp/configs/email_flow_limits.json` load path. | | **`core`** | `core.request_context.get_client_ip()` is the IP source for both `PasswordResetService`'s and `flow_throttle`'s rate-limit checks. `core.models.{TimeStampedModel, AuditFieldsModel, ActiveManager}` are the base classes `EmailDelivery` builds on — and per the `audit` app doc, models built on those base classes are eligible for (though not automatically enrolled in) platform-wide audit tracking; `EmailDelivery` was not observed in `AUDIT_TRACKED_MODELS` in the earlier `audit` doc pass, meaning delivery records are logged via `mailer`'s own loggers but not the central `AuditEvent` trail. | | **`django-allauth`** (3rd-party) | `CustomAccountAdapter.send_mail` is allauth's own extension point — `mailer` doesn't call into allauth, allauth calls into `mailer` (via the adapter) for its own built-in flows (password reset key emails, email confirmation emails), which is why `TemplateResolver.PREFIX_ALIASES` exists — to translate allauth's own prefix vocabulary into DjangoPlay's. | | **Celery** | Six of the seven callables in `flows/` are real `@shared_task`s, auto-discovered via `mailer/tasks.py`. | --- ## 7. Quick Reference — Sending a New Kind of Transactional Email Based on the pattern every working flow in this app follows: 1. Add a template-prefix constant to `utilities.constants.template_registry.TemplateRegistry` (not `mailer/constants/` — that module's `templates` reference is broken, §7.1) and create `account/email/{prefix}_subject.txt`, `{prefix}.txt`, `{prefix}.html` (plus optional `account/fallback/{prefix}...` for subject/text). 2. Write a `@shared_task(bind=True, ...)` in `flows/` that: fetches whatever domain object it needs, creates an `EmailDelivery.objects.create(task_id=self.request.id, template_prefix=T.YOUR_PREFIX, to_email=..., metadata={...})` row *before* attempting to send, calls `send_email_via_adapter(template_prefix=..., to_email=..., context=..., user=...)`, and calls `delivery.mark_sent()` / `delivery.mark_failed(reason)` in a try/except around the send. 3. If the flow can be triggered more than once for the same underlying object (retries, admin re-triggers), add an idempotency check like `support.py`/`bug.py`'s "does a `SENT` `EmailDelivery` already exist for this metadata key" pattern before sending again. 4. If the flow should be rate-limited, call `mailer.throttling.flow_throttle.allow_flow(flow="your_flow_name", ...)` explicitly at the call site (not inside the task) — it is never applied automatically — and add a matching entry to `webapp/configs/email_flow_limits.json` if the defaults aren't appropriate. 5. Register the task in `mailer/tasks.py`'s import list so Celery autodiscovers it — and if it's meant to be called synchronously rather than via `.delay()`, don't name it `*_task` and don't put it in `tasks.py` (see §7.4 for what happens when that convention is broken). 6. Never build a `reverse()`-based URL for a token-bearing link in more than one place — route through a single `mailer/links/*.py` builder (or a context provider, for password reset) and make every caller pass in an id/token, not a pre-built URL string, to avoid the class of redundant-then-overridden URL construction described in §7.3.