--- since: 1.2.1 --- # DjangoPlay — `apidocs` App > Verbose name: **apidocs** · Registered as `"apidocs"` in `INSTALLED_APPS` ## 1. Summary `apidocs` (`AppConfig.name = "apidocs"`, no verbose name set, `ready()` is a no-op) is DjangoPlay's **API documentation and API-traffic-analytics** app, built on top of `drf-spectacular`. It has two genuinely distinct jobs that happen to live in the same app: 1. **Serving OpenAPI documentation** — permission-gated Swagger UI and ReDoc pages, plus the raw schema JSON endpoint, with several `drf-spectacular` hooks (`apidocs/hooks.py`) that clean up the generated schema (strip internal doc-view endpoints from the spec itself, dedupe security schemes, turn `operationId`s into readable summaries). 2. **Logging and reporting on API traffic** — a single model, `APIRequestLog`, populated by a platform-wide middleware, with an admin view and a `subdomain_stats` aggregation service that other apps (`users`' console stats dashboard, `paystream`'s issue-tracker stats) build reporting on top of. **Important correction to the stated premise**: `apidocs/signals.py` defines a `pre_save` receiver (`auto_set_is_public_api`) that looks like it's meant to auto-classify `APIRequestLog.is_public_api` by resolving the request path and inspecting the matched view's `permission_classes`. **This receiver is never connected** — `apidocs/apps.py`'s `ready()` method is a literal `pass`, there is no `apidocs/__init__.py` at all, and a repo-wide search found no other import of `apidocs.signals` anywhere in the codebase. Since Django only registers signal receivers when the module defining them is imported, **this receiver never fires**. It isn't needed anyway: the actual, working mechanism for setting `is_public_api` is a separate, considerably more sophisticated resolver living in `core.middleware.api_request_logging` (see §3), which computes the value inline at log-creation time using the *raw* (un-normalized) request path — something `signals.py`'s version, which only ever sees the already-normalized `path` field, could never do correctly (see §6 for why). `apidocs/signals.py` reads as an earlier, superseded implementation that was left in the codebase after `core`'s middleware-based approach replaced it. --- ## 2. Architecture ### 2.1 Two independent subsystems sharing one app ``` ┌───────────────────────────────────────────────┐ ┌────────────────────────────────────────────────────────┐ │ DOCUMENTATION │ │ TRAFFIC LOGGING / ANALYTICS │ │ │ │ │ │ apidocs/urls.py │ │ core.middleware.api_request_logging │ │ /schema/ → CustomSpectacularAPIView │ │ .APIRequestLoggingMiddleware │ │ /swagger/ → SwaggerUIView (bespoke JS UI) │ │ (registered platform-wide in │ │ /redoc/ → CustomSpectacularRedocView │ │ paystream.app_settings.middleware) │ │ │ │ │ │ │ drf-spectacular generates the schema from │ │ ▼ │ │ every DRF view in the project; apidocs.hooks │ │ apidocs.models.APIRequestLog.objects.create │ │ (PREPROCESSING/POSTPROCESSING_HOOKS in │ │ │ │ │ paystream.app_settings.drf_spectacular) │ │ ▼ │ │ post-processes the raw schema before it's │ │ apidocs.services.subdomain_stats │ │ served. │ │ .get_all_subdomain_stats() │ │ │ │ │ (consumed by users' console │ │ │ │ ▼ stats dashboard, paystream's │ │ │ │ apidocs.admin.APIRequestLogAdmin issue-tracker stats)│ └───────────────────────────────────────────────┘ └────────────────────────────────────────────────────────┘ ``` These two halves don't call into each other — the only thing they share is the app namespace and, indirectly, the fact that the logging middleware's `EXCLUDED_*` path lists specifically exclude `/schema/`, `/swagger/`, `/redoc/` from analytics so the docs UI itself doesn't pollute traffic stats. ### 2.2 Where `APIRequestLog` rows actually come from `APIRequestLoggingMiddleware` (`core/middleware/api_request_logging.py`) is a `MiddlewareMixin`-based, `process_response`-hooked middleware, registered near the end of `MIDDLEWARE` in `paystream/app_settings/middleware.py`. Its docstring is explicit about its failure contract: **"This middleware must NEVER block or fail a request. Authentication is best-effort only. DB errors are swallowed by design."** — the entire body is wrapped in a bare `try/except Exception: logger.error(...)`, so a broken row insert (or any bug in the normalization logic) degrades to a missing log entry, never a 500 for the actual request. Per request, it: 1. **Filters** which requests are worth logging at all (`_should_log`) — only `GET`/`POST`/`PUT`/`PATCH`/`DELETE`; on the root domain, only paths under `/api/`, `/console/`, `/admin/` (explicitly excluding a short list of noisy/session-check paths); on any subdomain, everything except static/media asset paths. 2. **Resolves the subdomain** (`_extract_subdomain`) by comparing the request's `Host` header against `settings.SITE_HOST` — returns `"root"` for the main domain, or the subdomain label (`issues`, `docs`, …) for anything ending in `.{SITE_HOST}`. 3. **Best-effort re-authenticates** the user via `JWTAuthentication` if `request.user` isn't already set/authenticated (covers API calls that never went through Django's session middleware). 4. **Normalizes the path** (`_normalize_path`) into a canonical, parameterized form for grouping in analytics — e.g. `/api/v1/crud/issues/252/` → `/api/v1/crud/issues/{issue_number}/`, `/admin/locations/customregion/187/change/` → `/admin/locations/customregion/{id}/change/`. This includes **subdomain- and business-logic-aware special-casing**: on the `issues` subdomain specifically, a `POST` to a bare issue-detail URL is disambiguated by inspecting `request.POST["action"]` (`change_status` → `.../status/`, `add_comment` (+ files) → `.../attachment/`, `add_comment` (no files) → `.../comment/`) so that different POST actions against the same URL show up as distinct rows in traffic stats rather than being conflated. 5. **Resolves `is_public_api`** (`_resolve_is_public`) using the **raw, un-normalized** path (explicitly commented in the source: `# ← raw_path, NOT canonical`) — admin paths are always private; on the root domain and for any `/api/` path on a subdomain, it resolves the DRF view and inspects permissions (`_resolve_is_public_drf`, see §2.3); for non-API subdomain UI paths, it checks the path against hardcoded per-subdomain public/private prefix allow-lists (`_SUBDOMAIN_UI_PUBLIC_PREFIXES` / `_SUBDOMAIN_UI_PRIVATE_PREFIXES`). 6. **Creates the `APIRequestLog` row** with both `path` (canonical) and `raw_path` (original, unmodified — kept specifically for "audit trails, operational debugging, CSV exports, and security investigations" per the model's own field docstring). ### 2.3 `_resolve_is_public_drf` — handles action-based permissions, not just static ones Most DRF views declare `permission_classes` as a static class attribute, which is easy to introspect. Some of DjangoPlay's viewsets (the comment specifically calls out `IssueCRUDViewSet`, from the `genericissuetracker`/`paystream.integrations.issuetracker` layer) instead override `get_permissions()` to vary permissions **per action** (list vs. create vs. destroy, etc.). To handle this correctly, `_resolve_is_public_drf`: 1. Resolves the URL via Django's `resolve()`, against the correct urlconf for the detected subdomain (`_SUBDOMAIN_URLCONFS` — currently mapping `issues` → `paystream.urlconf.subdomains.issues`, `docs` → `paystream.urlconf.subdomains.docs`, with a commented-out placeholder for a future `api` subdomain urlconf). 2. If the resolved view has a `.cls` (DRF class-based view), it **instantiates the view class directly**, synthetically sets `.action = "list"`, `.request = None`, `.format_kwarg = None`, and calls `get_permissions()` on that bare instance — a deliberately minimal fake request context, just enough to exercise permission-resolution logic that only branches on `self.action`. 3. Falls back to static `permission_classes` if the action-based call raises, and to `False` (private) on any other resolution failure. This is meaningfully more capable than the dead `apidocs/signals.py` approach, which only ever checked a static `permission_classes` list and would have raised/skipped entirely on action-based viewsets. --- ## 3. Data Model — `APIRequestLog` Plain `models.Model` (not `TimeStampedModel`/`AuditFieldsModel` — this app doesn't participate in the soft-delete/audit-fields convention the rest of the platform uses, and it is **not** registered in `audit.lifecycle.registry.AUDIT_TRACKED_MODELS` — this log is itself the audit trail for API traffic, not something the `audit` app additionally tracks). | Field | Notes | |---|---| | `user` | FK → `UserIdentity`, nullable, `SET_NULL`... actually `CASCADE` per the model (`on_delete=models.CASCADE`) — worth noting this is `CASCADE`, not `SET_NULL`, meaning **deleting a `UserIdentity` row hard-deletes all of that user's `APIRequestLog` history** rather than orphaning it. This is inconsistent with how the rest of the platform treats audit-adjacent data (the `audit.AuditEvent` model, by contrast, stores actor identity as a denormalized string precisely so it survives the actor being deleted — see the `audit` doc). | | `path` | Canonical/normalized path (see §2.2 step 4). | | `raw_path` | Unmodified original path — added later than the rest of the model (migration `0006`), specifically for audit/debugging/CSV/security use cases per its own help text. | | `method` | HTTP method. | | `response_status` | Response status code. | | `timestamp` | Defaults to `timezone.now`. | | `client_ip` | Nullable. | | `user_agent` | Nullable, truncated to 500 chars by the middleware before storage. | | `is_public_api` | Default `False`; see §2.2 step 5 for how it's actually set (middleware, not the dead signal). | | `subdomain` | Default `"root"`, indexed — "never `None`... so aggregation always works" per its own docstring. | Indexes: `timestamp`, `(user, timestamp)`, `(subdomain, timestamp)` — all clearly tuned for the two consumption patterns (admin changelist filtering by recency/user, and subdomain-partitioned aggregation). Default ordering `-timestamp`. --- ## 4. `hooks.py` — drf-spectacular schema post-processing Wired in via `SPECTACULAR_SETTINGS["PREPROCESSING_HOOKS"]`/`["POSTPROCESSING_HOOKS"]` in `paystream/app_settings/drf_spectacular.py`: - **`exclude_docs_views(endpoints)`** *(preprocessing)* — drops any endpoint whose view callback lives under the `apidocs.views` module path, so the documentation app's own views (swagger/redoc/schema) never appear as entries inside the schema they're serving. - **`remove_none_auth(result, **kwargs)`** *(postprocessing)* — walks every operation in the generated spec and strips out duplicate/invalid security scheme entries, keeping only a single valid `jwtAuth` entry per operation (or removing the `security` key entirely if nothing valid remains). Exists to clean up a known drf-spectacular quirk where scheme names can end up duplicated (the docstring literally gives `"jwtAuthjwtAuth"` as the kind of artifact this guards against). - **`beautify_operation_ids(result, **kwargs)`** *(postprocessing)* — converts auto-generated `operationId`s like `users_crud_departments_list` into human-readable `summary` text ("List Departments"), by taking the last token as the action (mapped via a small `list`/`retrieve`/`create`/`update`/`partial`/`destroy`/`delete` → verb table) and the second-to-last token as the resource name. Only applies when the developer hasn't already set an explicit `summary` on the view. - **`paystream.app_settings.drf_spectacular.inject_servers(result, ...)`** *(postprocessing, defined in `paystream`, not `apidocs`, but listed alongside the `apidocs` hooks)* — injects the `servers` block from `settings.OPENAPI_SERVERS` (or a single default built from `settings.SITE_URL`) at request time rather than import time, specifically so the correct environment is reflected without needing to reload settings. **Dead code note**: `paystream/app_settings/drf_spectacular.py` also defines a `customize_schema(openapi_info)` function (intended to strip the schema's `info` block down to just `title`/`description`) that is **not** listed in either hook list — it's fully implemented but never wired up or called anywhere. --- ## 5. Documentation Views — permission model All three routes require authentication, but with a **meaningful asymmetry** in how tightly they're gated: | Route | View | Auth requirement | |---|---|---| | `/schema/` | `CustomSpectacularAPIView` (subclasses `SpectacularAPIView` directly) | `authentication_classes = [SessionAuthentication]`, `permission_classes = [IsAuthenticated]` — **any authenticated user**, no additional permission check. | | `/swagger/` | `SwaggerUIView` (a bespoke plain Django `View`, **not** actually the drf-spectacular-backed `CustomSpectacularSwaggerView` — see below) | Authenticated **and** (`is_superuser` **or** `request.user.has_perm("policyengine.view_swagger")`). | | `/redoc/` | `CustomSpectacularRedocView` (subclasses `SpectacularRedocView`) | Authenticated **and** (`is_superuser` **or** `request.user.has_perm("policyengine.view_redoc")`). | **Practical consequence**: any authenticated user — regardless of whether they hold the `policyengine.view_swagger`/`policyengine.view_redoc` custom permission that gates the *rendered UI* pages — can fetch the complete raw OpenAPI schema JSON directly from `/schema/`. The two human-facing doc pages are the more tightly-gated surface; the machine-readable schema underneath them is not equally protected. Worth confirming with whoever owns API-access policy whether this is intentional (e.g. the schema itself isn't considered sensitive, only the convenience of browsing it) or an oversight where `/schema/`'s permission check should mirror the UI views'. **Dead code note**: `apidocs/urls.py` defines `CustomSpectacularSwaggerViewDecorated` and `CustomSpectacularRedocViewDecorated` (thin subclasses that just add `@extend_schema(exclude=True)`), presumably intended to keep the doc-view URLs out of the generated schema at the class level. **Neither is actually registered in `urlpatterns`** — the live routes use the plain `SwaggerUIView` (bespoke, unrelated class) for `/swagger/` and the undecorated `CustomSpectacularRedocView` for `/redoc/`. This makes the `@extend_schema(exclude=True)` decoration doubly redundant even where it would apply: `hooks.exclude_docs_views` (§4) already filters out anything under `apidocs.views` by module path regardless of per-class decoration. `SwaggerUIView` itself is a **second, independent Swagger UI implementation** — a plain Django `View` rendering a custom template (`TemplateRegistry.APIDOCS_SWAGGER`) with hand-duplicated versions of the same login/permission redirect logic that `CustomSpectacularSwaggerView` (the unused drf-spectacular-backed one) also implements. Whoever maintains this app should be aware there are two Swagger UI code paths and only one is actually live. --- ## 6. `subdomain_stats.py` and the console/reporting consumers `get_all_subdomain_stats(logs=None, use_raw_path=False, top_n=None)` groups `APIRequestLog` rows by `subdomain`, returning the top-N most-hit `(method, path)` pairs per subdomain (default top 5 for UI display, callers can request more — e.g. `users`' CSV export path uses `TOP_N_PER_SUBDOMAIN_CSV = 20`). Two behaviors worth understanding: - It always groups by the **canonical** `path`, never `raw_path`, specifically because canonical grouping is what distinguishes semantically-different actions that share a URL shape (the module's own comment: "canonical `path` ... is the ONLY field that distinguishes action-POSTs (`/comment/`, `/status/`, `/attachment/`) from plain detail GETs, since `raw_path` is the same bare URL for all" — a direct callback to the `issues`-subdomain POST-action disambiguation done in the logging middleware, §2.2 step 4). - `use_raw_path=True` (used for **personal**, not aggregate, views) post-processes the grouped canonical paths back into a realistic example URL by sampling the most common `raw_path` for that group and substituting real IDs into the `{placeholder}` tokens (`_inject_id`) — e.g. showing a user `/issues/254/comment/` instead of the more abstract `/issues/{issue_number}/comment/` when looking at their own activity. Both this module and `users/views/ui/stats.py` (the console dashboard's stats view, which directly imports from here) maintain their **own separate, near-identical `EXCLUDED_PATH_KEYWORDS`/exclusion lists** to keep internal/noisy paths (schema, swagger, redoc, session-check endpoints, the stats views themselves) out of reported traffic — these two lists are not shared from a single source of truth, so a new internal path added to exclude would need to be added in both places to stay consistent between the two call sites. --- ## 7. `querysanitizer.py` — a small, genuinely widely-used utility Unlike `signals.py` and `hooks.customize_schema`, `apidocs/utils/querysanitizer.py` is **actively used across the platform**: `SanitizeQueryParamsFilter` is a `logging.Filter` that regex-redacts `password=...`/`token=...` patterns out of log messages, and `add_sanitization_filter_to_logger(logger)` attaches it. A repo-wide search found this called at **module import time** (i.e. once, when the module is first loaded) in roughly 15 files spanning `industries`, `locations`, `users` (JWT auth views), `entities`, and `utilities` — each of those modules grabs its own `logging.getLogger(__name__)` and immediately calls `add_sanitization_filter_to_logger(logger)` on it. It's a small, defensive piece of shared infrastructure that happens to live in `apidocs` rather than `utilities`, worth knowing about if searching for "where does log redaction happen." --- ## 8. Admin `APIRequestLogAdmin` (`admin.py`) — a single admin class registered the plain `@admin.register(...)` way (not the `AdminIconDecorator.register_with_icon` pattern used by every other app documented so far in this series), still subclassing the shared `BaseAdminPage`. Notable: `list_editable = ('is_public_api',)` lets an admin **manually override** the auto-computed classification directly from the changelist — the comment even says `# ← Manual override`, i.e. this field is explicitly designed to be correctable by a human when the automatic (middleware-based) resolution gets it wrong. The `auto_detected()` display method is, however, a bit misleading: its docstring implies it shows "was `is_public_api` auto-set," but the actual implementation just checks `"Yes" if obj.pk else "—"` — i.e. it always shows `"Yes"` for any saved row and `"—"` only for an unsaved one, which in the context of a changelist (every row has a `pk`) means this column will read `"Yes"` for every single row, regardless of whether the value was ever manually overridden via `list_editable`. It does not actually track provenance (auto vs. manually edited) at all. --- ## 9. Integration Points (cross-app) | App | How it touches `apidocs` | |---|---| | **`core`** | Owns the actual mechanism that populates `APIRequestLog` — `core.middleware.api_request_logging.APIRequestLoggingMiddleware` — which `apidocs` itself has no code path to trigger. | | **`paystream`** | `paystream/app_settings/middleware.py` registers the logging middleware in `MIDDLEWARE`; `paystream/app_settings/drf_spectacular.py` wires `apidocs.hooks.*` into `SPECTACULAR_SETTINGS`, defines the schema's security scheme (`jwtAuth`) and the `inject_servers` postprocessing hook; `paystream.urlconf.subdomains.*` supplies the per-subdomain urlconfs the middleware resolves views against for `is_public_api` detection; `paystream.integrations.issuetracker.services.issues_stats` consumes `APIRequestLog`/`subdomain_stats` for issue-tracker-specific traffic reporting. | | **`users`** | `users/views/ui/stats.py` (the console dashboard's API-usage stats page) is the primary consumer of both `APIRequestLog` directly and `apidocs.services.subdomain_stats.get_all_subdomain_stats()`; also imports `apidocs.utils.querysanitizer` for log redaction on its JWT auth views. | | **`industries`, `locations`, `entities`, `utilities`** | Import `apidocs.utils.querysanitizer.add_sanitization_filter_to_logger` at module import time to redact `password`/`token` values from their own loggers — the one piece of this app that's genuinely widely depended upon. | | **`genericissuetracker` (third-party, via `paystream.integrations.issuetracker`)** | `IssueCRUDViewSet`'s action-based `get_permissions()` is the specific case the middleware's `_resolve_is_public_drf` action-instantiation logic was built to handle correctly. | --- ## 10. Quick Reference — API Surface - `GET /schema/` — raw OpenAPI schema JSON (`IsAuthenticated` only). - `GET /swagger/` — Swagger UI (bespoke JS-based, `SwaggerUIView`; authenticated + `policyengine.view_swagger` perm or superuser). - `GET /redoc/` — ReDoc UI (`CustomSpectacularRedocView`; authenticated + `policyengine.view_redoc` perm or superuser). - No CRUD/read/list endpoints of its own for `APIRequestLog` — it's consumed only via the Django admin and the `subdomain_stats` service, not exposed as a REST resource.