--- since: 1.2.1 --- # `locations` — Geographic Hierarchy Engine > Registered as `"locations"` in `INSTALLED_APPS` · `verbose_name = "Geolocations"` ## 1. Summary `locations` is the platform's **canonical geographic reference data** app — a GeoNames-inspired hierarchy of continents → countries → states/provinces → districts → cities → granular addresses, plus a separate IANA timezone catalogue. Every other domain app that needs "where" (e.g. `teamcentral` addresses, `entities` registered offices, `industries` regional coverage) is expected to point at `locations.Location`/`locations.CustomCity` rather than storing free-text geography. It is one of the most heavily engineered apps in the platform for **data-quality plumbing**: normalization, slugging, soft-delete, full-text/trigram search, per-model change history, Redis-backed lookup caching, and cascading Select2 autocomplete widgets — but comparatively thin on business logic (it's a reference-data app, not a workflow engine). --- ## 2. Architecture ### 2.1 Module layout ``` locations/ ├── admin/ # 8 ModelAdmins, one per model, all via BaseAdminPage (utilities app) ├── forms/ # 8 BaseAdminForm subclasses, Select2 cascading widgets ├── migrations/ # 6 migrations — schema evolved incrementally (see §2.3) ├── models/ # 8 models — the hierarchy (§3) ├── serializers/ │ ├── base/ # BaseXSerializer — canonical field set per model │ └── v1/ │ ├── read/ # adds created_at/updated_at to base fields │ └── write/ # narrower, write-only field subset ├── views/api/v1/ │ ├── crud/ # DRF ModelViewSets (full CRUD) — router-based │ ├── read/ │ │ ├── list/ # plain ListAPIView per model (lighter than CRUD list) │ │ ├── detail/ # plain RetrieveAPIView per model │ │ └── history/ # simple_history-backed change-log endpoints │ ├── ui/ │ │ └── autocomplete.py # django-autocomplete-light (dal) Select2 sources, cascading │ └── ops/ │ ├── bulk_update.py # PATCH-many endpoints, field-allowlisted │ └── export.py # CSV export (cities only) ├── utils/global_region.py ├── exceptions.py # InvalidLocationData (ValidationError subclass, closed error-code list) └── urls.py → views/api/v1/__init__.py (mounts crud/, read/, ui/, ops/) ``` ### 2.2 The hierarchy ``` GlobalRegion ("Continent") └─ CustomCountry ──M2M──> GlobalRegion (a country can belong to multiple regions) └─ CustomRegion ("State/Province") ─FK→ CustomCountry └─ CustomSubRegion ("Administrative Division", e.g. district) ─FK→ CustomRegion └─ CustomCity ─FK→ CustomSubRegion └─ Location ("Township / Area") ─FK→ CustomCity AdministrativeCategory ─FK← CustomCountry.administrative_category (federal/unitary/autonomous/etc., PROTECT) Timezone ─FK← CustomCity.timezone (SET_NULL) · Timezone.country_code validated against CustomCountry.country_code ``` Every level below `Location` mirrors GeoNames' `admin1`/`admin2`/`admin3`/`admin4` code levels (region=admin1, subregion=admin2, city=admin3, location=admin4) — the model docstrings/help_text explicitly reference this (e.g. Kerala=admin1 `13`, Kolhapur=admin2 `594`, Hatkanangale=admin3 `4285`). ### 2.3 Migration history (schema evolution) `0001_initial` → `0002_full_current_schema` (major re-shape) → `0003` (name/code field alterations on country/region) → `0004` (HistoricalLocation options) → `0005` (region options) → `0006_administrativecategory_and_more` (added the `AdministrativeCategory` model + FK on country). This is a genuinely iterated schema, not a single generated migration — worth knowing before writing new migrations that assume a clean initial state. --- ## 3. Data Models All 8 models inherit **`TimeStampedModel`** (soft-delete: `is_active`, `deleted_at`, `soft_delete()`/`restore()` — see the `core` app doc) **+ `AuditFieldsModel`** (`created_by`/`updated_by`/`deleted_by` FKs to `AUTH_USER_MODEL`, `SET_NULL`, related_name templated as `%(app_label)s_%(class)s_created_by` etc. for uniqueness). All 8 also carry `django-simple-history`'s `HistoricalRecords()` — a **separate, model-level** change log (its own `Historical` shadow table) distinct from the platform's centralized `audit.AuditEvent` trail used by other apps. **None of the 8 models are registered in `AUDIT_TRACKED_MODELS`** (`apps.py.ready()` is a no-op `pass`) — so `locations` changes are visible only via the `/read/history/` endpoints (simple_history), not via the cross-app audit log. | Model | Verbose name | Key fields | Uniqueness (scoped to active rows) | |---|---|---|---| | `GlobalRegion` | Continent | `code`, `name`, `asciiname`, `slug`, `geoname_id`, `location_source` | `name` unique where `deleted_at IS NULL` | | `CustomCountry` | Country | `geoname_id`, `name`, `country_code` (ISO-3166 alpha-2), `administrative_category` (FK, PROTECT), `currency_code/symbol/name`, `country_phone_code`, `phone_number_length`, `has_postal_code`, `postal_code_length`, `postal_code_regex`, `country_languages`, `population`, `global_regions` (M2M) | `name` unique where active | | `CustomRegion` | State/Province | `name`, `code` (admin1), `country` (FK, CASCADE) | `(name, country)` unique where active | | `CustomSubRegion` | Administrative Division | `name`, `code` (admin2), `region` (FK, CASCADE) | `(name, region)` unique where active | | `CustomCity` | City | `name`, `code` (admin3), `subregion` (FK, CASCADE), `latitude`/`longitude` (checked -90..90 / -180..180), `timezone` (FK→`Timezone.timezone_id`, SET_NULL) | `(name, subregion)` unique where active | | `Location` | Township / Area | `city` (FK, CASCADE), `code` (admin4), `postal_code`, `street_address`, `latitude`/`longitude` (checked) | `(city, postal_code, street_address)` unique where active | | `Timezone` | Timezone | `timezone_id` (PK-like unique, IANA format regex-validated), `gmt_offset_jan`/`dst_offset_jul`/`raw_offset` (decimal, validated -12.0..14.0 in 0.25 steps), `display_name`, `country_code` (validated against `CustomCountry.country_code`) | `timezone_id` unique | | `AdministrativeCategory` | Administrative Category | `code` (choices: federal/unitary/quasi_federal/confederation/autonomous/special_admin/dependency/disputed/unknown), `name`, `slug`, `description` | `code` and `name` both globally unique | **Common model behaviors** (repeated per-model, not shared via a mixin — each model hand-rolls its own `save()`/`soft_delete()`/`restore()`): - `clean()` — required-field checks, ASCII/length validation on admin codes, and `normalize_text()` applied to `name`/`asciiname`/`location_source` (and `description` for `AdministrativeCategory`) - `save(*, user=None, skip_validation=False)` — calls `clean()`/`full_clean()` unless skipped, sets `created_by` (only if `pk` is new) and always `updated_by`, auto-slugs via `slugify(name)` if `slug` is blank, wraps everything in `transaction.atomic()`, and logs at debug/info/error levels around the DB write - `soft_delete(user=None)` / `restore(user=None)` — guarded (raises `ValidationError` if already in the target state), sets/clears `deleted_by`/`deleted_at`/`is_active` **Notable inconsistencies worth flagging** (harmless today, but real): - `CustomRegion.save()` has an extra step not present elsewhere: on first creation it manually calls `self.history.create(history_type='+', ...)` right after `super().save()` — a workaround, per the inline comment, for backfilling history on old records; every other model relies on `simple_history`'s automatic post-save signal alone. - `CustomSubRegion.py` contains a large commented-out duplicate of its own `save()` method left in place above the active one — dead code, not a functional issue, but worth cleaning up if this file is touched again. - `Timezone` diverges from the rest of the app: it doesn't use `transaction.atomic()` decorators on `save`, sets `created_by_id`/`updated_by_id` directly instead of the FK object, and its `soft_delete`/`restore` are idempotent no-ops (log + `return self`) rather than raising, unlike every sibling model. - `AdministrativeCategory` has no `soft_delete()`/`restore()` override at all — it inherits the bare `TimeStampedModel` versions and is linked from `CustomCountry` with `on_delete=PROTECT`, so it can never be hard-deleted while any country references it. ### 3.1 `Location.add_or_get_location(...)` — known dead/broken code `Location` defines an `add_or_get_location(cls, city_name, region_name, country_name, ...)` helper intended as a Redis-cached "find-or-create the whole hierarchy from raw strings in one call" utility (get-or-create cascading through country → region → subregion → city → location, with each level cached in Redis as zlib-compressed JSON, `LOCATION_CACHE_TIMEOUT` default 3600s from `app_settings/cache.py`). **It is not usable as written**: the first parameter is named `cls` but the method is never decorated `@classmethod`, and it references `city` and `street_address` locals inside `location_cache_key = f"location:{city}:{postal_code}:{street_address or 'none'}"` *before either variable is assigned* in the function body (both are only bound later, and `street_address` is hardcoded to `None` right before the final `get_or_create` regardless). Calling this method today would raise `UnboundLocalError`. It doesn't appear to be called anywhere else in the codebase (not from views, admin, or management commands) — treat it as an aspirational/in-progress utility rather than working functionality if you plan to build on top of it. ### 3.2 Postal code validation (`utilities/utils/locations/postal_code_validations.py`) `Location.clean()` calls `validate_postal_code(postal_code, country_code)`, which looks up `CustomCountry.postal_code_regex` for the given `country_code` (raising `PostalCodeValidationError` if the country doesn't exist, returning `None`/skipping if `has_postal_code=False`) and validates the postal code string against that per-country regex. This means postal code validation is **entirely data-driven** — to support a new country's postal format, populate `CustomCountry.postal_code_regex`, not application code. --- ## 4. HTTP API Mounted at `path("api/v1/locations/", include("locations.urls"))` in `paystream/urlconf/base.py`; `locations.urls` delegates everything to `locations/views/api/v1/__init__.py`, which fans out into four sub-namespaces: ### 4.1 CRUD (`/api/v1/locations/crud/…`) — full DRF `ModelViewSet`s via `DefaultRouter` | Router prefix | ViewSet | basename | |---|---|---| | `countries/` | `CustomCountryViewSet` | `country` | | `regions/` | `CustomRegionViewSet` | `region` | | `subregions/` | `CustomSubRegionViewSet` | `subregion` | | `cities/` | `CustomCityViewSet` | `city` | | `locations/` | `LocationViewSet` | `location` | | `timezones/` | `TimezoneViewSet` | `timezone` | | `global-regions/` | `GlobalRegionViewSet` | `global-region` | | `administrative-categories/` | `AdministrativeCategoryViewSet` | `administrative-category` | Each ViewSet extends the platform-shared **`BaseViewSet`** (from `utilities.api.generic_viewsets` — full detail belongs in the `utilities` app doc; summarized here since it governs all behavior): - **Auth:** `JWTAuthentication` + `IsAuthenticated` - **Permissions:** *dynamic Django model permissions* — `get_permissions()` maps the DRF action to a Django permission string (`list`/`retrieve`→`view`, `create`→`add`, `update`/`partial_update`→`change`, `destroy`→`delete`) and checks `request.user.has_perm(f"{app_label}.{action}_{model_name}")`, e.g. `locations.view_customcountry`. This is standard Django's built-in permission system, not `policyengine` (see that app's doc for where `policyengine` *is* used elsewhere). - **Throttling:** `CustomThrottle` (rate defined in `utilities.api.rate_limits`) - **Filtering:** `DjangoFilterBackend` + `OrderingFilter` + `SearchFilter`, with per-model `filterset_fields`/`ordering_fields`/`search_fields` (e.g. country: filter by `name`/`country_code`/`currency_code`/`country_languages`; search across `name`/`asciiname`/`alternatenames`/`country_code`) - **Trigram search override:** each `list()` additionally layers a manual `Q(field__trigram_similar=search)` filter reading `?search=` directly (on top of, not instead of, the `SearchFilter` backend) — this is why country/city/etc. search feels fuzzy/typo-tolerant rather than exact-substring - **Caching:** list/retrieve responses cached (`cache_timeout = 172800`s / 48h) under a hash of `{model}_{list|detail}_{user_id}_{page}_{sha256(query_params)}`, invalidated on create/update/delete - **Errors:** `error_class = InvalidLocationData` — so 400 responses from this app carry the `{"error", "code", "details"}` shape defined in `locations/exceptions.py`, not DRF's default error format, for create/update/soft-delete failures - **Serializers:** `read_serializer_class` for list/retrieve, `write_serializer_class` for create/update/partial_update (resolved via `SerializerByActionMixin`) - **Delete is soft:** `destroy` ultimately calls `instance.soft_delete(user=request.user)` (via `SoftDeleteMixin`), returning `204` — rows are never hard-deleted through the API Serializers follow a strict **base → read/write** layering per model (e.g. `BaseCountrySerializer` defines the canonical field tuple; `CountryReadSerializerV1` adds `created_at`/`updated_at`; `CountryWriteSerializerV1` narrows to only the writable subset, e.g. excludes `id`, `administrative_category`, `global_regions` for country writes). This versioned (`V1`) base/read/write split is the pattern to follow if/when a `v2` serializer set is ever needed. ### 4.2 Read-only (`/api/v1/locations/read/…`) — lighter, cacheable, GET-only - **`list/{model}/`** — `BaseListAPIView` (from `utilities.api.generic_views`): `IsAuthenticated`, cached similarly to CRUD list (`cache_timeout=172800`), but no write path at all — a deliberately smaller surface for read-heavy consumers (e.g. dropdown population) than the full CRUD list. - **`detail/{model}//`** — plain `RetrieveAPIView`, same read serializer. - **`history/{model}/`** — `BaseHistoryListAPIView`, backed by each model's `simple_history` manager (`queryset = Model.history.all()` or model-specific `history_queryset`), filtering/ordering disabled by design (`filter_backends = []`) — returns the full historical record set for the model using the same read serializer (so historical rows are shown with current-schema shape, not a diff format). **Only 6 of the 8 models have a history endpoint** — `Location` and `AdministrativeCategory` are absent from `read/history/__init__.py` even though both have `HistoricalRecords()` on the model; their history is tracked in the DB but not exposed via this API. ### 4.3 UI helpers (`/api/v1/locations/ui/…`) — Select2 cascading autocomplete Built on `django-autocomplete-light` (`dal`), all requiring `IsAuthenticated` + `CustomSearchThrottle`, all filtering to `deleted_at__isnull=True, is_active=True` and using trigram (`__trigram_similar`) matching on `?q=`: | Endpoint | Model | Cascades on (`forwarded`) | |---|---|---| | `global-regions/` | `GlobalRegion` | — | | `countries/` | `CustomCountry` | — | | `regions/` | `CustomRegion` | `country` | | `subregions/` | `CustomSubRegion` | `region` | | `cities/` | `CustomCity` | `subregion` | | `timezones/` | `Timezone` | — | The `forwarded` mechanism is how the admin/forms UI implements dependent dropdowns (pick a country → region options narrow to that country, etc.) — see `LocationForm.SELECT2_CONFIG` in `forms/location.py` for the frontend wiring (`dependent_on` chains mirror this). ### 4.4 Ops (`/api/v1/locations/ops/…`) — bulk PATCH + CSV export **Bulk update** — `PATCH`, body `{"updates": [{"id": ..., "field": value, ...}, ...]}`, one `BaseBulkUpdateAPIView` subclass per model with an explicit `allowed_fields` allowlist (fields outside the allowlist are silently rejected per-row into an `errors[]` array, not a hard 400). Every successful field update goes through `obj.save()` then `simple_history.utils.update_change_reason(obj, self.change_reason)` — so bulk edits are recorded with a human-readable reason (e.g. `"Bulk update of countries"`) distinguishing them from normal single-record saves in the history log. | Endpoint | `allowed_fields` | |---|---| | `bulk/global-regions/` | `{"name"}` | | `bulk/countries/` | `{"name", "iso2", "phone_code"}` ⚠️ see note below | | `bulk/regions/` | `{"name", "country"}` | | `bulk/subregions/` | `{"name", "region"}` | | `bulk/cities/` | `{"name", "region", "subregion"}` | | `bulk/timezones/` | `{"display_name"}` | ⚠️ **Bug:** `CustomCountryBulkUpdateAPIView.allowed_fields` lists `iso2` and `phone_code`, but `CustomCountry`'s actual model fields are named `country_code` and `country_phone_code`. As written, any bulk-update request attempting to touch a country's code or phone code will hit `setattr(obj, "iso2", value)`, which silently creates a throwaway Python attribute on the in-memory instance rather than updating a real model field — it will **not** persist, and won't error either (no `errors[]` entry, since the field *is* in the allowlist). Only `name` bulk-updates actually work for countries today. **Export** — `GET /ops/export/cities/` streams a CSV (`Content-Disposition: attachment`) of every city (`id, name, region.name, country.name` via `select_related("subregion__region__country")`) — the only export endpoint in the app, and it is explicitly `exclude=True` in its `@extend_schema`, so it's intentionally hidden from Swagger/ReDoc (see §6). --- ## 5. Admin & Forms All 8 models get a `BaseAdminPage` subclass (shared infra from `utilities.admin`, documented fully in the `utilities` app doc) registered via `@AdminIconDecorator.register_with_icon(Model)`. Notable app-specific behavior in `LocationAdmin`: - **Large-table protection:** `get_queryset()` short-circuits to `qs.none()` on the unfiltered changelist view once row count exceeds `auto_load_threshold` (10,000) — prevents the admin from ever trying to render/paginate an enormous unfiltered `Location` table by accident; filtering by country/region/subregion/city (each wired as a cascading AJAX filter via `filter_config`) is the intended way to browse. - **Read-only-friendly:** `allow_view_only_add`/`allow_view_only_change = True` — view-only admin users can still open (but presumably not persist changes to) add/change forms, a `BaseAdminPage` feature. - **Custom changelist template** (`admin/custom/locations/changelist.html`) and a structured `form_layouts` config (grouped fieldsets: Hierarchy → Address → Geo → System) rather than Django admin's flat `fieldsets`. - Computed list-display columns (`get_country`, `get_region`, `get_subregion`, `get_township`) walk the FK chain defensively (`obj.city and obj.city.subregion and ...`) to avoid `AttributeError` on partially-populated hierarchies. `LocationForm` (and its siblings for the other 7 models) extend `BaseAdminForm` and declare a `SELECT2_CONFIG` dict per field describing AJAX-backed, app/model-scoped, optionally `dependent_on`-chained Select2 widgets — this is the form-level counterpart to the `ui/autocomplete.py` endpoints in §4.3. --- ## 6. API Documentation Visibility Every CRUD/read/UI view in this app carries `@extend_schema(tags=[...])` (`drf_spectacular`), so all of them appear in the platform's Swagger/ReDoc output (see the `apidocs` app doc for how tags/schema generation work globally) — **except** `CityExportAPIView`, which explicitly sets `exclude=True`, deliberately keeping the CSV export endpoint out of the public API docs (it's an operational/admin tool, not a documented public contract). --- ## 7. Security Notes - Every endpoint across all four sub-namespaces requires authentication (`IsAuthenticated`); there is no anonymous read access to geography data via the API (autocomplete included). - CRUD write actions are additionally gated by real Django model permissions (`add_*`/`change_*`/`delete_*`/`view_*`), so authentication alone isn't sufficient for mutation — a user needs the specific permission assigned (typically via Django groups) to create/edit/delete any location-hierarchy record. - Delete is always soft (`is_active=False`, `deleted_at` set) at the API layer; there is no hard-delete endpoint exposed anywhere in this app. - `AdministrativeCategory` uses `on_delete=PROTECT` from `CustomCountry` — the DB itself refuses to remove an administrative category while any country still references it, regardless of API-level soft-delete semantics. - Bulk-update's per-row `allowed_fields` allowlist is the main injection guard on that endpoint (arbitrary field names can't be set) — but as noted in §4.4, the country allowlist references non-existent field names, which is a data-integrity footgun (silent no-op) rather than a security hole. --- ## 8. Integration Points - **`core.models.TimeStampedModel` / `AuditFieldsModel`** — soft-delete + created/updated/deleted-by plumbing (see `core` app doc) - **`django-simple-history`** — per-model shadow history tables, exposed via `/read/history/` for 6 of 8 models - **`django-autocomplete-light` (`dal`)** — cascading Select2 sources for both the admin (`forms/`) and any frontend consumer of `/ui/autocomplete/*` - **Redis** (`core.utils.redis_client`) — used (in principle) by `Location.add_or_get_location()` for hierarchy lookup caching; see §3.1 for why this path is currently non-functional - **`utilities` app** — supplies `BaseViewSet`, `BaseListAPIView`/`BaseDetailAPIView`/`BaseHistoryListAPIView`, `BaseBulkUpdateAPIView`, `StandardResultsSetPagination`, `CustomThrottle`/`CustomSearchThrottle`, `BaseAdminPage`, `BaseAdminForm`, `normalize_text()`, and `validate_postal_code()` — `locations` is effectively a showcase consumer of nearly every shared building block `utilities` provides (full detail deferred to that app's doc) - **Downstream consumers** — any app storing a "where" (e.g. `teamcentral` member/employee addresses, `entities` registered offices) is expected to FK into `locations.Location`/`CustomCity` rather than duplicating geography — worth confirming this pattern holds when we document those apps. ---