Protect the data lifecycle

Protection need

Select one data class with a real owner and purpose. Example: route access events contain subject, device, route, decision, reason, source, time, and correlation data. Security operations need enough detail to detect and investigate abuse. Users and service owners need protection from unnecessary collection, broad internal access, indefinite retention, and accidental credential logging.

State the security and privacy properties. The records need integrity, availability for a bounded investigation period, restricted disclosure, accountability, and predictable deletion. They do not need full request bodies, bearer tokens, session cookies, or every identity-provider claim.

Map data actions:

1. Collection and generation.
2. Validation and classification.
3. Use in access, detection, support, analytics, and investigation.
4. Disclosure and sharing.
5. Transformation and derivation.
6. Storage, indexing, caching, and replication.
7. Backup and recovery.
8. Retention, deletion, and media sanitization.

Include metadata and derived values. A route name, time series, device signal, group, or correlation graph can reveal sensitive behavior even when request content is absent.

Security objectives and requirements

For every field and derived value, record:

• Specific purpose and owner.
• Source and data subjects.
• Required accuracy and freshness.
• Classification and potential harm.
• Allowed services, roles, users, and operations.
• Required precision and whether a coarser value works.
• Destinations and external sharing.
• Retention and deletion trigger.
• Backup, restore, and recovery need.
• Evidence and review trigger.

Minimize before collection. Use a correlation identifier instead of a credential. Use a policy reason code instead of the complete policy input where investigation does not require it. Redact or omit sensitive headers and query values. Separate security evidence from product analytics and support access.

Apply object, tenant, purpose, and role authorization to data access. Protect exports as new copies with their own owner, expiry, and evidence. Encrypt suitable network and storage boundaries. Keep keys and authorized runtimes out of the same attacker boundary where the threat model requires separation.

Define retention from operational and recovery need. Use shorter online retention and controlled archival where suitable. Apply automatic expiry to primary data and downstream systems. A legal or incident hold, when required by the owner, must be a specific state with approval, scope, expiry review, and access control rather than a silent permanent exception.

Security invariants and evidence

Example invariants:

• No credential, full assertion, secret, or unnecessary request body enters the event pipeline.
• Only named security and service-owner roles can query identifiable events, and every access is recorded.
• Analytics receives only the fields and precision required for its separate purpose.
• Every export has an owner, destination, classification, expiry, and access record.
• Restoring a backup reapplies current deletion and access policy before use.
• Data older than the retention bound is unavailable from primary, index, cache, replica, and ordinary restore paths.

Evidence includes schema review, field-level tests, access-policy tests, sample pipeline inspection, export inventory, retention jobs, backup restore exercises, deletion queries, key policy, and media sanitization records. Test with canary records whose expected lifecycle is known.

Failure cases

Test the complete lifecycle:

• A library logs a request or authorization header during an error.
• A new identity claim appears and is copied automatically into every event.
• An analyst role can query another tenant or export all data.
• A dashboard cache or search index outlives the primary record.
• A queue dead-letter store has no retention.
• An incident hold never expires.
• A backup restores deleted data into a searchable environment.
• A support bundle includes raw events or keys.
• A retired storage device is reused without approved sanitization.
• Deleting an encryption key affects unrelated records under the same key.

Use final-system observation. A successful primary deletion API does not prove that the search, warehouse, export, backup, and support copies are gone.

Design tradeoffs and residual risk

Detailed evidence improves detection and investigation and increases confidentiality and privacy risk. Short retention reduces breach impact and can limit long investigations. Strong access controls reduce use and do not remove risk from administrators or compromised services. Aggregation can reduce identifiability and can still expose small groups or rare behavior.

Record unavoidable copies, delayed backup expiry, external processor boundaries, and exceptional holds as explicit residual risks. Assign owners and test their controls.

Pomerium boundary

Pomerium can produce access and authorization evidence and can control access to protected data tools. Operators choose log fields, export destinations, storage, query permissions, retention, backup, and deletion. Application events and business records remain application-owned.

Pomerium cannot remove credentials that another proxy or application logs, delete copies in an external analytics system, or apply retention to a support export. The data map must cross the product boundary.

Exercise

Choose one access event, identity profile, support record, or application object. Build a lifecycle table with one row per field, derived value, and copy. Include purpose, owner, source, subject, class, precision, users, operations, systems, sharing, retention, backup, deletion, and evidence.

Remove at least one unnecessary field or copy. Run tests for an unauthorized query, export expiry, primary deletion, index deletion, and isolated backup restore. Inspect one error and one support bundle for sensitive leakage.

Evaluation checklist

• Does every field, derived value, and copy have a current purpose and owner?
• Can the system collect less data, lower precision, or a short-lived value?
• Are security, product, support, analytics, and external sharing purposes separated?
• Do authorization and evidence cover query, export, administration, and recovery?
• Do expiry and deletion propagate to queues, caches, indexes, replicas, backups, and restored environments?
• Are holds, exceptions, and external processors scoped, approved, monitored, and reviewed?
• Can the team prove the final lifecycle with canary data and restore tests?