Designing a provenance-aware employer data model
Most data models start with fields. This one started with a question:
What can this system honestly say about an employer shown on a LinkedIn or SEEK page?
The obvious answer is tempting: “This employer is accredited.”
It is also incomplete.
The official accredited-employer data describes a legal employer. A job-platform page describes a public profile, company, or advertiser. The two may refer to the same organisation, but they come from different sources and do not arrive with a shared key.
If a system puts both into one record and calls the result isAccredited, it loses the most important information: why the result exists and how confident the system should be.
This post explains the data model behind NZ Accredited Employer Inspector. The project helps job seekers check an employer on LinkedIn or SEEK NZ against Immigration New Zealand (INZ) accredited-employer data. Its main design rule is simple:
Keep each claim separate until the system needs to explain a result.
Start with two different claims
Every positive result contains at least two claims.
The first is an official accreditation claim. A legal employer, identified by an NZBN, appears in official data with an employer name, an optional trading name, and an accreditation expiry date.
The second is a platform identity claim. The LinkedIn company page, SEEK company profile, or SEEK advertiser being viewed belongs to that legal employer.
INZ is authoritative for the first claim. It is not authoritative for the second.
That sounds like a small distinction, but it changes the whole model. An official record must not be treated as proof of a page-to-employer match. A community mapping must not be treated as proof that accreditation is current. A name comparison must not quietly become a user confirmation.
The model keeps those facts apart, then returns them together with clear labels when the extension needs to show a result.
What “provenance-aware” means here
Provenance is a plain idea: for each piece of data, keep track of where it came from and what it means.
For this project, that means answering a few questions for every important value:
- Is it an official fact, a community statement, a system-derived result, or operational state?
- Which source supplied it?
- When did the service accept or observe it?
- Can it be stored as a fact, or must it be recalculated when the next request arrives?
This is not paperwork added after the database schema. It is part of the product behaviour. The UI, API, and Open Data exports all depend on these answers.
The canonical record: one legal employer per NZBN
The employers table is the canonical dataset. It has one record per NZBN, not one record per company page and not one record per search response.
It stores the latest accepted official observation for that legal employer. The core official fields are the employer name, trading name, NZBN, and accreditation expiry. These are INZ facts.
The table also stores service-owned fields, including lastVerifiedAt and lastVerifiedSource. They answer a different question: when and how did this service last accept the observation? lastVerifiedAt is not an INZ field, so it should never be presented as one.
Official observations arrive through two paths:
- a validated MBIE Official Information Act snapshot; and
- a live INZ result from a user-triggered extension check.
Both paths can update the canonical record, but they do not have equal timing. A newer live observation must not be overwritten by an older bulk snapshot. The importer stages and validates the complete snapshot before one canonical update, which means a partial input cannot partly replace the dataset.
The point is not to make the schema clever. It is to prevent an older source from silently making a record less current.
Community mappings are useful, but they are not official data
The product still needs to connect a public page to an NZBN. That mapping lives in its own association data.
A community association says: “An extension user selected this NZBN for this public platform identity.” It is a matching statement, not an accreditation statement.
The system stores the public platform identity and the selected NZBN separately from employers. It records one choice per extension installation, using a hashed random installation ID. The installation ID is not a LinkedIn or SEEK account ID. It is not authentication. It only lets the product recognise a user’s earlier choice and count distinct anonymous confirmations.
Keeping mappings separate gives the system useful options:
- a person’s previous choice can take precedence for their own browser;
- a unique community winner can help future users;
- competing choices can be shown as disputed instead of being hidden; and
- the official employer record stays unchanged, regardless of how many people select it.
This is a good example of why a relationship belongs in its own model. A platform page and an NZBN are different entities. Their connection has a source, a count, conflicts, and a history of user choices. Those properties do not belong on the employer record itself.
Exact-name matching is a derived result, not stored evidence
Sometimes no community mapping is needed. A platform display name may exactly equal one official employer name or trading name after narrow normalisation.
The normalisation is intentionally limited: Unicode NFKC, trimming, collapsing repeated whitespace, and lowercasing. It does not remove punctuation, company suffixes, or words. Similarity is not evidence.
The match is accepted only when it identifies exactly one NZBN. If the same name belongs to multiple employers, the system asks the user to choose. It does not guess.
Most importantly, an exact-name match is derived on demand. It is not saved as a community association.
That decision avoids a subtle data-quality problem. A persisted match can outlive the condition that made it valid. A company page can change its display name. A later official observation can change an employer or trading name. A second NZBN can later introduce the same name. Recomputing the rule on each resolution is simpler and more correct than trying to maintain a stored conclusion.
The system can therefore say “Automatic exact INZ name match” without implying that a person confirmed it or that INZ supplied the platform association.
Official data, community mappings, and derived exact-name matches meet only at resolution time. The derived match is never written back as a fact.
Resolution combines evidence; it does not rewrite it
When the extension asks the API to resolve a platform page, the Worker uses a strict order.
First, it checks for the current installation’s saved association. Next, it checks whether the community has one clear winner. If neither exists, it checks for a unique exact official or trading-name match. If there are explicit candidates but no safe selection, it asks the user to confirm one. Only after those paths fail does it reuse a fresh no-match observation or allow one live INZ lookup.
This order matters because it makes precedence explicit:
- a stored user or community association takes precedence over an automatic name match;
- a unique exact-name match is useful, but does not become a community confirmation;
- a positive official record takes precedence over a past no-match observation; and
- an ambiguous result remains ambiguous.
The resolution response includes the official employer record and the match method. The client can then show both dimensions: the current accreditation information and the reason the page was linked to that employer.
That is better than returning one opaque “match score”. A score hides the policy. A named match method lets users, support staff, and future maintainers understand the decision.
A no-match is an observation, not a verdict
Negative results are easy to model badly.
A search that returns no published INZ result does not prove that an employer is unaccredited. The page may use a brand name. The source may have changed. The query may be too narrow. The platform identity may be weak.
The service stores a no-match only for the exact platform identity and normalised display-name query that produced it, and only for a short configured period. It is a cacheable observation of a lookup, not a property of an employer.
The wording follows the model: No published INZ match. It does not say Not accredited.
This may look like a copywriting detail. It is actually a database decision made visible to the user. The model stores an observation with a scope and an expiry, so the interface should describe an observation with a scope and an expiry.
Refresh state is operational, not official provenance
The model also stores data that controls refreshes: when a refresh was attempted, its outcome, and when another attempt may start. This data prevents several users from sending duplicate live requests for the same NZBN.
It is useful operational state, but it is not proof that an official record was verified. A failed refresh or a cooldown must not change the meaning of lastVerifiedAt.
The same separation appears in the user experience. If a record needs a refresh but INZ returns no published result, the product retains the older dated record as context and says that live verification needs review. It does not quietly delete the record or claim fresh verification.
This is one reason I avoid an all-purpose status field. “Accredited”, “matched”, “recently checked”, “refresh blocked”, and “no result” describe different dimensions. Combining them into one enum would create invalid combinations and unclear UI states.
What I chose not to model
Some simpler models were deliberately rejected.
I did not use a single boolean such as isAccredited. It cannot say which legal employer is meant, where the value came from, or whether it is current.
I did not put a LinkedIn or SEEK identity directly on the employer record. One employer can have multiple public identities, and one identity can have competing community selections.
I did not persist fuzzy name matches. They would turn a guess into durable data, with no clean way to explain or invalidate it later.
I did not use a general search-response cache as the central model. The product is a shared employer dataset, not a collection of old API replies.
Each omission removes a source of accidental certainty. That is usually worth more than saving one table or one query.
The model is part of the interface
Good data modelling is often invisible when everything goes well. Here, it should be visible at the exact moments when certainty matters.
Users can see whether a result comes from a platform association or an exact-name match. They can see the official verification date and accreditation expiry. They can see when the system needs their confirmation. They are not asked to trust a black box that has flattened different kinds of evidence into one label.
The same model also makes the public surfaces safer. The Public API exposes a narrow, allowlisted view of official employer records. Dated Open Data snapshots expose a fixed public projection. Neither needs to expose platform identities, community confirmations, hashed installation IDs, refresh leases, or no-match observations.
The schema is not the whole product, but it defines what the product can say truthfully. That is why the main design work happened before the UI details: decide what each record means, decide what it does not mean, and keep those boundaries intact as the system grows.
Related resources
This project is independent open-source software. It is not an INZ or MBIE product and does not provide immigration or legal advice. For consequential decisions, verify the legal employer name and NZBN with the employer and consult the official INZ accredited-employer information.