The metric problem
Six months after publishing an architecture portal, someone will ask whether it is being used. The available answer is usually a page-view count, and page views are close to meaningless here.
A reader who opens the portal, finds their answer in thirty seconds and leaves has had a perfect experience and generates one view and a catastrophic bounce rate. A reader who cannot find anything and clicks through nine pages in frustration generates excellent numbers. The standard web metrics are inverted for a reference resource.
Four signals that actually mean something
1. Links to the portal in other documents
The strongest signal. When a business case, a decision record, a change request or an audit response links to a portal page instead of embedding a screenshot, the portal has become the reference. That is the whole goal, and it is countable — search your document estate for the portal's URL.
2. Questions that stop arriving
Before publishing, note the three questions that reach the architecture team most often. If those questions genuinely stop, the portal is working. If they keep arriving unchanged, either it does not contain the answer or nobody can find it — and the fix differs depending on which.
3. Corrections reported by readers
Counter-intuitively, a portal that generates correction reports is a portal being read carefully by people who trust it enough to bother. Silence usually means indifference rather than perfection.
4. Requests for a new perspective or catalogue
Someone asking for a view that does not exist has understood what the portal is for and hit its edge. That is engagement, and it tells you exactly what to build next.
The one quantitative measure worth having
If you want a number, measure coverage of the questions you were asked. Keep a list of the questions that reach the architecture team. Each month, mark which of them a reader could have answered themselves from the portal.
That percentage is the only figure that maps to the portal's actual purpose, and it goes up as a result of decisions you control — adding a catalogue, fixing metadata, improving search.
What to do about a portal nobody reads
Three causes, in order of frequency:
- Nobody knows it exists. The most common by a wide margin. A portal announced once in an email is invisible. It needs to be linked from wherever people already go — the intranet architecture page, the onboarding pack, the change process.
- It answers the wrong questions. Published because the repository existed, rather than because an audience had a need. Go back to the question list.
- It is not trusted. Usually because nobody can tell how current it is. A visible publication date and a stated cadence fixes more adoption problems than any feature.
A test worth running
Ask three people outside the architecture team to answer one real question using the portal, and watch without helping. Ten minutes each.
You will learn more from those thirty minutes than from any dashboard. The failures are almost never the ones you expected, and they are almost always in navigation and vocabulary rather than in content — people search for the word they use, not the word you modelled.
Instrumenting without surveillance
A static portal on an internal server produces web server logs, and it is tempting to mine them. Two cautions before you do.
The first is practical: internal traffic is dominated by a small number of heavy users and by monitoring probes, so aggregate figures mislead. The second is that you are now processing data about identifiable colleagues' reading habits, which has obligations attached and is worth being deliberate about.
If you do look at logs, look at which pages rather than which people. The distribution of page popularity tells you what to invest in; the identity of readers tells you very little you can act on.
The six-month review
A short, repeatable exercise that keeps a portal useful:
- Read the question log. Which questions can now be self-served?
- List the five most-visited pages. Are they the ones you expected?
- List pages visited never. Should they exist?
- Ask three readers to complete one task each, and watch.
- Pick one thing to add and one thing to remove.
The removal step is the one people skip and the one that keeps a portal navigable. Content that nobody reads is not free — it dilutes search results and lengthens every list a reader scans.
The first month tells you less than you think
Launch traffic is curiosity. An announcement email drives a spike, everyone opens the portal once, and the numbers look excellent for about eleven days. Reading anything into that period is the most common mistake in portal measurement.
The number that matters is what the traffic settles at in month three, and specifically what proportion of it is people returning. A portal with fifty visits a month from forty different people is a curiosity. A portal with fifty visits a month from twelve people is a working reference for twelve people, which is a genuine result.
Instrumenting a static portal
A generated static portal has no backend, which rules out the server-side analytics most organisations already run. Three options, in descending order of how much you will learn and ascending order of how much you will argue about privacy.
| Approach | What you learn | What it costs |
|---|---|---|
| Web server access logs | Which pages, how often, from which internal subnets. No user identity unless the server authenticates. | Nothing. The logs already exist; someone has to parse them. |
| A privacy-preserving analytics script | Page views, referrers, the search terms typed into the portal's own search box — which is the single most useful signal here. | A script tag and a conversation with whoever owns the cookie policy. |
| Authenticated portal behind SSO | Which roles read which perspectives. This answers whether the scoping was right. | Real infrastructure, and it changes the portal from a static artefact into a service. |
Most estates should start with the first and move to the second when someone asks a question the logs cannot answer.
What the search box tells you
If you instrument one thing, instrument the portal's own search. The queries people type are a direct statement of what they came for, and the ones that return nothing are a direct statement of what your architecture does not describe.
Three patterns show up quickly in the failed queries:
- Vocabulary mismatch. People search for the name the business uses and the model holds the name IT uses. This is fixable in an afternoon by adding aliases, and it is worth more than most modelling work.
- Missing scope. Repeated searches for a domain that was never modelled. Now you know what to model next, from evidence rather than from a workshop.
- Wrong expectations. Searches for things an architecture repository will never hold — server hostnames, licence counts, support contacts. Worth answering on the landing page rather than modelling.
The question adoption metrics cannot answer
A portal can be read constantly and still fail, if what people read is out of date and they act on it anyway. Usage is a measure of reach, not of value, and a stale portal with high usage is worse than an unused one.
Pair every adoption number with a freshness number: the age of the publication, and the proportion of the estate touched in the last quarter. Those two together say something. Either alone can be read as success while the thing quietly rots.
Adoption by audience, not in aggregate
A portal built for five audiences and measured as one number tells you nothing actionable. Total visits going up could mean the delivery teams found it useful, or it could mean one auditor visited forty times during a review and everyone else stopped.
Splitting by perspective is usually possible even without authentication, because each audience lands on a different entry page. What you are looking for differs by group:
| Audience | Healthy pattern | Warning sign |
|---|---|---|
| Delivery teams | Frequent, short, search-driven visits. They arrive with a question and leave with an answer. | Long sessions browsing the tree. That means search failed and they are hunting. |
| Architects | Regular visits to the catalogue and matrix pages. | No visits at all — they use the modelling tool directly, which is fine, but then they are not the audience. |
| Risk and audit | Sharp spikes around review cycles. | Spikes followed by an email asking for a spreadsheet. The portal did not answer the question. |
| Suppliers and partners | Shallow, scoped, infrequent. | Deep traversal outside their scope, which means the scoping did not hold. |
Getting qualitative signal without a survey
Surveys about internal tooling get single-digit response rates and the responses come from the people who already care. Three cheaper sources say more.
What gets pasted into chat. If people share portal links in delivery channels, the portal has become a reference. If they share screenshots instead, the links are too fragile or too slow and that is worth fixing.
What the architecture team stops being asked. The questions that arrived three times a month and now arrive once a quarter are the portal working. This is the strongest signal available and it requires nobody to instrument anything — it needs one person to keep a tally for a month before launch and a month after.
What people ask for that is not there. Requests are engagement. A portal generating no requests is either perfect or unread, and it is not perfect.
When low adoption is the right answer
Not every portal should be widely read, and treating adoption as a universal goal produces bad decisions — most often, adding content for audiences who were never going to come, which dilutes it for the ones who did.
A portal whose purpose is regulatory evidence has an audience of perhaps six people, twice a year. Judged on traffic it is a failure. Judged on whether those six people could answer their questions without asking an architect, it either worked or it did not, and the traffic number is irrelevant to that.
Decide before launch which of the two kinds of portal you are building, because the measurement follows from it and retrofitting the goal to the numbers you happened to get is how portals acquire features nobody needs.
Reporting it upward without overclaiming
At some point the numbers go into a slide for someone who funds the practice, and this is where portal measurement usually goes wrong in the other direction: a chart of page views presented as evidence of architectural value.
It is not, and the audience knows it is not. What survives scrutiny is a smaller claim with a clearer line to it: the portal answered these questions, which previously arrived as requests to the architecture team, at this rate. If the tally from before and after launch exists, that single comparison is worth every chart.
Pair it with one honest negative. The searches that returned nothing, or the audience that never came, or the domain still unmodelled. A report with a gap in it is read as a report; a report with no gaps is read as marketing, and the next request for investment is weighed accordingly.
Instrumenting without a privacy argument
Portal measurement stalls more often on a privacy objection than on a technical one, and the objection is usually reasonable: nobody wants a record of which employee read which part of the architecture, least of all the works council.
Most of what is worth knowing does not need one. Page-level counts, referrers, search terms and return-visit rates are all answerable from aggregate data with no user identity attached, and a privacy-preserving analytics tool that sets no cookie needs no consent banner and raises no objection.
The two things that do need identity are which role reads which perspective, and whether a specific audience adopted. Both are worth having and neither is worth a fight in the first year. Start with the aggregate version, use it to show the portal is read, and raise the identified version later with evidence rather than as a speculative request.
The five numbers to show the sponsor
All of the above condenses, for the person who funds the portal, into a one-page quarterly note with five numbers and a sentence each — a format chosen because it survives being forwarded.
Weekly returning readers, because return visits are the only traffic number that means adoption rather than curiosity. Distinct teams represented in those readers, because a portal read deeply by one team and ignored by nine is a different product than its total suggests. The answer rate — the proportion of arriving questions the portal resolved without a follow-up email, sampled rather than surveilled. Citations: how many design documents, risk assessments and decision records this quarter linked to portal pages, counted from the documents themselves, because being cited is what "source of truth" looks like in the wild. And corrections received, reported proudly rather than sheepishly — readers who report errors are readers who rely on the thing, and a quarter with zero corrections means either perfection or abandonment, and it is never perfection.
What the note deliberately omits is page views, time on site and anything else that rises when the portal is confusing. It also carries one sentence of interpretation per number and one action the numbers justify — retire the unread perspective, invest in the catalogue the citations cluster around. Sponsors renew what they can narrate; five numbers with a story each is a narration, where an analytics dashboard is homework. The portal's continued existence is decided in that reading, twenty minutes a quarter, which makes this note — not the portal's search or its diagrams — the single highest-leverage page the practice publishes.
Measurement, done this way, is not surveillance of readers — it is the portal listening to them. The numbers say where attention lands, the failed searches say what vocabulary is missing, the corrections say what trust looks like in action. A portal that listens gets better; one that only broadcasts gets abandoned politely. The five numbers are simply the listening, written down.