Safe on its own, risky together

I hadn't realised how much the UK gives away until I started using it.

Every English bus operator publishes its timetables, and where each bus actually is right now, through the Department for Transport's Bus Open Data Service. National Highways publishes road and lane closures. The Environment Agency publishes live river levels and flood warnings, no key needed. The Met Office publishes forecasts and severe weather warnings. National Rail publishes live departure boards. postcodes.io turns any postcode into a location, free. And if you're with the right energy supplier, it'll hand you your own half-hourly meter readings through an API.

All of it is free, or close to it. Most of it is under the Open Government Licence. I've wired most of it into the private console behind this site, and it's become some of the most useful software I run.

None of it is as tidy as the documentation suggests.

A single bus journey arrives in three different standards: one for the timetable, one for where the bus is, one for disruption. Road closures come in a fourth, and that one has no field for the road's name. A closure is a line of coordinates, and the road is only mentioned in a free-text comment. The flood API measures distance in kilometres. Everything else in my code uses miles.

My favourite: the council's public bin calendar. It looked like a schedule. It turned out to hold 59 entries across 13 years, because it only ever lists the exceptions. An ordinary week has no entry at all. Data can be complete, official and still not answer the question you thought it did.

Which is fine, really. It's raw material, and working out what it actually means is most of the fun.

On its own, each feed is mildly interesting. Joined up, it tells you things nobody publishes.

Put the bus timetable next to where the buses actually were, and keep it for a while, and you know how reliable each route really is. Add the walk to the stop, and you know when to actually leave. Put river levels next to the forecast and the flood warnings, and you know whether the river near you is a story or a problem.

Every input is public. The useful bit only exists because someone joined them up, and that turned out to matter in a way I hadn't expected.

The same joins that make this useful also describe me. Which bus stop I care about. Which station. Which stretch of road, which river gauge, which bin round. My energy use, half hour by half hour, which is to say when the house is empty. Each one is harmless. Together they give away where I live and a fair amount of my routine.

Security people call this the mosaic effect. Individually innocent pieces of data become sensitive once they're combined. I'd known the term for years, but it took building it myself for it to mean much.

It's why there are pages of my own console I'll never screenshot for this site. Every feed on them is public, but the combination isn't.

I'm not a security specialist, and none of what I've done is exotic. It's mostly deciding, on purpose, what gets joined, where, and who can see the result.

Every call to an outside service happens on my server, never in a browser, so the providers see a server asking about an area rather than a person browsing. The map of Great Britain is one file on that server, rather than a tile provider learning which streets I look at. My phone reports its location to my own server, not someone else's service, and the public site in between can pass it on but never read it.

And I've started treating the joined-up history as more sensitive than any of the feeds that went into it. The raw feeds are public. The history is backed up, access-controlled, and never published as it is.

This is where it stopped being about my house for me, and AI is the reason.

I think a lot of organisations have been protected by obscurity rather than design for years. The sensitive spreadsheet sat in a shared folder anyone could open, and it was safe because nobody knew where to look. An AI assistant connected to that data, or a retrieval system built over it, removes the obscurity overnight. It will happily find and combine everything a person can technically reach. That's exactly what makes it useful, and exactly the mosaic effect, at the scale of a whole organisation.

So I'm not sure the old line between a data strategy and a security strategy holds any more. Who can see what, how data is classified, what gets joined and where: that used to be governance housekeeping. Put AI on top of it and it becomes a security control. I've written a data strategy, and I'm now building with retrieval over an organisation's own data. Both have nudged me the same way: I've stopped asking only whether data is sensitive, and started asking what it becomes next to everything else.

The free data the UK publishes is a genuine gift, and I'd encourage anyone curious to go and build something with it. I haven't found a better way to understand what data can do.

Just watch the joins. Looking back at my own console, the most sensitive thing on it isn't any of the data I collect. It's what I made by putting it together, and I'm not sure I'd have noticed if I hadn't built it myself.