When AI became part of the architecture

I've spent a lot of time recently tinkering with my home lab. What started years ago as somewhere to host a couple of websites has gradually picked up all sorts of responsibilities: Home Assistant, security cameras, monitoring, automation, and the odd script that solved a very specific problem and somehow became permanent infrastructure.

Over the years it turned into a collection of useful things that worked perfectly well on their own but didn't really work together.

If I wanted to check something while away from home, I'd end up connecting to the VPN and jumping between interfaces, SSH sessions and separate dashboards, trying to remember which system held what I was looking for. Nothing was broken, it was just scattered.

At some point I'd had enough of the friction and started building a single interface to sit over the top of everything. The original goal wasn't particularly ambitious. I just wanted one page where I could see what was happening without having to remember where every service lived.

What surprised me was how quickly Claude Code helped pull it all together. Some of my services had APIs and some didn't. Sometimes it could integrate with them directly, and other times it spun up little collectors or translators to get the data into a consistent shape.

A few evenings of casual experimentation turned into something much larger than I'd planned. Everything was feeding into a common data layer and showing up in the same UI, and health information, alerts, service status and notifications started to interact. Once that foundation existed, I found myself pulling in random things from the outside world too: bin collections, weather alerts, train delays, bus arrivals.

I'd already accepted that Claude made me faster. What I hadn't expected was how much cheaper it made exploring an idea. Things that would previously have sat in the back of my mind for months, because they weren't worth a whole weekend, became something I could try after dinner. Most went nowhere and a few turned into useful tools, but either way the gap between "that's an interesting idea" and "let's find out" got a lot smaller.

As it grew I started treating it almost like a personal operating system, and whenever there was a source of information I cared about, I'd wonder whether it belonged there. Most additions were fairly mundane. Eventually I started looking past dashboards towards tools I used regularly, and that's when I got to sheet music.

I've accumulated a fairly untidy collection of PDFs over the years, mostly pieces I play regularly, and they've always been awkward to use when travelling. Every solution I've tried has been some variation of "here is your PDF on a smaller screen", which is fine until the original PDF isn't very good. A bad scan stays bad, poor formatting stays poor, and you're stuck with whatever happened to be in the file.

Because these are pieces I actually want to sit down and play, I wasn't interested in getting to 95% accuracy and then spending an evening fixing mistakes by hand. I just wanted to upload them and get back to practising.

I started looking into Optical Music Recognition (OMR) tools to see if I could do better, and it quickly became one of those projects where you spend several evenings telling yourself you're "just doing research" before admitting you've already started building something.

I began with Audiveris, a traditional open-source OMR engine, and built a standalone service around it. The results were deeply frustrating. It kept missing ties in the first few bars, failed to end octave-shift markings so they stretched on forever, and invented random notes wherever the rhythm was slightly ambiguous. I spent days trying to fix it.

I tried other libraries like oemer, which in my testing could take an astonishing amount of time over a single page and still missed every tie. I looked at third-party cloud APIs too, but sending personal files through someone else's servers just to get bad data back felt wrong.

All that time I'd been hunting for better recognition, more accurate models, more training data. It took me a while to notice I was still treating it as a traditional software problem. Eventually I stopped asking which OMR engine to use and started asking what the music was actually saying.

That worked far better than I expected. I'd assumed it could recognise notation, but it also seemed to understand what it was looking at. It could spot when something didn't look right, suggest other readings and explain why it preferred one, which felt more like reasoning about the score than classifying shapes on a page.

When I gave it the exact passage where Audiveris had failed, it found the missed tie and worked out where the octave shift was supposed to end.

That changed the direction of the project completely, although trusting it blindly in one pass didn't work either. In one test it hallucinated a repeated chord where there was actually a moving melodic line.

So I built a validation loop. The pipeline renders a page from the PDF, splits it into small musical systems, and has Claude turn the notation into structured data. The system then compiles that into a MusicXML file, renders it back into standard notation and compares the result with the original image, scoring its own confidence and flagging the bars that need a closer human look.

What had started as a music project had quietly turned into a workflow of small, well-defined AI tasks. There was no chatbot and nothing conversational about it, just a sequence of jobs that could be run, checked and moved on from.

The current version is already genuinely useful. I can upload music, read it through a clean web interface with notation that reflows properly on different screen sizes, and gradually build up metadata, annotations and practice logs around it. It's solving the problem that sent me down the rabbit hole in the first place.

Since writing this, I've taken the sheet music platform out. The AI part worked, but everything around it didn't: splitting files, a metronome, marking up each piece's structure by hand. It was too fiddly, and it wasn't how I actually wanted to practise. The one piece worth keeping, a PDF reader that behaves the same in every browser, now lives in my file store.

That's the side of cheap experiments that gets talked about less. When an idea costs a few evenings to try, it costs very little to drop, and I learned more from building it than I would have from wondering.

Looking back, the funny thing is that I never set out to build an AI project. I just wanted a cleaner way to handle my music, and AI only came into it because every traditional route seemed to need more infrastructure, setup and specialised tooling than I wanted. The most straightforward path turned out to be something I was already paying for.

I've reached that conclusion surprisingly often lately. A couple of years ago my instinct would have been to look for a library, then a framework, then a service. Now I often wonder whether the right answer is to give an AI a well-defined job and let it get on with it. Sometimes the answer is no, occasionally very definitely no, but every now and then you end up somewhere you didn't expect.

In this case, a casual project to put an interface over my home lab ended with an AI-powered sheet music platform that, for a while, I used almost every day. Somewhere along the way AI stopped being the thing helping me build software and became another component I could design around.