Every project starts somewhere. This one started with two things that happened to collide at the right moment.

The Constraint

I wanted to build an end-to-end data engineering project. Not a tutorial, not a sandbox — something real, with actual data, running in production. The full stack: ingestion, transformation, storage, a live dashboard.

The constraint I gave myself was simple: free, open source tools only. I wasn’t going to spend money on a personal project just to prove I could build something. If the tools couldn’t hold their own without a credit card, I’d find different tools.

The Data Problem

The harder constraint was the data. I needed something real. Not synthetic, not historical-only, not something I’d lose interest in halfway through. Weather data felt generic. Titanic and movies felt like tutorial territory. I wanted data I’d actually want to look at after the build was done.

Then I remembered that I love football.

Football data from a major European league would mean: a live API with ongoing updates, a season structure that maps cleanly to a data model, and enough statistical depth to make analytics interesting. Fixtures, results, lineups, referee assignments, standings — the shape of the data is almost purpose-built for a star schema.

The Personal Angle

The league was the next question. I’d recently moved to Denmark. I follow the big European leagues like most football fans, but I realised I knew almost nothing about Danish football — the players, the clubs, the rivalries, how the season works.

That’s the moment the project clicked into focus. I wasn’t just going to build a data engineering showcase. I was going to build something I’d actually use: an analytics product for Superligaen, the Danish premier football league, aimed at people like me who want to understand it better.

That’s how this started. A love of data engineering, a constraint around cost, and a new country I wanted to understand through the sport I already loved.

The rest is what went wrong — and eventually right.