Fun with data
Factory Internet ingests millions of events every day. Working through that data can be challenging, fascinating and occasionally fruitless, but there is usually something interesting to discover — even when it is not what the team originally went looking for.
For one of our team, that curiosity extends beyond work and into a home network and lab setup. With around 150,000 logs generated each day, their home has a level of visibility into its systems that reflects the approach taken with customer environments. For a single household, that is quite a lot of data.
The motivation is as much about learning and experimenting as keeping an eye on security. And recently, it led to a weekend project with a rather different purpose: helping their wife spot the northern lights.
From security events to solar activity
Their wife enjoys astronomy — the one involving stars, rather than star signs — and has a particular interest in the Aurora Borealis.
To keep track of potential viewing opportunities, she regularly checks Lancaster University’s AuroraWatch UK website at
https://aurorawatch.lancs.ac.uk. After discovering that they had missed a promising window for a possible sighting, she asked whether anything could be done to make monitoring easier.
For a data enthusiast, that was an invitation to get building.
The AuroraWatch project provides an API: a way for software to retrieve the data behind its online graphs automatically. That became the starting point for a project which occupied a considerable portion of the weekend.
Turning data into useful alerts
Next, a small Linux container was set up within their existing home infrastructure. Its job was straightforward: run a Python script to collect data from the API, then use a Telegraf agent to send it into an existing InfluxDB database.
With that database already connected to Grafana, a tool for visualising data, they could reproduce the project’s online graph within their own dashboard.
Recreating something already available on a website might seem an elaborate exercise, but it offered two useful benefits:
- Longer data history: retaining readings allowed activity to be reviewed and trends explored over time.
- Automatic notifications: connecting the data to an existing alerting system meant there was less need to keep checking the website.
A few Grafana rules later, tiered push notifications were in place to flag rising activity and potentially favourable viewing conditions.
It was a satisfying result. However, the intended beneficiary had some practical feedback.
A useful dashboard needs a useful audience
Their wife appreciated the effort but pointed out a flaw: the notifications went to someone who was often busy, out of earshot or liable to miss them.
The dashboard also required her to keep checking another interface. From her perspective, that did not solve enough of the original problem.
It was a useful reminder that collecting and displaying data is only part of the job. The information also needs to reach the right person in a form they can use.
So, they returned to the same API with a different approach.
The next step involved Rainmeter, a free, open-source Windows desktop customisation tool available at
https://www.rainmeter.net.
Rainmeter allows users to create desktop widgets for anything from computer resource monitoring to, in this case, aurora activity.
With some AI assistance to generate the required Lua code — a language they were less familiar with — they created a small desktop widget showing the current activity level and an indication of potential aurora viewing conditions.
The widget also included a clickable link to the Grafana dashboard for anyone wanting a closer look at the data.
Now, the information was visible directly on their wife’s desktop, without relying on someone else noticing a notification or remembering to check a website.
The colours might not suit everyone’s taste, but those, along with the other elements of the widget, could be adjusted.
Small project, practical benefit
A little data, some existing tools and a weekend of experimentation made a shared interest slightly easier to enjoy.
The project also illustrated something familiar from the team’s day-to-day work: the value of data comes from making it useful. Whether monitoring a customer’s systems or watching for the northern lights, the aim is to turn information into something that helps someone act.
If anyone is interested in recreating it or building something similar, get in touch. We would be happy to share the elements of the project!
