Weekly Report: 11th November 2019
Merge Requests on FrontierNav
After building the automation pipeline last week, I moved onto the main feature driving it: Merge Requests.
It's worth mentioning again that FrontierNav's data is database-free. The data is packaged as part of the website for various reasons. The dynamic nature of it makes it very easy for changes to conflict and individual changes may not be valid without the whole.
Given this, there are two ways people can contribute to FrontierNav:
- Providing me the data which I transform to be FrontierNav-compatible,
- Using FrontierNav's UI to modify data.
The current goal is to make 2 easier so that I'm not constantly spending time doing 1.
Currently, users can use the Data Tables to modify data in a basic spreadsheet-like manner and add markers on the map. However, to apply that data for others to see, users need to export the changes and send it to me manually. Then, I need to re-apply those changes locally and deploy them.
This process is slow and tedious, even when I'm the only one acting on them.
The idea behind Merge Requests is pretty simple.
- Users can now submit their changes directly from FrontierNav without needing to export them.
- An admin can review those changes within FrontierNav and approve them.
- An automated pipeline picks up approved changes and deploys them.
This week, I pretty much implemented this entire process. I won't be relying on the automation just yet as it's not been fully proven. Instead, I'll run the same scripts manually to make sure it's working.
I was going to put a recording of the process here but OBS is being a bit glitchy at the moment.
As much as I hate tying FrontierNav's features to Firebase's Real-Time Database, I admit it's a massive convenience and hard to avoid.
I may have made it even harder to avoid when I wrote firebase-rules, which has allowed me to re-use chunks of logic that enables things like rate-limiting, readable conditional statements and manual indexing.
The only thing really stopping me is are the usage limits but even at that point, it might be easier to pay up than move to something else.
So why do I hate it? Because Firebase is extremely opaque. It doesn't provide much in the way of details. Which I don't blame it for, that's its selling point and that's why I use it. But when the time comes where I outgrow it or I lose access to it (knowing Google), I need to be ready.
A while back, I decided that all features driving FrontierNav should use the web client. This was to avoid writing one-off scripts and piling on technical debt. If a feature is available on the web client, it's technically available to everyone, including myself when I'm not on a workstation.
So when I implemented Merge Requests, I needed a way to automate the deployment process as though a human could do it. This way, if the automation is no longer available, I could easily do it myself.
Initially, I used Nightwatch to run these automations. Nightwatch is what I use for integration testing so it supports browser automation. However, it's not a good fit for general automation. Nightwatch is focused specifically around writing tests and steering away from that is difficult within its test-oriented framework.
So I moved over to WebdriverIO which recently split its test runner from its automation. Perfect. I had to figure out a few things that Nightwatch provides out of the box, but it wasn't a big deal. And WebdriverIO's documentation is so much better.
I'm now planning to move over entirely to WebdriverIO in the future, with my own wrapper to avoid being locked into a framework. Nightwatch's activity has been a bit on-and-off recently with a focus on selling their testing solution, and the documentation isn't very good.
Thanks for reading.