FrontierNav Weekly: 30th March 2020

Progress Report


Cloudflare started reporting that it was only caching 25% of my content so I made some optimisations to improve that. Previously it used to be more than 50% so I'm not sure what changed.

Data Changes

Cloudflare Caching Issues

After improving CDN caching, I ran into some issues with how Cloudflare stores 404 responses causing the Merge Request process to break. So it's partially turned off right now, which kind of sucks. Things can be a lot faster if it wasn't for that. I'm waiting for Cloudflare to respond because I'm pretty sure it's a problem on their end.

Map Tile Optimisations

Optimising map tiles using pngquant had a huge impact. Since a lot of map tiles are pretty small and don't have a lot of colours, pngquant's lossy algorithm saved 50% in file size without losing visible detail.

Map Images (the maps that load as a single image rather than tiles), lose too much quality with pngquant. So I might introduce a process to convert them to map tiles automatically to benefit from the savings.

For other images, I used optipng and jpegtran to strip excess data which had a ~5% improvement overall. Since it's a lossless optimisation it doesn't save much and lossy optimising JPEG (using cjpeg) typically introduces visible artifacts. Though, I haven't experimented thoroughly since I'm pretty happy with the current savings.

Map tiles in general would be better completely avoiding PNG and using JPEG. I currently only use PNG for the maps that need transparency. I could remove the transparency but then they'll have a solid colour background which is kind of ugly.

The ideal solution is to use lossy WebP which is like JPEG with transparency. But Safari has yet to move over so I'm not in a rush.

Backend Optimisations

I freed up some resources on the server by replacing some services with lightweight alternatives. It's a bit low-level so I'm not going to go into it too much. Basically the server is now somewhat hidden so security checks can be focused on one spot rather than inefficiently spread all over. This allows the server to handle more requests.

Mentioning Data Changes

Now that the Merge Request process is somewhat complete, I'm wondering if I should bother mentioning data changes in these weeklies. There's a merge request history on FrontierNav so I don't think it's worth repeating. Instead, weeklies will focus on development of new features rather than data entry.

Next Up

I'll be working on a calendar for Animal Crossing: New Horizons which can be used for other games too that either have in-game or real-world calendars.

I'll also be adding data for Xenoblade Chronicles: Definitive Edition, assuming that it has similar data to the Wii version. The "Future Connected" expansion will be added as a separate game like "Torna ~The Golden Country~" is for Xenoblade 2.

Both games will involve adding more features around data-entry and guide creation.

Thanks for reading.