Jahed Ahmed

Image Uploads on FrontierNav

One of the most common processes when adding new data is editing and uploading images and other media. Currently, I'm basically rsync-ing images directly to the server. If anyone has images to share, I need to download them and rsync manually. So it made sense to streamline this process on FrontierNav after I introduced Merge Requests last week.

Security

The main issue around handling images are the security risks. This is true for pretty much all user-generated data. Anything in the image processing pipeline can have bugs and vulnerabilities, ready to be exploited. In fact, it's pretty common to see new disclosures for these sorts of issues every now and then. Even the tools that are used to make images "safe" are vulnerable.

Given this never-ending issue, most applications split their content into separate services; isolating the potentially bad parts from the good parts. You can see this in your network logs with domains containing phrases like "usercontent".

Firebase conveniently provides asset storage where users can directly upload files with strict rules. Given these files are hosted and served through Firebase and Google Cloud Storage, it's already pretty isolated from the rest of FrontierNav.

Versioning

As FrontierNav's data is versioned, images will need to be versioned too. For example, if one version of data pointed to "character.jpg" which had a full-body view of a character, but then "character.jpg" was replaced with a mugshot. Sure, newer versions know its a mugshot, but previous versions are now pointing to a mugshot thinking it's a full-body.

To solve this, all images must be named using a hash and file size. So when an image changes, it's uploaded as a new file instead of overwriting an existing one. "Versions" of different images are tracked in the data using their hashes rather than being tied to the filename.

Another benefit to this is that images are automatically de-duped. In a basic sense. Images that are a few pixels different won't be de-duped as they'll have different hashes.

Local Storage and Offline Support

Images uploaded to FrontierNav are not really uploaded immediately. They're only uploaded on Merge Request. This avoids rough edits from needlessly polluting storage space.

A positive to this approach is that FrontierNav now supports loading images offline from local storage, IndexedDB to be exact. The rest of the app isn't offline-capable, but this is a major step forward.

Other Media

Technically, nothings stopping me from allowing other media like videos and audio to be uploaded using the same process. But I don't have a need for them yet. Once there is a need, I'll open that up.

Image Editing

All of the above solves one part of the problem: uploading. The other part is a bit more difficult: editing. It's fine to edit images locally, then upload them. But for basic processes like cropping and resizing, it can be a bit tedious to open, edit, save and upload individual images.

Though it is tempting to implement an Image Editor just for the fun of it, I'm going to hold myself back. Instead, I'll focus on adding more data and reducing more urgent points of friction in the data entry process.

Thanks for reading.

Related Tags