jahed.dev

Experimenting with IPFS on FrontierNav

I've mentioned moving FrontierNav to decentralised infrastructure a lot. A utopia where anyone can share data together. No centralised authorities, and no bottlenecks (i.e. me) for approving changes.

So I recently tried integrating IPFS. Compared to other decentralised protocols, it's stuck around and is gradually gaining in popularity.

Before continuing, it's worth mentioning that I am by no means an expert on IPFS. These are my experiences and how I currently understand the technology.

The easy route: HTTP gateways

IPFS has a trick known as HTTP gateways. Essentially, all the decentralised stuff happens server-side and web clients can access files through a HTTP server; which itself can be decentralised through traditional means. So adding pseudo-IPFS support is a matter of adding and hosting files on IPFS and changing URLs to point to those files through a gateway.

So given a typical FrontierNav resource like:

https://static.frontiernav.net/<namespace>/<hash>.<size>.<ext>
https://static.frontiernav.net/earthbound/dfg08fd.3523.jpg

In IPFS world, it becomes:

https://ipfs.frontiernav.net/ipfs/<cid>
https://ipfs.frontiernav.net/ipfs/fd0dfvnldkwsd

Since FrontierNav stores file size and mime types separately, the fact that IPFS files are stored as content hashes (CID) without an extension isn't an issue. But to be more web-friendly, they can be stored and addressed through directories where each file points to the actual file's CID.

https://ipfs.frontiernav.net/ipfs/<directory_cid>/<size>.<ext>
https://ipfs.frontiernav.net/ipfs/vbkjhdsfg08y3/3523.jpg

This actually works well for map tiles. Instead of having to maintain CIDs for every tile (hundreds of files), we can use a directory which follows a typical map tile template.

https://ipfs.frontiernav.net/ipfs/vbkjhdsfg08y3/<zoom>/tile_<x>_<y>.jpg
https://ipfs.frontiernav.net/ipfs/vbkjhdsfg08y3/1/tile_0_1.jpg

This use-case is currently live on FrontierNav for Octopath Traveler's map, though you'll need to enable an IPFS Gateway in your settings first. I used Pinata's free tier to pin the map tiles and you can use any public gateway to access them like Cloudflare or IPFS.

Using something like IPFS Companion or Brave, it's also possible to automatically re-write HTTP requests as IPFS requests which go through a local IPFS node. So, free lunch! Kind of...

Limitations

The limitations of IPFS are immediately obvious. It's slow. Really slow. I don't know the underlying implementation, but as an outsider, I can make some observations. Unless if the node we're accessing already has the files, it'll just take ages to discover them. So public gateways aren't that useful. Maybe they're handling too many files. The only solution is to host our own gateway against our own nodes which definitely already has the files pinned and ready to share.

That's all good. However, what if we want to go full IPFS and let visitors use their own nodes? They'll have the same problem. Unless if a file is already moving around in the network, and the network cares to keep it around, finding it is going to be tough.

Torrents solve this using trackers. Trackers are servers which track the peers of a specific torrents. Torrents can include tracker URLs so that anyone who accesses it will use the same trackers and find other peers.

IPFS's equivalent is bootstrap nodes, which are peers we can add manually. Instead of trying to find our way through the IPFS network, we connect directly to the one we know has the files, and use it as an entrypoint to a more focused network.

Unlike torrents, adding IPFS bootstrap nodes is separate from attempting to download content. We need to add the nodes manually first, then attempt to grab the file. In a web browser, that's easy to do if we're using js-ipfs as we have full control over the IPFS node. However, if we're integrating with go-ipfs (via IPFS Companion), which is a lot more mature, it's a manual process for the visitor as we don't have that access.

js-ipfs has its own set of issues. As network control is limited in web browsers, there are various workarounds requiring self-hosted WebRTC and websocket servers. Even then, it may not work depending on the visitors network setup and web browser of choice; I couldn't get it to work on Firefox.

Considering the goal of moving FrontierNav to IPFS was to reduce centralised infrastructure and maintainenance, using js-ipfs is pretty much a no-go right now. So the only option visitors have involves these steps:

Unless if a visitor is really into IPFS, they're not going to bother, so the network will remaining more or less centralised. Ideally visitors should share content amongst themselves seamlessly as they navigate but it's just not possible right now. 4 years ago, I thought something like Beaker Browser would solve that, but progress on that front has been slow and volatile.

Conclusion

I've been going on about it for years now. However, web-based decentralisation is still very much in experimental stages. IPFS, despite what its marketing websites says, is still in "alpha", which is prominently displayed in their go-ipfs startup and everywhere else. js-ipfs is even more experimental and lags far behind go-ipfs. It's expected, but I wish it was moving faster. FrontierNav's current design is pretty much made for decentralisation, it just needs the tech to unlock that potential.

Thanks for reading.