Jahed Ahmed

GameDev Weekly: 24th February 2020

Progress has annoyingly been slow for a bit over a week due to personal reasons so I couldn't even pick up momentum on my GameDev work.

One thing I'm certain of is that I really should stop "thinking" about doing stuff and just do it. Thinking just leads to more problems and unknowables. It's an absolute waste of time. Doing stuff actually solves problems and eliminates dead ends.

Hopefully, I'll be more productive next week.

Thanks for reading.

FrontierNav Weekly: 24th February 2020

Progress has been slow for a bit over a week due to personal reasons. This weekly also covers the week before as I didn't get a chance to write it up for that same reason.

Merge Request Flow

Currently, Merge Requests need to go through me and it requires a full application deployment to apply. That kind of sucks, especially when I have to wait 10 minutes. So, I've been working on making data updates separate from the application.

This will also allow chosen "Editors" to update and moderate games in the future so that I'm not the bottleneck. To add to this, anyone will be able to add games to FrontierNav and with enough valuable Merge Requests, those contributors can become "Editors".

This work is almost done. Annoyingly I didn't get much time to work on it even though it should only take a few hours to implement and would unlock a lot of potential.

Activity Timeline

There's been enough updates now since I introduced the activity timeline to make it public. It's a good way for people to discover FrontierNav's range of features and I've been using it personally so see how alive the website is. So, it's a priority after improving the Merge Request flow.

GitHub Actions

I've been using GitHub Actions to run deployments for a few months now, but recently it's been really acting up. Deployments hang forever, ChromeDriver fails to launch, and so on. I've just turned it off and went back to manual deployments. It's pretty easy to re-enable again when I need it so I'm not too fussed.

Thanks for reading.

GameDev Weekly: 10th February 2020

This is my first GameDev Weekly so it's going to be hard to provide complete context on what I've been working on.

The general idea is to improve my game development skills through iteration. Those skills being: design, programming, art, music and writing. Prioritised in that order. I'm hoping I'll get into a state similar to what FrontierNav is for my web development skills, where I have a single project to iterate on.

Unlike web development, I don't have professional experience with game development. Maybe I need it, maybe not. So I'm going to try making a game without it to see where my bottlenecks are then deal with them as needed.

Read more

FrontierNav Weekly: 10th February 2020

Text Attachments

To progress the "Wiki", I've introduced text attachments. Text attachments, like image attachments, represent text files that you can upload. In the future, the text can be entered and uploaded through FrontierNav rather than requiring you to upload files.

Right now, you can manually add an overview Text Attachment Property to any Entity Type and it will render it as basic markdown on the Entity Overview page.

An Alternative to Text Attachments

After implementing text attachments, I didn't really like it. As mentioned before, it's too arbritary and prone to editorialisation. Something I want to avoid polluting the main database with.

Instead, I've been thinking about a different solution. Rather than having a single "Wiki" page, individuals should be able to create their own guides and link them to specific Entities. These guides would not be in the main database, they'll be in the user database like the forums and completion tracking are.

This would allow anyone to contribute without having to go through the stricter Merge Request progress. To add to this, the features required to make these guides would be useful across FrontierNav, as they're essentially powered up forum posts.

Email Link Login

Over time, FrontierNav will be moving away from social network logins. The main reason is because social networks are constantly changing their privacy policies and I'm constantly having to keep up with them. It's tedious to managed so many integrations and it just keeps growing. There's always the risk of people losing access to their accounts if any of the integrations stops working. This is especially worse for Twitter as they don't even provide an email to use for account recovery.

So, you can now login by providing just your email. A login link is sent to it to login. Pretty simple. Not as convenient as a one-click OAuth login, but since FrontierNav keeps you logged in, it's not a constant annoyance.

I'm avoiding Email/Password logins because it's a huge responsibility to store salted and hashed passwords and deal with hacked accounts.

It's Not A Back Button

I noticed a lot of people use the FrontierNav logo on the top-left as a back button. However, it's a home button. People would press it, realise they went to the wrong place, go back and press something else. To save RAM, FrontierNav cleans up data when going back to the main homepage so this user flow is not only wrong, but it also wastes resources.

So now, when navigating a game, the top-left navigation item is the game's homepage rather than FrontierNav's. When on the game's homepage, it changes back to the FrontierNav logo and takes you to the main homepage.

The consequence of this is a loss of branding, but I can always add branding elsewhere in the future (like the bottom-left).

User Library

To progress user libraries and allow users to add their own games, I've made some minor improvements to how libraries are stored. Libraries can now be sorted and filtered by status (i.e. 'playing', 'completed', etc.).

Activity Timeline

Almost forgot to mention, there's now an activity timeline to see what people are up to on FrontierNav. Activities include things like adding games to their library, posting on the forum and creating merge requests.

I have not made it public yet as the timeline does not include any activity before it was implemented so it's a bit empty.

This will be intergrated into Discord somewhere down the line for easier access and notifications.

Thanks for reading.

Weekly Report: 3rd February 2020

FrontierNav

I've mostly been looking into which feature to focus on next and trying out various ideas.

Notebooks

Kind of like Wikis, Notebooks can help people complete specific goals like completing a quest or finding a set of collectibles. This would reduce the need to click around FrontierNav as Notebooks can embed entity data, maps and other visualisations in a single page for convenience.

There are some pitfalls I want to avoid with this feature. A lot of them related to the problems with Wikis. Duplication, inconsistencies, etc. However, I think it's best to get it started and see where it goes than assume the worst.

Discord Integration

Before implementing any integrations with Discord, I'll need to better display activity on FrontierNav. So when users take certain actions, like marking things as complete and adding new games, those are visible for others to see and engage with. Then, I can easily bring that data into Discord for convenience.

That's the main approach I'll be taking with third-party integrations. They shouldn't really replace any interfaces, as it compromises FrontierNav's activity, but instead provide convenient access to them.

Once that basic integration is done, I can look into more complicated integrations like quick search.

Other Features

While looking at the potential of the above two features, I couldn't help exploring other areas.

Completion and progress tracking could be greatly improved by provided clear 'achievements', either from the games or user-generated. This would allow users to clearly record what they start and complete. Combined with activity sharing, it would allow others to see what people are up to and what they've achieved.

A good way to let the community grow organically is to allow them to add any games to their library without having me police which games are supported. Progress tracking would complement this as you'd essentially have a hierarchy: To complete the game, complete these objectives.

Choosing One

I've been going on about giving FrontierNav a visible 'heartbeat' for a while so I'll be doing the Discord Integration next. Specifically the focus is on making user activity more visible on FrontierNav. Setting up a Discord Bot comes with additional financial cost, so I might not actually integrate it into Discord depending on how useful it ends up being.

GitHub Actions Problems

I spent more time than I wanted to dealing with failing test cases on GitHub Actions (their remote automation service). To keep it short: tests were only failing in GitHub's environment, I tried to figure out what was going wrong, eventually gave up, reverted everything, and the tests were passing again.

My development process wasn't blocked by any of this since, knowing these issues would come up, I'm able to run everything locally. But at the end of the day, it was a wild goose chase. I hope it doesn't become a regular thing.

Thanks for reading.

Weekly Report: 27th January 2020

FrontierNav

Image Uploads

Image uploads are more or less done. There's still work left to migrate all the existing images over to the new process but it's not urgent.

The browser storage issue from last week is kind of mitigated now by uploading and clearing space on a merge request. So if users do run out of storage, they can create a merge request and carry on.

Map Uploads

With image uploads implemented, users can now upload their own maps. Along with the ability to add their own markers, FrontierNav now supports fully user-contributed maps.

Map Editing in FrontierNav

Xenoblade X of course is using its own mapping logic so it doesn't benefit from this. As mentioned previously, migrating it to the shared logic is a lot of work which I'll likely do if and when a re-release or sequel is announced.

GIFs on the Web

When recording previews for FrontierNav's features, I always bounce around between using GIFs and other video formats. It's definitely true that GIFs, archaic as they are, still drive a lot more engagement, even on platforms like Twitter which converts them to more suitable formats. I wish we moved away from GIFs by now, but it seems to be the Internet Explorer of video formats.

The preview I recorded above is over 10MB as a GIF. It's less than 3MB as an MP4. Probably a lot less as a WebM. It's a lot easier to convert a GIF to something else than the other way round. ffmpeg's GIF output isn't great whereas screen recorders like Peek do an excellent job at optimising GIFs.

Platforms like Patreon also don't allow uploading videos directly; prefering a third-party like Vimeo. However, they do allow image uploads (up to 512MB!), so GIFs work well there.

Taking all of this into account, I'll probably continue recording as GIF and exporting to other formats. Having said that, the MP4 of the GIF I uploaded to Tumblr ended up as a green, glitchy mess after it when through Tumblr's encoding pipeline. And they have a small size limit on image uploads. So... whatever.

Back on Schedule

As a rule, I'll be posting weekly updates every Friday now. It's a good way to end the week and have a record for Monday. Monthly updates will be on the Friday closest to the last day of the month.

Thanks for reading.

Weekly Report: 20th January 2020

FrontierNav

Image Uploads

I've been working on making Image Uploads public this week which will allow users to add images to the data tables. Eventually, it will also allow other attachments like audio and video.

Most of the work involved moving away from path strings that reference manually uploaded files to dynamically generated file names which may either be:

I took this approach to avoid having abritrary files being uploaded by anyone. Moderating content on the internet is a pain so I'd prefer to avoid it by making things ephemeral by default. I'd also prefer not to pay for bandwidth on data that isn't useful, so only uploading when a merge request is created does that.

I implemented the process a few months back, but my main fear holding it back is the ephemeral nature of browser storage. As far as I know, browsers only really guarantee a few megabytes of storage, if that. But I decided to make it public this week anyway instead of sitting on it. Sadly, it didn't take long for me to start seeing the first QuotaExceededError.

I mentioned it before so I guess it's time for me to take my own advice: stop using local storage. It just isn't reliable for anything. It's a shame since there's clearly a lot of work being put into it by browser vendors, but at the end of the day it's a glorified cache.

I'll be removing the local step now and jump straight to temporary remote storage. Hopefully bandwidth usage remains below any limits.

Offline Support

Following on from my problems with browser storage, I don't think the recommended approach of implementing offline support for web apps is going to work with FrontierNav. That is, using Service Workers and the Cache API. FrontierNav has a ton of data, and the chances of it fitting in whatever space a random user's browser provides me isn't reliable enough.

So, chances are, the only way to get offline support is to essentially ship an Electron app which downloads 'game bundles' and stores it on disk without relying on the browser. A pretty large effort which isn't going to get done any time soon. Especially if I want to add mobile support. Unless if there's a sudden large demand for it, it's not going to happen.

Discord Integration

In my on-going research into finding ways to increase FrontierNav's exposure, I figured a great way is to introduce a Discord bot. One that can be added to popular Discord servers for users to query.

The downside is, unlike Slack, Discord does not support outgoing webhooks. I'll need a server connected to Discord 24/7 to process any queries. That's extremely wasteful and maintenance heavy compared to a serverless solution driven by webhooks. To make the most of it, I could also start introducing features for FrontierNav's Discord server to better integrate it with the web app and keep users informed.

I have some fears around overly relying on a third-party to keep in touch with FrontierNav's userbase. I did implement a simple real-time chat feature into FrontierNav for fun, but it's a lot of maintenance. Email is barely used and comes with its own burdens too. In comparison, Discord is already an extremely popular platform with solid features that comes with no financial burden. So, much like Firebase, it's too good to avoid.

Thanks for reading.

Weekly Report: 16th December 2019

I didn't publish a weekly for the week before since it was a holiday week. So this weekly will be a combination of the two, that is the 16th and 23rd.

All of the updates are related to FrontierNav. I haven't really blogged much about specific topics for the last few months. Maybe I should.

Read more

Weekly Report: 9th December 2019

FrontierNav

Since last week's migration to data-driven features, FrontierNav is starting to feel a lot more powerful. I've spent most of this week ensuring more of the in-code features are covered by data-driven features. It's been a week of deleting lots and lots of code.

Last week I mentioned focusing on Lufia II's data entry, but I couldn't fight off the temptation of removing so much technical debt. I was constantly hitting it whenever I searched the codebase. So instead of putting up with it, I went ahead migrated the last 2 games: Xenoblade 2 and Xenoblade X. Xenoblade X is still a work in progress since it's the oldest and has the most edge cases.

Formula Properties Never?

One of the most obvious feature requests for the data tables are formulas. Since the tables are kind of like spreadsheets, having a way to use formulas seems inevitable. Whenever I need a new feature, the idea of using formulas always comes up but I always look for an alternative.

Formulas are too general purpose. It's code. And for a data-driven approach, introducing that will exponentially increase complexity when it comes to data retrieval, processing and migration.

Time will tell however as there may come a point where existing features become simpler when migrated to formulas.

Reaping the Rewards

While I was migrating Xenoblade 2, everything just started falling into place automatically. That's the great thing about having a data-driven approach to problems. Add an icon in one place, and everything connected to it will use the same icon. Change a relationship to go elsewhere and everything else follows it immediately. It really does "just work". I haven't hit a data-related bug for the entire week.

Other Changes

Thanks for reading.

Weekly Report: 2nd December 2019

FrontierNav

Data-driven Sidebars

Previously FrontierNav's sidebars were hard-coded React components that grabbed data and displayed them in various ways. Over time, the layout of these sidebars gradually converged into some simple components as patterns and similarities emerged across the various datasets.

I briefly mentioned the data migration I did last week, I didn't have much to say about it but it enabled me to finally take the steps to generate layouts using just the data. No coding necessary.

Astral Chain was the first game I migrated to this approach since it's a fairly recent addition. Pokemon Sun/Moon was the next since there wasn't a lot of data.

The two hard ones are Xenoblade 2 and Xenoblade X. There's a lot more data, but more importantly, they make use of some custom components that are difficult to standardise. For example, Xenoblade X lets you choose Probes for its Probe Simulation using the Sidebar. Both have a range of custom styling to better match their games.

Of course, all of these edge cases can wait as they're not needed for most games. This comes back to my main goal: to fully document Lufia II purely through the web client to prove that FrontierNav's data editing features are viable.

Other Changes

Browsers & Local Storage

Browsers come with ways to persist data locally using Local Storage, IndexedDB and other APIs. I've always been reluctant to use these features as most browsers tend to treat them as disposable. I can't have users making hundreds of changes stored locally and expect it to stay there. Things can go wrong.

Firefox 71 for example has broken local storage, at least the Fedora build of it. FrontierNav's authentication details are stored locally to persist logins. That doesn't work now on that browser, so users get logged out whenever they load the page. Even LastPass doesn't let me login, so I'm stuck using my phone to get passwords. I could rollback, but then I lose security updates which are more critical. I just have to wait for the patch to be release. The situation sucks.

Thanks for reading.