Something I mentioned in passing in this week’s Hype Train is that fan events, live letters, and reveals from the Square Enix team are usually far more impactful for Final Fantasy XIV than the same events for WoW when Blizzard does them, and I found myself thinking this is worth elaboration.
If you’ve played WoW for any time since Wrath of the Lich King, and maybe in more obscure corners of the internet before then, you would know that when a PTR patch went up, nearly all of its contents would leak like a sieve right out onto the internet. Same for a new expansion alpha or beta test – even without being in the game or being able to access the content on the test server, it inevitably leaks out just enough data to piece together new dungeons and raids, new story quests, and pretty much every other major piece of content that is coming. Even before models end up in the files, you can often see icon art that reveals armor looks and items coming in the patch.
So when Blizzard has an event like the recent Season 2 Q&A, it is destined to contain questions that hint at or outright state things that are coming in currently tested patches, and the community approach to spoilers tends to be looser and less well-defined as a result. For Blizzard, this is kind of inevitable and makes sense to do – rather than hiring a QA army and using tons of internal testing resources, they can instead push players into the content to seek out bugs and problems, which many players will do willingly. Top-end raiders use PTR and beta to test raid fights and gain an advanced understanding of fight mechanics before the point it really begins to matter.
Of course, depending on the patch and the types of content, sometimes these tests don’t reveal very much, and a patch has to go live and break things in front of millions in order to discover the root cause. Getting mythic raiders to test non-raid PTR content is like pulling teeth, and getting an average player to play methodically enough to uncover major problems and deliver a report with enough detail is similarly hard – but at least the mythic raider is offering a ton of feedback, even if some may not be actionable.
The end result is that at best, you get far more eyes on the early product, revealing the flaws, and it should launch in better shape for it – at the cost of losing the oomph that some of the major story moments would have had. This hand-off is something that Blizzard accepts. At worst, however, the content ends up shipping with bugs that weren’t found because a few thousand eyes on the content playing it very casually could not reveal deeper-rooted problems, and so it ships in a rough state, requires numerous hotfixes and maybe even minor patches, and you still had to give up all the lore moments for it.
After all of the years I’ve played WoW, the way in which Final Fantasy XIV handles its patches and expansions is so drastically different that it actually is amazing.
FFXIV doesn’t have a PTR, or an alpha or beta test for its content. The game’s validation process happens internally, with Square Enix QA analysts creating and running through tests for new content, trying to break it from dozens of angles. This can be scary, because it means that new content may have bugs that are not widely known and given the volume of changes that can be in some FFXIV patches, this can be a problem.
However, something amazing is also true – while there are often little bugs (and a few occasions where balance was completely skewed for a week or so), the game usually launches in relatively good shape! Even better, for a story-driven game like FFXIV, it keeps its cards close to its chest, so the story is nearly untainted and a good Free Company (guild) will often have spoiler rules in place so that everyone can play the story quests at their own pace and enjoy them, as everything is a surprise!
The end result is that the game tends to be more enjoyable on patch day, as everything is more of a mystery and there are genuine surprises waiting around many corners, coupled with a minimum of bugs.
Now, the question I would pose is this: can WoW do something similar? I certainly think it would be possible – with enough QA analysts and testers, you could theoretically build the framework upon which a completely internal test could take place. I think there are a few logical challenges that Blizzard runs into that Square Enix does not, however. Zones in FFXIV are functionally separate from one another and so changes to one zone do not bleed into others, and the way in which dungeons and raids are added to the game often doesn’t require much testing, as the mechanics are simpler and portals into dungeons often end up dumped into existing spots (one dungeon in the Stormblood patch cycle is literally on an overlook with no major distinguishing features to mark it other than a mount that is added there during the quest that unlocks it). Dungeons in FFXIV almost always require an unlock quest, so testing that is typically much easier, I would imagine, and once in the actual dungeon, FFXIV dungeons are usually simply 3 bosses and a small handful of trash, which would not require a lot of testing. Raids are similarly small, with normal raids being split into individual boss encounters and alliance raids being more akin to WoW raids, but with less trash and slightly less mechanical density.
WoW’s tapestry of interwoven mechanics would make testing the fights harder, plus, with fewer obvious tells compared to FFXIV, raid and dungeon testing is also largely about design comprehension rather than actual bug fixing. FFXIV uses large orange telegraphs to tell you when a thing is going to happen and to move, and has slower-paced combat and raid fights, while WoW has a million things happening per second and you have to be a fair bit more attentive to really catch everything that is going on. WoW also tends to make larger changes to its world, necessitating a lot more work to ensure that changes do not negatively impact player experience.
I think the larger problem with WoW’s PTR model is that players are not advised of everything that should be tested, nor does Blizzard always seem aware of every individual component that has changed. While one can (and I would) argue that Blizzard probably should be aware of the changes they’ve made to the game, I wonder what impact the player experience would suffer if there was just an indexed list of every single change made to the game data with a patch for PTR testers to examine. Surely, such a list would be published alongside all the datamining that already takes place, and one can imagine that it may further deteriorate the gameplay experience for everyone – sapping the mystery out of new content.
However, as the Battle for Azeroth trudges on towards its continued patch cycle, a point that sticks to me is that nearly every patch and even the non-patch season launch have had numerous problems. The point of having a public test for players is to gather that feedback in advance, and while I am sure Blizzard is seeing and reading the stuff they get, I often wonder how much action is taken from that. Further, it creates a tense environment in which players often perceive testing as a meaningless exercise, since things seem to largely ship in the same state as the test server anyways, short of raid encounters (those darn Mythic players) and some limited things that are very clearly broken. While I think the Reddit-based edgelords slinging out joke titles like “Beta for Azeroth” are largely wrong and should probably just be ignored, it is hard to feel optimistic about the game’s state when each patch for BfA so far has had some major bug or gameplay issue. Take the recent state of War Mode, for example – the Horde crying about being outnumbered as Alliance farm them for reward gear (I shed no tears for them after the first week of just this expansion), and the end result being that Blizzard decided that the quest Against Overwhelming Odds would no longer be offered at lower percentages of outnumbering…30 minutes after the quest had already been available for the current week.
Meanwhile, when I play FFXIV, I rarely encounter any glaring issues that make me as frustrated, and it legitimately fascinates me how different the approach of these two companies are and how the results of their approaches to quality assurance are completely inverted from what you might expect to see. Even better, the story has moments of legitimate tension and surprise to offer, because while there are dataminers who work with live files in the game, there is no equivalent to the information economy that exists around WoW – explicitly because Square Enix gatekeep the hell out of their in-development code. Now, I might generously attribute that to a desire to maintain story secrecy – and some of it is. Patch notes in FFXIV read like riddles when the details conveyed are intended to be masked to allow an in-game experience to occur. Even still though, there are a lot of challenges that FFXIV faces – worldwide simultaneous launches with 4 different localization teams who work from the original Japanese data and need to often conjure up well-fitting translations of every new addition to the game with a tight deadline, managing content across PC, Mac, and PS4 versions of the game (and prior to Stormblood, the PS3 as well), and these things only make the feat of their relatively painless releases with no public testing all the more impressive.
So, in the end, a question – if you could snap your fingers and get WoW to have completely sealed testing, even if it meant content launched with bugs, would you take that in order to get a completely spoiler-free launch window?