The Reaction To The Dragonflight Release Date – What Needs Testing, Anyways?

In my last post on this topic, I expressed a degree of doubt that Blizzard’s statement of a 2022 calendar year launch for Dragonflight as an expansion was realistic. Ultimately, the way I see it, there’s a combination of factors – how much content needs testing, the balance and new systems (of which there are still some), and how these things can be focus-tested through a mix of staggered content rollouts to a public alpha and beta test and internal testing – that plays against the stated intention of Blizzard. However, I’ve seen a few interesting discussion points come up in the days since, and so I wanted to look at them a little bit before Final Fantasy XIV’s news cycle begins going crazy with a patch live letter and a new small patch in the coming days.

Talents Mean Less Testing

A common refrain I have seen is something in the vein of how talents, as they are, reduce the testing burden on Blizzard because it isn’t the interlocking layers of endgame and post-level cap progression we’ve had since Legion – just a straightforward, standard rollout of talents over the course of leveling. Some people have even stated simply (and incorrectly, I’ll add) that “they’ve done talents before so it’s an easy rollout.”

Here’s the thing – this sentiment isn’t altogether wrong. The idea that talents require little testing as they’ve done them before absolutely is, because it quite obviously doesn’t take a rocket scientist to look at the Dragonflight talent system as stated and realize that it is vastly different from any system that came before. It looks like a tree and acts like a tree, but what is loaded into it and how it is structured and rewarded is quite different than any other system we’ve seen before. Blizzard is going to be tying active abilities at both the class and spec level into the trees, and using them as the primary guide of leveling progression for a character, so a lot of testing will need to be done. The very nature of this new system, in fact, allows for a lot more theoretical build diversity and will require some amount of players in the wild making weird combos to find any gaps.

However, there is a lot running in favor of the idea that talents ease the testing burden. For one, a set of points in the double digits has a lot more opportunity to be unbalanced without disrupting overall class and spec balance, because the relative impact is smaller per point. In the current system, you get 7 total points, and each one, as a result, has been designed to be immediately noticeable, significantly powerful, and loaded with effects. The desire to have 21 unique choices per spec at that level of value and power creates a problem with balancing, because each ability has to be strong and interesting and it means that imbalances are both more likely and larger when they do occur. Dragonflight will have 61 talent points in total, and by that very nature, each one no longer needs to be so good it can carry you for 5-10 levels without another choice. Passive choices no longer have to create massive buffs to multiple abilities – you can get a choice that is just a 1% increase to critical strike chance for one ability, and that would be fine, because you’ll get 60 other points to put in elsewhere with similarly small effects. It’s still a delicate balancing act, but at the same time, there’s far less numerical value to worry about creating balance, and so talents can even be slightly unbalanced just given the raw number of choices. In the current system, every talent must be roughly equal with some tolerance for situational variance or playstyle choice, because a talent that comes in at 5% weaker when it is 1/7 is bad compared to a talent point that might be 5% weaker than another one but is 1/61 of your total build. The true challenge is to make sure that a player’s total overall power is balanced, so that two completely different 61 point trees end up with the same relative level of power (with situational variance built in for things like single-target versus AoE builds, etc).

Another key benefit of the new endgame systems is that, so far at least, there don’t seem to be other models for power gain outside of gear. If that remains the case (I will disclaimer this heavily because I don’t trust that Blizzard is fully off the borrowed power teat until I can see it firsthand), then there are no synergies or layers to test. In each of the last several expansions, testing any player power required analyzing layers and layers of it – in Shadowlands alone, you had to know how each Soulbind tree affected a given class or spec, how each Conduit interacted, how Legendary choice factored into these things, test the active Covenant abilities, and then you had to play these things off each other and with the talent structure in-place. That creates a lot of permutations, because you had to attempt, at the very least, to know how most of the 2,187 possible talent builds per spec would work with the 12 available Soulbinds in total and the number of Conduits, and I am not doing all that math but just at Soulbinds alone you’d get to a ridiculous number of permutations and then adding just one legendary choice to that creates a staggering number of additional choices in the totality of endgame choices you can make. While you can have stuff like gear on a template and adjust for outliers pretty easily, these systems are a lot harder to nail down. Dragonflight, to our current understanding, has none of this save for a larger talent tree. The number of permutations available is probably still pretty large, but crucially, you’re no longer accounting for multiple layers of choice with multiplier effects that feed off of smaller, but highly impactful choices – and the limits placed on the talent trees themselves create an effect that constrains the possibility space quite helpfully.

So I think that talents are a blessing overall still here, in that each choice needs to do less than they do today by a wide margin and that opens up the possibility for less testing needed to arrive at a satisfactory outcome. There is probably going to be a big focus on testing system feel – the flow of the trees, how the active ability choices in the class tree feel and work and what gaps they create compared to Shadowlands, and then the big focus is going to be on the pivotal multi-choice slots in the spec tree where you get a choice of two big abilities. Will this be easier to test without the added layers of complexity? Of course it will. Will it be a downright simple, no testing required because we all know talents situation? Of course it won’t. Will it end up actually taking less time to test these things? That I don’t know and I am not confident enough in any direction to even speculate.

The Effect of Fewer Squishes

From what we know right now, a character level squish is not in the cards for Dragonflight. We’ll go 60 to 70, right on the 10 level track we all know and are pretty comfortable with over the last decade of WoW and the 5-level experiments in the middle. That alone creates a drastically reduced need to test for balance – because the game isn’t going to need to understand how to squish down the content that is already there for character level and the Shadowlands implementation of that squish should continue.

Will there be an item level squish? We don’t know either way as of now. I looked high and low for information and couldn’t find anything to suggest a squish either way. The past few expansions, whether or not item stats have created huge increases in player power, there has been a squish. It is possible they may not even know as of yet – the Shadowlands demo at Blizzcon 2019 had the existing gear from BfA on characters at the existing item level and stat values, which did not end up carrying into the finished product. At this point, just on habit alone, I almost expect an item level squish again – but Shadowlands has even lower overall stat values per character than BfA did when it was squished down. Item level is lower, but health is, damage output is, you name it, it’s lower right now. Blizzard has skipped item squishes once in prior history since they started doing it – the WoD prepatch squished, then Legion did not, but Legion scaled up ridiculously high, taking tanks from 700,000ish health in late WoD and putting them up to over 12 million by the end of Legion.

The item level formula in WoW is something we’ve actually been told now by the developers, at least in a loose sort of way. Each item level increases the power of an item by 1%, but this 1% is a multiplicative increase, which means the higher the item level goes, the higher value a 1% increase actually is. Thus, scaling is exponential, not linear, so that leads to why Legion was so inflated – with item levels approaching 1,000, the value of a single level was massive and growing at an unchecked rate. By doing what they’ve now done in both BfA and Shadowlands, the team has curbed this growth massively, bringing down the values a lot by reducing gear item level directly instead of what the first squish did, which was keep the levels intact but flatten the exponential scaling in old content, which made higher gear item levels create a steep climb as you escaped Pandaria and moved into WoD content.

When Shadowlands ends, the highest item level gear will be at 311 item level, itself less than the first tier dungeons in BfA offered. Will a squish be necessary? I…think not, at least now. If Blizzard’s aim is to get the game out by the end of the calendar year, then I think it would be a calculated and smart move to simply leave item levels alone for now, let the curve ride up as it will, and then consider flattening it back down in 11.0. If we assume an inflation of around 160 item levels (about as many as the average player had from BfA into current Shadowlands) then Dragonflight would still end under 500 item level, the point where scaling began running away in Mists of Pandaria.

Content Testing – A Lot Still To Look At

One of the biggest parts of testing an expansion, though, is just the baseline content of it, which Dragonflight is promising a lot of and a lot of new stuff. The seemingly easier testing is the zone and leveling flow – making sure players can get through the story of the expansion at a satisfying pace without hitting brick walls or obvious gaps. Shadowlands actually notoriously had problems with this – as a phase 1 beta tester there, I hit big experience walls where the on-rails path through the new zones left me 2 levels short of the finale, and because there were fixed levels on side quests per zone, it meant that early sidequests like those in Bastion or Maldraxxus were grey or green, so my first character to play the full experience of the Shadowlands beta was walled out of the finale and unable to finish. It required a fair amount of buffs and readjustment for Shadowlands to have a decent progression of levels at launch, and it did stick the landing overall, but the beta period was quite hairy for it.

There’s also the major addition of Dragonriding, which is going to require a non-trivial amount of testing to ensure that players can use it well but also that the limits Blizzard clearly wants to have on player movement are in-place and not able to be subverted. There’s also the customization systems for your dragon and the progression mechanics that will increase your ability and duration of flight – all of that will need detailed analysis and players attempting to break it.

The new tradeskill gameplay, like gear, stats, and orders, will all also need testing. This represents a huge change to existing subsystems in the game and it is going to take a lot of player perspective to make sure that this is working right. This (and all of the new systems and gameplay modalities, really) are also going to need external players trying them and seeing how clear they are, if the quests and tutorials around them explain things properly, and then to identify and close any gaps.

Lastly, we know we’re getting reworked versions of some Shadowlands systems, mostly Renown, and that may require some testing as well. The indication is that Renown should basically be a story gate and should advance all dragonflights simultaneously, but we don’t know for sure what form that will take in the finished product. The idea of Renown returning at all feels bad to me and sets me on alert, and I think that the early builds are going to help determine how much hope I hold out for the expansion. Renown served as a needlessly overwrought gate to content, where starting an alt later or getting into the game post-launch meant overcoming a wall with an obfuscated catch-up system that was incredibly poorly explained in the game, and I much prefer systems like what they did with Zereth Mortis (and almost all past time-gated story content) where they simply unlock content at given intervals post-patch, so a returning player or late starter has a backlog they can sprint through instead of needing to unlock each intermediary step and then keep moving with the weird pace-breaking Renown instilled. I could rant about Renown all day (and already have too much here) so moving on!

Will Dragonflight Make 2022?

In my post on the announcement, I was very cynical about the odds of Blizzard making 2022 with this expansion, and I laid out a case I still believe in there. Having said that, I do think that it is fair to say that the design of Dragonflight, such as we know it today, is a net positive for the goal of reaching a 2022 release date. If it truly has no borrowed power systems, if Renown is solely a story gate, if there is no item level squish, then that leaves Talents, new systems like Dragonriding, and the new content added with the expansion to test. That is no small feat, but yet it stacks up quite favorably compared to the immense burden that Shadowlands testing was, with Torghast, the Maw, the zone flow and leveling, layers of borrowed power, and all of the ways those things integrated with each other. 4 months to just test content seems pretty reasonable, all told, and yet I still find myself somewhat skeptical of the release window claimed. A big part of why is simple – there’s no public build yet, and while a 10.0 build has hit the CDN in the last week, we have yet to see much of anything coming from it. At the same time, Blizzard has started making more noise that they liked the mystery around things that they encrypted for PTR and did not test, and so that may signal more content hidden from the public and left untested outside the halls of Blizzard. That’s…fine to a point, but Sepulcher of the First Ones is one of the worst balanced raids they’ve ever made even with public testing of most bosses, so I find the idea of leaving content even more players will touch off the PTR and untested by critical eyes troublesome from that perspective.

If Blizzard has already been rigorously testing the content internally, it might be fine, coupled with public rounds of testing designed to isolate content into smaller pieces for specific, direct feedback – but I have my doubts. In spite of the factors in favor of Dragonflight being a relatively easy testing window, there is still a lot that would need testing and will absolutely need to be checked in depth, and with each passing day that neither an alpha or beta test launch to the public, the less faith I have in a 2022 release. Is it likely that the actual development tasks needed are done? I would bet they are, or pretty close to it, at least. But for Dragonflight to be what World of Warcraft actually needs – a well-received, revitalizing expansion for a franchise on a downward turn – it needs to get it right 100% of the way.

That is my main concern over anything else. Could they make 2022, sure, of course they can – throw enough development resources at it, make some assumptions about how things will play out based on limited public testing coupled with internal testing, and then balance and fix post-launch – it can be done and it could even be good! For me, though, the concern is this – launching in 2022 without a serious public test that shows follow-through on the hollow promises of player feedback acceptance would just be another WoW expansion like the last two – alienating to core players, slowly grinding the community down and out of the game, and taking the once venerated brand of Warcraft further down into the mud. Much of my fence sitting right now is based on this – I have no proof that Blizzard is backing up their words with actions yet on the quality of the game and responsiveness to players. They say they are, and that’s great, but we’ve been around this block before and the Dragonflight announcements just didn’t have enough content to assuage my fears. Until a test is in my hands (or someone I trust that I could watch and get input from), then all I know is that Blizzard has promised much the same for years and has yet to truly deliver on that open, iterative design process with fan involvement. They’ve done better in late Shadowlands, as the mire they found themselves in became ever clearer, but it took a lot of extenuating circumstances to get there.

So if they make 2022, a thing that is very possible and at least somewhat likely, then my concern is that it is rushed and will trip over itself. And, well, time will tell.

10 thoughts on “The Reaction To The Dragonflight Release Date – What Needs Testing, Anyways?

  1. The cynic in me would comment “nice collection of straw men you have there”. If the “they” that are commenting this are on some fora somewhere, they need to up their game. Especially that one about testing talents. It’s as if they don’t know how software development works.

    NARRATOR: THEY DON’T

    Truth is (and this doesn’t make for a nice long blog post but) somebody was asked if they could do it, they said they could, so now the dice are rolling. I imagine that Kinder Gentler Blizz will roll out some massive crunch for this, and, 50/50 chance, announce they couldn’t make it and offer refunds if you really want ’em, then release on or around the date they released TBC (o.g.) and try to dress that up to make it look like they meant to do that. Hell, I imagine they have people in marketing already working that angle.

    But yes there will be bugs. The difference now after Cata is that things are easier to tweak since Cata released, and they’ve been working to make that even easier ever since. We’ll see some bugs, some real cripplers, wouldn’t rule out a CTD or two, and everybody will claim THEY WERE RIGHT and hate-play the game anyway.

    Like

    1. The “would” in your comment is misleading, but I suppose that is the joke.

      The Wowhead post about estimated timelines for expansion releases was a goldmine that included a fair bit of this sentiment: https://www.wowhead.com/news/is-dragonflight-too-early-world-of-warcraft-expansion-timeline-comparison-327478

      I really enjoy their writing, but Wrathofkublakhan’s post on pre-orders also calls out an adjacent idea – that talents are existing spells and thus won’t need much if any testing: https://coffeecakesandcrits.wordpress.com/2022/06/23/pre-order-dragonflight/

      To be frank, your comments for quite some time now have been veering towards condescending, passive/active-aggressive, and tedious. If that trend continues, your commenting privileges will be revoked.

      Like

      1. I guess my cynicism towards cheap-seat commentary (not pointing at you, you’re basically just bringing it to light) is getting the better of me. For over a decade I’ve been seeing this whole “all you have to do is …” attitude that exposes a complete and utter misunderstanding of software development lifecycles, much less a complex game like WoW. I’ve had my say on that on my own blog several years ago and that makes me feel like I’m screaming into the void sometimes; former players of the game never leave, but new ones arrive with the exact same misconceptions and so the spirit continues.

        Feel free to suspend, immediately.

        Like

  2. Given how open the Betas/PTRs have been for Blizz in the past, there’s been little reason to go into them if you want to be surprised at the content when the servers go live on a new expac. Admittedly it has been encouraged by Blizz’ own focus on raids, but the greater community that raids really doesn’t give a rat’s ass about story and development of the game. By pushing some of that out into mystery-land, it means that Blizz is beginning to reassert some creative control over the creation process, which tends to get lost when people are given full access to the game via a PTR.

    And before you mention it, yes, yes, I know that only a subset of players actually go into the Beta or the PTR, but the fact that it’s open and people already have guides written and posted (or, in today’s age, shot and uploaded on a streaming service) when the game is released means that Blizz has a “solved system” upon release: everybody knows the meta and “why aren’t you doing the One True Path anyway, scrub??” Taking that back under control isn’t a bad thing, even though it does entail some risk. It’s been a long time since Blizz had an unkillable boss problem that had to be patched, but I’d much rather have that and have people learn about the game just like every other game (board or pencil-and-paper or video), by playing it themselves, rather than everybody basically ignoring most of the options presented to you in favor of just one path that the theorycrafters have figured out.

    I remember back in the day when Mists got so much beta testing for so long that I speculated that the average player already knew how the game would play beforehand, so why would they continue to sub once their six month sub offer (complete with bonus features) ran out? They’d already played the game in beta –and there were times when it felt like I was practically the only person to NOT play it in beta– so it was no longer a new game to them at all. And the subscription numbers kind of bore that supposition out: once the initial “new game smell” wore off, people left. Up until Cataclysm released, WoW hadn’t really had any sub losses; it was either flat or an upward trajectory. Cata introduced Blizz to falling numbers, and Mists codified the “burst of interest upon release and then a plummet after” that became standard for WoW releases afterward. Trying to counter that –and the rush to the end that characterizes WoW– by making it all new to people isn’t necessarily a bad thing.

    Like

    1. I did the beta one year, it was the most boring time of my life. I DID work in QA at the time, but QA for card processing software is SO MUCH DIFFERENT than for a game that to this day I still don’t know if I just didn’t know how to game QA, or if Blizz was being criminally skimpy on disclosing what changed from build to build.

      But I WILL get in on the PTR as soon as possible to start working on any addons I bring to the party, such as the ones I do to gather crafting stats. And in the same vein eventually it’s good to have the guide makers in there getting the guides ready (but, yo WoWHead, flush your yield stats before rolling over to production, will ya?)

      Beta has its uses, but most people (I think) hit it with the wrong emphasis for Blizz to benefit from it much.

      NEXT WEEK, ON MY PODCAST, I WILL BE IN DRAGONFLIGHT BETA AND WE’LL GET THE REAL SKINNY. DON’T FORGET TO HIT LIKE AND SUBSCRIBE!

      Like

      1. I used to do software QA as well, but that was back in the 90s for a CAD/CAM/CAE product that competed with Pro/Engineer and Catia. While I coded for the custom test harnesses that we used, the beta testing was so unlike open betas for gaming that they were worlds apart. In ours, we’d have some of our big clients bring in their CAD designs and try to work with them on the new stuff, and they had a laundry list of things they specifically wanted to test. Given that the overall design goal –a bug free product– was something that both gaming and productivity software companies desire, the need to keep certain aspects of development under wraps makes game development far more difficult to QA/Beta test. Well, there’s also the hardware aspect that makes it difficult to test as well, but at least Blizz doesn’t necessarily have the issues that Bioware (for example) has dealing with Frostbite.

        IMHO, by the time a game company reaches a public beta, they’ve signaled that they believe this build is going to be stable enough to eventually release, and they’re mainly looking for show stoppers or areas where there are glaring errors. If a public beta looks like crap, then they should never have released it as a public beta and management screwed up handling the QA beforehand. Now, that HAS happened in the past, and even with my old company, where myself and other QA people told our upper management that the product wasn’t ready for release but we were ignored because “release schedule” and “damn the torpedoes!”. (Which sounds a lot like how EA handles things like release schedules now that I think about it.) But some days the builds are an adventure just to get back to where you were in stability in the previous build.

        But yeah, I’d bet a month’s salary that crunch is brutal and ongoing right now over there.

        Like

    1. Unless Proletariat were already on the job, I see no way this can result in quicker turnaround. SIX MONTHS to ramp up on a hugely complex app AND finish testing AND release it. The math doesn’t work for me.

      ALTHOUGH – with a name like “Proletariat”, what are the odds they’ll make it into next year without unionizing? 😀

      Liked by 1 person

  3. I’m not completely sure if the item squish is really such a problem compared to the other things, where I mostly agree with your assessment – if we look at power gains while leveling in past expansions there have been several modes of failure. Sometimes you could still to the new raids in your last-expansion BiS raid gear. Sometimes you replaced your BiS at old level +3. So the power gains for, say 60 to 70, are not that different that outright squishing, but it’s all guesswork 😉

    One sentence where I think you might be misremembering is “assume an inflation of around 160 item levels (about as many as the average player had from BfA into current Shadowlands)” – because I still have the numbers here 😉 I was a casual with a lot of time in BfA, didn’t raid, no M+ – and my best-equipped toon was at iLevel 89 (432 before) at the new char level 50, that was still full epics and post-Nazjatar. A staple of alts that just reached max level and collected some gear were right around the 80 (416/417 before) mark (also noticed this when I started leveling in SL and could barely join random normal dungeons) and the totally neglected ones reached max level at iLvl 54-60. So I’m not saying this is average, but your 110 number is already (kinda) deep into Raid/M+ territory.

    Like

    1. That’s a fair point – and for me personally, I was pretty deep in Heroic raid gear (no M+ though) at the tail-end of BfA, so that definitely skews the item level I rolled in with (and while I remember it being higher, some time on Wowhead looking at dropped item levels casts doubt on it!). That being said though, I do think the item squish bears some problems – not for new content, though, as Dragonflight will be designed from the start for whatever the target item level is (although how balanced to that target it will be remains an open question).

      When I’m thinking of item level squish problems, I think of two things – the way they try to keep some form of scaling in the expansion prior to the squish (for Shadowlands, everything pre-BfA was compressed down into a roughly 50 item level range while BfA gear went from 50ish to 95 save for the Heart of Azeroth and the legendary cape) and how old content reacts to the squish scaling. A substantial part of the endgame for some is farming old transmog and mounts solo, and every time they’ve done an item squish, they’ve had some measure of difficulty getting it right in older content.

      They’ll need to ensure Shadowlands content is still viable in correct gear while making sure that power progression remains properly mapped for content where the item level is more squished-down. That being said, I’m still not 100% sure they have to squish as-is – a 200-item level curve expansion wouldn’t be big enough to necessarily break stats at current levels, but it seems like an item squish is just an inevitability every expansion now to keep the curve flattened and give power creep some room to live.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.