MMO-Champion BlueTracker - GC: Tanking Testing Assumptions

2008-10-01 08:41:54

Just a couple of quick questions:

1. When you guys test mitigation/avoidance of Paladins versus other classes (Wars, DKs, Bears) are you assuming we have Blessing of Sanctuary and they do not?

2. Similarly, when testing Warrior mitigation, are you assuming they hit Shield Block every cooldown or not?

Thanks.

Sabindeus

2008-10-02 22:39:06

Re: GC: Tanking Testing Assumptions

We test a lot of different ways. We test characters alone, self buffed and raid buffed. We do very controlled tests and we do tests where we just tell a skilled player to go nuts and we do actual raids. We get a lot of information from outside Blizzard as well, but you asked about how we test.

It's difficult to know exactly when a tank is going to end using their cooldowns. Sometimes it's probably fine to spam them and other times they need to be saved for a very specific situation. Which abilities you use is also determined by what classes are present in the rest of your group. That's just another reason to gather a large body of data.

We think the paladin and warrior are pretty close now. I do have some concerns about eventual scaling problems now that block is so good. Critical Block may end up letting warriors bypass paladins eventually, but that will be a few raid tiers from now, so we'll have plenty of time to evaluate the situation and make adjustments if necessary.

You're still going to have a hard time convincing me that 2% mitigation difference will determine which class everyone uses for their MT. Consider:

1) You're going to have a hard time even knowing what that number is. Unless a few really eloquent theorycrafters manage to convince the entire raiding community, most of the time you're going to have to rely on actual raid data to make decisions, and that kind of variance is going to be really hard to detect.

2) Every boss is different. Mitigation deltas may be miniscule on a slow-hitting boss but noticeable on a fast-hitting boss for instance.

3) A difference likes 2% assumes both tanks have the best gear possible for their slot. Gear can make a very big difference in mitigation, yet you tend to see guilds stick with their MT for a long time. You don't suddenly drop your guy when someone with a better shield and 10% more mitigation comes along. Why? Because your guy's loyalty, dependability, knowledge or sense of humor is more important than that 10% difference.

4) Don't forget player skill has a huge role in here too. I've done Archimonde with an MT that couldn't stance dance to save his life. But despite that er... handicap, the group is on M'uru or something now. Yeah I know, everyone has antecdotes. The moral of my story is that if a tank who can't use his cooldowns can make it to Sunwell, surely the tank with 2% less mit can. If min-maxing tumped all, they would have dumped him.

5) Paladins and druid tanks are already tanking a lot of content in BC, and that's in spite of the current design (i.e. the BC version, not the LK one) that warriors are the best single boss tanks. And, in spite of things like crushing blows. And, in spite of the difficulties druids and paladins had getting gear. If they can tank already when they're supposed to be OTs, then they should probably be great when we are actively trying to make them MTs.

Ghostcrawler

2008-10-03 10:05:12

Re: GC: Tanking Testing Assumptions

I am aware of the sites you mention and the various threads across the Internet. I am in communication with great tanks all the time. One of the problems we face is that each tank tends to conclude they are the worst -- they're nervous the community will fixate on one as the best, which is totally understandable. The numbers in WoW are very complicated -- if they were simple, much of the depth and theorycrafting wouldn't exist. Unfortunately, that also means it's easy to look at data and come to different conclusions, or even generate different data. Procs, reactionary abilities and cooldowns can change things dramatically. You can't assume they're always up but you can't ignore them either.

I think the BoSanc argument does hold some water, and that's something we're going to look at. I'm specifically refering to the case where the paladin is the eternal OT because he can also buff the warrior MT.

Q u o t e:
I don't understand. Are you saying that warriors and paladins are so close that you cannot determine a difference in mitigation/effective health? Or are you saying that you know warriors are 2% better but are choosing not to fix it? If it's the latter, then it makes no sense to me.

I'm saying the former. We aren't designing the paladin to be purposely inferior. But because their abilities are so different, I'm also not naive enough to say that their survivability will be the same in all situations. Promising 100% the same stats with the same gear for every fight would be silly. I'm promising it will be close enough that you can use any of the 4 classes as your MT (assuming they're a good player).

Q u o t e:
As long as it's clear that the number exists and the Paladin is inferior, it's a problem.

I totally agree with this. But things are rarely "clear" in this game. Is it clear what class did the best dps in BC? Was it clear what class was the best healer? If you change small assumptions in your calculations (maybe a proc isn't up as much as you think) then you get different numbers. On the other hand, I think the numbers *were* pretty clear that warriors were better tanks than paladins in most of end-game BC, but despite that, there were still a lot of paladin tanks.

Q u o t e:
GC thats the problem. Since it took till now to make us viable we need the edge to compete for MT. All these guilds on all the realms already have a MT chosen. I happen to be lucky enough to be my guilds MT already. For others it wont be so easy. They will remain OT to there guilds already branded MT. Our class just doesn't stand on level ground to claim it.

I agree the WoW community can be surprisingly conservative in adapting to new paradigms. But that's not a great reason to give anyone the edge. It didn't take very long at all for players to realize that Protadins were head and shoulders above other tanks for running 5-mans, especially say Shattered Halls. Shadow Priests went from lol to mandatory in between Naxx and Karazhan. Some guilds will keep a warrior as their MT because they like that player in that role. But not all will. I wouldn't be surprised at all to see a lot of changes in guild structure (it happened in BC) because the raiding landscape is so different now. You can choose to skip 25-player content completely and still go against the Lich King. A lot of players assume that the perception will be the 25s where the big boys play and the 10s are the kiddie pool, but I'm not so sure that's going to happen.

Ghostcrawler

2008-10-03 13:51:36

Re: GC: Tanking Testing Assumptions

I still think you guys are putting too much faith in your theorycrafting. There is just no way to produce precise and accurate numbers down to fractions of a percent. You have to make assumptions about e.g. how often Shield Block is up, and effects of things like Ardent Defender, and rate of incoming damage (which varies hugely from boss to boss). Change any of those assumptions and the results change.

Good math is essential in theorycrafting, but so are good statistics. When you are coming up with estimates like this, what you are really coming up with is a number plus or minus some amount of variance. In this case (and in most theorycrafting experiments) the variance is very large. Imagine your result is actually something like "Our calculations suggest that paladin mitigation is 3% lower than warrior mitigation, plus or minus 10%." In other words, it could be anywhere from 13% low to 7% high. I'm not doubting your math here. I'm just saying don't be so convinced that your calculations reflect reality. That's why we call it theorycrafting and not fact. Make sense?

(By the way, all of this applies to us too. We have the resources to do some very extensive testing, but sometimes the results turn out different when the changes go live. Players figure out a different rotation that changes everything. Or some obscure trinket or set bonus we forgot about makes a mediocre ability crazy good.)

We do test with and without Blessing of Sanctuary, and with and without lots of other raid buffs. It is a big puzzle with a lot of variables. :)

We like Blessing of Sanctuary as a buff you can put on anyone. However, we might do something like let it benefit the paladin more than other targets. (Self buffing gives a better result than buffing someone else.) Or we might just add more paladin mitigation somewhere else in the tree. That all assumes you are deficient. If 2-5% makes a big difference for you (I don't think it will make that much of a difference, but for sake of argument), then buffing paladins 2-5% when they didn't need it could also make the other tanks non-starters. We'll keep testing it.

Ghostcrawler

2008-10-03 17:43:02

Re: GC: Tanking Testing Assumptions

Okay, here is one change you don't have.

Shield of the Templar now also reduces all damage taken by 1/2/3%.

Critical Block is something we're still keeping our eye on. I'm not going to promise a paladin version, but it's totally something we'll consider if paladin blocking starts to fall behind.

Whoever said it, yes it's absolutely going to be a challenge trying to get parity among 4 classes with totally different mechanics, and in some sense the warrior to paladin comparison is the easy one. :) But we don't like having only one MT class, and we don't like giving everyone very similar gear, talents and abilities to tank, and we don't like the situation where you need a different tank for every fight because they all excell at certain types of encounters and suck at others. So the option we're left with is to have 4 who are different but relatively equal.

Ghostcrawler