Regional Flag@Sigma: re AMA and on the topic of scaling.Source
avatar
Pandanaconda
Target Source
#1 - 2017/07/10 01:44:00 PM
In the most recent reddit AMA, an interesting discussion ended up popping late; not because it was unimportant but rather, I believe, because you were trying to answer it properly. I will adress this in particular to sigma and quote you a lot, to begin with by this:

It's hard to promise going on in this level of detail very often (even though it's a sort of discussion we enjoy a lot--probably why we do this job).


I hope to provide enough for you to enjoy your job here. The topic was class scaling, and wether or not looking at the data/simulations were an accurate way to assert either.Although this will again be centred on Windwalkers, I will try to make most things generic enough than a lot of it could be applied to many specs. I will start by saying that as you said you would, I will put on my scientist hat, which I don’t really take off, and avoid logical missteps. I’ll avoid assuming that generic windwalker guides provide perfect information for every scenario, that players are playing perfectly at any level, that simulations are an accurate representation of gameplay performance, that public logs are a full picture of the game, I’ll avoid forgetting about windwalker’s aura buffs to other players (master of combinations and windwalking.) However I know you’ll find mistakes in my reasoning anyways, because this is such a volatile matter it is hard to treat.And to begin with, I’ll go through your last answer, and state where I disagree partially or totally.

1) Possibly--the data shows that the WCL "normalized score" decays within a tier. At least--it possibly does, and very slightly. Looking at your Nighthold 7.2 link (the most recent/relevant one as it reflects the most current design), your downward-sloping linear fit is not very pronounced, and does not have very high slope (it's only the compressed y-axis that makes it look steep). Looking for example at the same chart with all classes included: http://imgur.com/a/RDLxB , the Windwalker line (you can make it out if you look closely) does not perceptibly change relative to the other specs.


It is very interesting to look at the relative slope of windwalker’s performance compared to other specs. However, as months go by, gear increase is obviously not the only thing affecting the data sample in logs. The population who has access to clearing content increases, just like the fact that more and more “alts” are able to clear. This has an effect on the data. Also, the way encounters are dealt with changes drastically. But also, mid tier patches happen with somewhat relevant changes. Windwalker has for instance been subject to regular “band-aid” buffs mid tiers. These skew the data: for sure wind walker barely performed worse as tier 19 went on, but it received a buff to it’s abilities mid patch, and allthough the two most popular legendaries saw their effects reduced, one of them saw such a substantial change (the emperor’s capacitor) that it’s use became overwhelming in the following patch. Just the same, 7.2 unlocked new traits, modifying the relative balance of specs greatly. Split personnality is a trait that affects windwalker’s behavior so much we cannot look at the data and ignore these changes.As I stated, as a tier goes on, encounters are dealt with in a different fashion. AoE becomes less relevant as fights get shorter usually: for instance, if a fight has regular spawns of adds, the shorter the fight, the lesser the amount of extra health you have to deal with. Spellblade Aluriel’s health did not vary between week one and 7.2.5. However, we went from killing 6 waves of adds on mythic difficulty to only 3: if the fight got 50% shorter, the adds probably lived 75% shorter. This is obviously something you’re aware of, and a big reasons classes you force to rely on cleave behave worse and worse as time goes by. Just the same, bloodlust uptime goes up as patch lasts longer and fights become shorter. This historically has not benefited windwalkers as much as others. Just the same, I could show this kind of factors usually make shadow priests fall out of flavour and Arcane mages perform better. A bit of a simplification but you’ll find some truth in this I’m sure.

I will also admit the most recent iteration of wind walker with tier 20 and the set up we have been running at the highest level of performances has way less of these issues to some extent: with tier 20 4pieces bonus, we are way stronger than before in shorter fights with higher bloodlust uptime, and less reliant on adds to perform well as our AoE isn’t as strong as before on 2-4 targets.

avatar
Game Designer
Target Source
#7 - 2017/07/22 07:40:00 AM
10/07/2017 14:56Posted by Pandanaconda
http://www.peakofserenity.com/2017/07/09/windwalker-scaling


I don't know if there's a better thread where this is being discussed, but this thread seems closely related to the work done in the post.

Restating something I'd already said to the author--stat scaling is a massively complicated problem, and basically any rigorous knowledge fed in about it is a helpful addition. The main impediment to discussion on such a large dataset being offered as feedback is having to go through the whole methodology--enough to be sure we're all on the same page as to exactly what it's showing. I expect something as ambitious as "make a chart of all stats for all DPS specs at once" is likely to wind up as a project of perpetual refinement rather than something you can just "do" one day.

Which is all to say: basically all I can do is start with scattered questions/comments as I go through, rather than any kind of grand conclusion.

----

The best chart to spent most time on is probably the "% DPS gained per 475 stats" one.
--Versatility clocks in at 1.00% (+/- 0.01) on nearly all specs. Does that mean that the profiles being used have 0 Versatility?
--More broadly, what "current" stats are being used to generate the weights? As explained above, that affects how stats look relative to each other when viewed as a % of total DPS. If the profiles aren't constructed in a uniform way, some might have more lopsided stat distributions than others.
--Given how few things remain that are left out of Versatility, it might be cleanest to bring them all in at this point. No promise yet.
--For specs that are significantly above 1.00%, are they double-dipping Versatility or is that a simulation artifact?

"I didn’t include Weapon DPS in the averages as there was no easy way to do so and be fair to all specs. If you come up with a way that works, let me know."
--I believe the correct way is to make a weighted sum of Primary and WDPS based on whatever ratio they exist at currently on typical gear (similar to the advice I gave for Stamina).
--The variation in primary stats is a bit curious--looking at casters, where there's no WDPS complication. If, on a caster, all DPS other than trinkets is proportional to Int, the value of Int on all of them should be nearly identical (again after correcting for special cases such as the portion of Demonology damage that scales with Stam). Maybe what I'm saying is: the variation in Int's value seems like a sort of "control"--one way of getting insight into the magnitude of systematic error in the model.
--Trinkets also throw a minor wrench in the works if they're not treated uniformly in the comparison. It seems like you'd have to either avoid RPPM-damage-proc trinkets on everybody (those don't scale with primary), or add another factor representing trinket ilvl and fold it into primary just as with Stam and WDPS.