As he waits for his Dungeon Master's Guide to arrive Surf shows how he assesses monster CR...
Part 10: CR Evaluation
I want to open this instalment of the series by apologising to everyone who has been waiting for it. I sit here waiting for my Dungeon Master's Guide to arrive and reflect on what a slippery beast CR has been to nail - I have have been chasing it relentlessly since my previous article. There are many ways it can be assessed and I have designed and evaluated no less than four separate CR assessment methods during this time. Each has their pros and cons, but none on its own quite manages to cover assessment of all monsters to date. The method presented here is the most reliable of the crop.
The development of a reliable CR assessment method has been a priority of mine since well before the release of D&D 5e. In fact you might say it was the catalyst for my original D&D Next monster analysis series, which ultimately that had enough impetus and interest to see the series reborn once D&D 5e was released.
On the journey from playtest to release there have been a few hints about CR calculation from Wizards of the Coast. To me key amongst these has been the following...
- Mike Mearls tweeted that CR "...calculation includes both offense (atks, dmg) and defense (AC, hp)".
- In Legends & Lore 07/07/2014 Mike tells us a bit about the relationship between CR and XP.
- The Basic D&D DM PDF v1 gives some advice on encounter building, touching on CR.
- In D&D Next Q&A: 04/25/2014, Rodney Thompson told us that we'll calculate CR "...by comparing the villain/monster that you've designed to a set of baseline values to determine its CR".
We know the Dungeon Master's Guide will contain information on monster creation and evaluation, but we don't know just how much or what its nature will be. We don't know how mathematically focussed and "scientific" it will be or how much it will rely on the assessor's judgement. Our hopes may be high, but there are a lot of unknowns.
Based on our dissection of D&D 5e monster math we should now in a reasonable position to build a method for assessing CR, one that stands on it's own and can be of ongoing value. We do need to note that this method is only as good as the data on which it is based and that some of the specifics are necessarily subject to revision as more data and information come to light. The completion of our Monster Manual analysis and the release of the Dungeon Master's Guide are two key events which are likely to trigger such revision.
The method presented below uses a derived Challenge Index to assess CR. First we look at how an initial Challenge Index Baseline can be determined. Then we will look at Challenge Index assessment and its two major components, the Defensive Index and the Offensive Index; along with the standard adjustments used during these calculations. Finally we review our baseline to produce a final Challenge Index Progression and reflect on the accuracy of this method. To close off the article we will check how our Example monster measures up.
Challenge Index Baseline
In order to assess monster CR we need something to measure monsters against. The more parameters we have to use directly in the comparison the more difficult the assessment becomes, using a single number is a more natural and elegant approach. In statistics an "index" is a composite number that aggregates multiple indicators in order to summarise and rank them. It's a way of bringing together multiple statistics for assessment. And that's a very good fit for what we want to do with Challenge Rating. In this case we can score defense and offense separately and combine them into a Challenge Index.
Before we get into how we determine a Challenge Index for an individual monster we need to determine the baseline Challenge Index progression. We can approach this in a number of ways, however a simple and elegant method is best when all else is equal. If our assessment method deals with many of the variables by their deviation from what is standard for that CR then we can assume that those are correct in our base progression. This means in our baseline they become a constant of "1", for multiplication and division operations, or "0", for addition and subtraction. That means we are able to simply ignore them and focus on Hit Points and Damage for our progression baseline, along with the weight each of those might have.
This would mean our baseline Challenge Index can be determined thus...
CI_Progression=(CR_Hit_Points × Defensive_Weight) + (CR_Damage × Offensive_Weight)
This approach held up under preliminary analysis and testing, allowing effort to move on to evaluating all the creatures we have assessed to date. Of course this became an incremental and iterative process - make a pass over the monsters, assess the results, tweak our base progression, review the results, rinse and repeat. Numerous theories were tested, many discarded and some stayed for refinement, but such is life building an index up on data that isn't fully known. Slowly a method of evaluating actual CR, incorporating all defensive and offensive elements while demonstrating an acceptable level of accuracy, came into focus.
D&D 5e monsters generally have Hit Points much higher than their Damage and this suggests that the Offensive Weighting would likely be at 1, or 100%, meaning we could omit it altogether. Although it did in fact work out this way the Offensive Weight remains in my spreadsheets to this day - it proved useful on occasion and I needed to prove that the Offensive Weighting could remain at 1. Defensive Weighting was clearly a value less than one - some fraction to bring its weight for evaluation purposes into line with offense. A number of passes over the data included reevaluating the value of this constant. Interestingly there were several candidates that kept resurfacing, but in the end 0.5, or 50%, was clearly the best match.
By graphing these derived numbers we can calculate a formula close to the resulting progression. And we see that the progression is similar to the following linear formula...
CI_Progression=16 × CR + 5.2
This will not be our final progression table, as we'll revisit it once we have assessed all of our sample monsters. But it is a good starting point.
Prediction: TODO:
Challenge Index Assessment
Once we have a baseline challenge index chart to work from we need to understand how we should assess individual monsters. Indeed, to prove our progression chart we need to evaluate all of the monsters in our sample data and achieve a high level of accuracy with our results to feel confident using this table moving forward.
The high level process for determining challenge index is fairly simple, gaining complexity as we drill into its components. Because the components of challenge index are fairly self-contained (the braces mean they are essentially two separate numbers added togethor) we can express the calculation itself as...
| Challenge_Index=Offensive_Index + Defensive_Index |
Since this process becomes moderately complex it seems wise to lay it out with a step-by-step plan that readers can follow. Those with an appetite for math and formulae need not fear, as these are also supplied at the start of each section...
- Take note of the monster's target CR, or intended CR. This is used at several points during the evaluation process to determine how a monster varies from "standard" for a given statistic. If it's a monster you are building yourself you'll know the target CR and if it's a published monster it will have a listed CR.
If you don't know the monster's target CR you'll need to work it out roughly from the monster's Proficiency Bonus. The easiest way to do this is if the monster has a saving throw - simply subtract the appropriate ability modifier from that save. Alternatively you can work out the Proficiency Bonus from the creature's skills or Actions. Once you know the Proficiency Bonus check page 8 of the Monster Manual or page 4 of DMDnDBasicRules_v0.1.pdf. This will give you a range of CRs with that Proficiency Bonus. Pick one of these that seems appropriate, or simply one from the middle of the range.
- Calculate the monster's Defensive Index.
- Calculate the monster's Offensive Index.
- Add the defensive and offensive indexes together to obtain the challenge index.
- Check the challenge index against the chart in the Challenge Index Progression section. If it is outside the boundaries for the target CR we may have to redo our calculation against a higher or lower CR.
Defensive Index
A monster's defensive index is a moderately complex appearing formula that incorporates a creature's various defensive capabilities and can be expressed as...
| Defensive_Index=(Monster_HP + Monster_HPAdj) × | |
| ((Monster_AC + Monster_ACAdj + Monster_MS - CR_StdAC - CR_StdMS) × Unit_Val + 1) × | |
| Defense_Weight | |
In reality this isn't too complicated to work out and lends itself well to a process. About half of this formula deals with calculating the Defense Multiplier and breaking this out into a separate precursor formula helps clarify things somewhat...
| Defense_Multiplier=(Monster_AC + Monster_ACAdj + Monster_MS - CR_AC - CR_MS) × Unit_Val + 1 |
| Defensive_Index=(Monster_HP + Monster_HPAdj) × Defense_Multiplier × Defense_Weight |
Let's have a look at the variables and constants that make up these formulae...
| Monster_AC | The listed Armor Class of the monster being evaluated. |
| Monster_ACAdj | A positive or negative notional adjustment to the monster's AC for Traits, Reactions and other elements not easily factored directly into that monster's base defences. For example a monster with the Camoflage Trait has a +1 here. Some common standard adjustment values can be found in the Standard Adjustments section. |
| Monster_MS | The count of the monster's miscellaneous stats. Simply sum up the number of Saving Throws, Skills, Damage Resistances and Damage Immunities that the monster has. |
| CR_AC | The average AC for creatures of the target CR (from here). |
| CR_MS | The average number of Miscellaneous Stats for the target CR (from here). |
| Unit_Val | This is a constant value denoting what a single increment or decrement to values like AC and To-hit are "worth". This should always be 0.05, or 5%. |
| Monster_HP | The listed Hit Points of the monster being evaluated. |
| Monster_HPAdj | A positive or negative notional adjustment to the monster's Hit Points to allow for various aspects of monsters not factored into that monster's base HP pool. For example a monster with the Relentless Trait has a +5 here. Some common standard adjustment values can be found in the Standard Adjustments section. |
| Defense_Weight | A multiplier used to scale the Defensive Index up to an appropriate level for inclusion in the Challenge Index. This should be set to 0.5, or 50%. |
For those less mathematically inclined here is a step-by-step description of how to calculate a monster's Defensive Index...
- Determine the monster's Defense Multiplier...
- Note the monster's AC.
- Add any AC adjustments to the monster's AC. This can require some judgement on the part of the person assessing CR. Some known adjustments can be found here.
- Add the number of Saving Throws, Skills, Damage Resistances and Damage Immunities that the monster has.
- Look up the standard Armor Class for the target CR here and subtract that value.
- Look up the average Misc Stat sco5e for the CR here and subtract that value.
- Multiply the result by 0.05.
- Add 1 to obtain the monsters Defense Multiplier.
- Note the monster's average Hit Points.
- Add any adjustments to the HP. This can require some judgement on the part of the person assessing CR. Some known adjustments can be found here.
- Multiply the result by the Defense Multiplier calculated at 1 above.
- Multiply the result by 0.5.
Offensive Index
A monster's Offensive Index is calculated in a manner similar to its Defensive Index, it's just a little simpler and source values from different aspects of a monster. The formula for doing this may be represented as...
| Offensive_Index=(Monster_Damage + Monster_DmgAdj) × | |
| ((Monster_Attack + Monster_AttAdj - CR_Attack) × Unit_Val + 1) | |
Note the absence of an Offense_Weght or similar. Offense is factored at 100%, which is the same as using an Offense_Weight of 1.0. This results in no change and so that constant may be omitted altogethor.
As with Defensive Index we can make Offensive Index easier to digest by splitting it into two parts - calculation of the multiplier and the remainder of the computation...
| Offense_Multiplier=(Monster_Attack + Monster_AttAdj - CR_Attack) × Unit_Val + 1 |
| Offensive_Index=(Monster_Damage + Monster_DmgAdj) × Offense_Multiplier |
The variables and constants that make up these formulae are...
| Monster_Attack | The listed To-hit value for the primary attacks of the monster being evaluated. Note that for Spellcasters this is generally their "to hit with spell attacks" value. |
| Monster_AttAdj | A positive or negative notional adjustment to the monster's To-hit value for Traits and other elements not easily factored directly into that monster's attack bonus. For example a monster with the Pack Tactics Trait has a +1 here. Some common standard adjustment values can be found in the Standard Adjustments section. |
| CR_Attack | The average Attack Bonus for creatures of the target CR (from here). |
| Unit_Val | This is a constant value denoting what a single increment or decrement to values like AC and To-hit are "worth". This should always be 0.05, or 5%. |
| Monster_Damage | The calculated average Damage of the monster being evaluated. Working this out can be an interesting and variable process. Much of the effort of monster construction and evaluation goes into this one aspect of D&D 5e monsters. Readers may find some help working this out in Part 7. |
| Monster_DmgAdj | A positive or negative notional adjustment to the monster's average Damage to allow for various aspects of monsters not factored into that monster's assessed average Damage. For example a monster with that inflicts a 5hp disease (e.g., the Werewolf) has a +3.5 here. Some common standard adjustment values can be found in the Standard Adjustments section. |
And the step-by-step process for calculating a monster's Offensive Index...
- Determine the monster's Offense Multiplier...
- Note the monster's To-hit value for its primary attacks.
- Add any adjustments to the monster's To-hit value. This can require some judgement on the part of the person assessing CR. Some known adjustments can be found here.
- Add the number of Saving Throws, Skills, Damage Resistances and Damage Immunities that the monster has.
- Look up the average Attack Bonus for the CR here and subtract that value.
- Multiply the result by 0.05.
- Add 1 to obtain the monsters Offense Multiplier.
- Work out the monster's average Damage. This can be an interesting journey all on its own. You may gain some assistance from Part 7.
- Add any adjustments to the Damage. This can require some judgement on the part of the person assessing CR. Some known adjustments can be found here.
- Multiply the result by the Offense Multiplier calculated at 1 above.
Standard Adjustments
Some of the adjustment values used in assessing CR are unique to that particular monster, or are somewhat open to interpretation, and others do not appear to impact CR whatsoever. But many seem to be static and standardised. These Traits, Reactions and other aspects of monsters are relatively easily enumerated, or at least appear to be derived from elsewhere in a fairly predictable manner.
The following list is not complete and should be considered a work in progress. However it is what I am using at this point in time and is shared in the hope that others will find it useful in evaluating monster CR...
| Antimagic Susceptibility | -5 HP | Keen Senses | +0.25 AC | |
| Blood Frenzy | +2 To-hit | Life Drain | +(Life Drain amount × 0.75) Damage | |
| Camoflage | +1 AC | Magic Resistance | +2.75 AC | |
| Disease (5hp) | +3.5 Damage | Pack Tactics | +1 To-hit | |
| Echolocation | -6 AC | Parry | +2.5 AC | |
| False Appearance | +1 AC | Rampage | +1 To-hit | |
| Flyby | +1 AC | Reckless | -4 AC, +4 To-hit | |
| Frightful Presence | +2 To-hit | Regeneration | +(Regeneration_Amount × 2) HP | |
| Gaze Attack | +2.5 AC | Relentless | +5 HP | |
| Incorporeal Movement | +1 AC | Sure Footed | +0.5 AC | |
| Invisibility | +2.5 AC | Vulnerability | -(Monster_HP × 0.2) HP |
Challenge Index Progression
After evaluating all monsters and iteratively refining our approach we end up with a set of results fairly close to our initial Challenge Index Baseline.
| CR | Challenge Index | ||
|---|---|---|---|
| Min | Target | Max | |
| 0 | 0 | 5 | 7 |
| ⅛ | 6 | 8 | 10 |
| ¼ | 9 | 11 | 15 |
| ½ | 13 | 17 | 21 |
| 1 | 19 | 23 | 32 |
| 2 | 29 | 38 | 48 |
| 3 | 44 | 54 | 63 |
| 4 | 60 | 69 | 78 |
| 5 | 75 | 84 | 94 |
| 6 | 90 | 100 | 109 |
| 7 | 106 | 115 | 124 |
| 8 | 121 | 130 | 139 |
| 9 | 136 | 145 | 155 |
| 10 | 151 | 161 | 170 |
What we see in the results is a data that pretty closely matches the following linear equation...
CI_Progression=15.3 × CR + 7.7
The results are a particularly good match for creatures of CR1 or higher, with over 90% of sample creatures evaluating to "within CR". Most monsters in this range that don't fall clearly within CR aren't too far from these CR boundaries.
Monsters lower than CR1 are less solid matches, with the apparent curve using this progression slicing in too high for most samples - the progression "as is" is just a bit too high for them. We can adjust easily for this by calculating the challenge index of monster below CR1 based on the CI of CR1. While this brings us much closer to target we find we are still a little out and we find that we need to apply a minor 0.22 adjustment to base CR to get the best match. This gives us...
CI_below_CR1=CR1_CI × (CR + 0.22)
With our target values thus determined all that remains is to determine an acceptable level of overlap between adjacent Challenge Indexes and to produce an appropriate table.
Assessment Accuracy
Many readers will wonder up-front how accurate this method is. Let us consider for now the creatures available before the release of the Monster Manual, which is what my spreadsheet contains at the time of writing...
- 129 monsters are of CR1 or higher and 117 of these fall within the boundaries of their intended CR.
That is a 90.7% match rate. - 95 monsters are below CR1 and 83 of these evaluate as appropriate for their CR.
The match rate for these is 87.4%. - That's a total of 224 monsters, of which 200 clearly evaluate to "within intended CR".
For an overall match rate of 89.3%.
This raises a number of questions... Why do some monsters not fall within CR? Is this accurate enough? Do we have any other concerns about this level of accuracy? How useful is this approach?
It's not difficult to achieve a higher level of match by tweaking the overlap between challenge index values. This is an approach I am loath to apply as it removes a great deal of clarity from the CR evaluation process. I would be appaled to see a first-tier RPG company apply such crude measures.
Instead I believe that we will find the answers to most of these questions in closer examination of the data...
1. Fuzzy Boundaries
I believe that CR progression is a scale without hard boundaries between adjacent CRs, this means that some overlap between monsters of adjacent CR is likely to occur. We see evidence supporting this in our sample data.
The spread of values for both Hit Points and Damage vary significantly, overlapping with the data spread of adjacent CRs. Indeed the overlap in these "bands" is not only with those of immediately adjacent CRs, but with bands two or more steps removed. This is visually obvious if we drill in on creatures from CR0 through CR10 on scatter graphs. The same phenomenon can also be seen with both AC and To-hit values, the ceiling on those is simply lower and the overlaps spread further across the CR spectrum.
That said we expect a viable CR evaluation method to yield an index which tightens this data up significantly. If we use the method in this article to assess the CR of all CR0 through CR10 monsters in our sample data and then produce a scatter graph of the resulting challenge indexes by CR, we see that this is indeed the case. But we also see that there is still some overlap between the bands of adjacent CRs.
Why? Because the boundaries are "fuzzy" and do overlap.
Consider, regardless of the evaluation method, a scenario where the upper boundary for CR4 is Challenge Index 75 (or CI75) and where we have two similar monsters, one that evaluates to CI75 and another that evaluates to CI76. The CI76 creature simply has one HP more than its CI75 counterpart. Now, in terms of game experience, what is the difference between the two creatures? The answer is "almost nothing"! It's not noticeably harder to kill a monster with 101 HP than the same monster with 100 HP.
2. Judgement Calls
During the process of assessing any given monster there are frequently small judgement calls that need to be made. What is this effect "worth"? How much does that trait impact AC? While these can contribute small amounts of variation to a monster's final CI these are not the decisions I am talking about here. These judgement calls have a broader scope than that.
Consideration of the presence and nature of CR assessment's fuzzy boundries leads to some interesting conclusions...
- There are monsters that are "weak for their CR", "average for their CR" and "strong for their CR". Even when the difference is not pronounced it does still seem to be present.
- The difference between a "High CR8" and a "Low CR9" monster can be very negligible. In fact it can be close to nothing.
- Where a monster sits squarely in the border area between two adjacent CR bands, what do we do? We can go back and adjust it until it clearly falls into one CR or the other. But the data suggests that's not what Wizards of the Coast generally do. Instead it appears that someone has made a judgement call about whether the monster in question should be a "Strong" creature of the lower CR or a "Weak" monster of the higher CR.
Do these suppositions have any merit? What signs can we see in our data that this has been done? Well quite a few, to be honest!
Take a look at the Young Green Dragon in the Basic D&D DM PDF. This monster is CR8, has 136HP, AC18, +7 To-hit and does an average 81.83 Damage.
Now compare it to the Bone Devil from the Monster Manual spoilers. This CR9 monster has 142HP, AC19, +8 To-hit and inflicts an average 46.00 Damage.
Sure the Bone Devil has 6 extra Hit Points, one extra AC and one extra To-hit. But it does barely more than half the damage of the Young Green Dragon. The general level of saves, resistances, immunities and other attributes are close enough between the two that we can treat them as roughly equivalent in those areas. And yet it's difficult to see the CR9 Bone Devil as a significantly tougher fight than the lower-CR Young Green Dragon!
Another significant comparison is the Chimera (CR6, 114hp, AC14, +6 to-hit, 67.42 damage) and the Stone Giant (CR7, 126hp, AC17, +9 to-hit, 38.00 damage). Both of these creatures are quite simple to play and to evaluate. But again it's a bit of a stretch to see the Stone Giant as a significantly tougher fight than the Chimera.
There are a number of other similar examples in our sample data.
Some monsters are clearly stronger than monsters in the next CR up and some are weaker than those in the next CR down.
That doesn't happen when there's a mathematically sound evaluation method, with firm boundaries between CRs, which requires minimal human intervention. Now I personally am pretty confident that the method we've used is mathematically sound so I can only conclude that there are times in D&D 5e monster CR assessment when a human judgement call is made and that this is most likely to occur with monsters that would otherwise be borderline.
3. Granularity At Low CR
A difference of one Hit Point for a CR30 creature is very little. But that same 1 HP difference is far more significant for a CR0 monster. The same holds true for damage and for many effects, Traits and other aspects of monsters. In the same way small differences in CI at the lowest CR bands is far more significant that at the highest CR bands.
As a result monsters of CR1 or lower can be amongst the most difficult to evaluate. For example the Thug (CR½, 32hp, AC11, +4 to-hit, 10 damage) and the Lion (CR1, 26hp, AC12, +5 to-hit, 10 damage) can be difficult to categorise with casual examination. One won't necessarily understand immediately that a lone Lion can quite easily take a level 1 PC down to 0 HP in a single round, due to it's high mobility, Pounce and the high proportion of it's attack with advantage (due to prone and Pack tactics). While a single Thug doesn't enjoy the same level of accuracy and will be hitting far less often.
Well informed judgement calls become more important at the lowest bands of CR due to this granularity issue.
Example: Human Pyromancer
OK so let's step through the process and see how our Human Pyromancer shapes up, based on this method!
Human Pyromancer | |||||
| Medium humanoid (human), any alignment | |||||
| Armor Class 12 | |||||
| Hit Points 77 (14d8+14) | |||||
| Speed 30ft | |||||
| STR 9 (-1) | DEX 14 (+2) | CON 12 (+1) | INT 19 (+4) | WIS 12 (+1) | CHA 12 (+1) |
| Saving Throws Dex +5, Int +7 | |||||
| Skills Arcana +7, Perception +4 | |||||
| Damage Resistances Fire | |||||
| Languages Common, Ignan | |||||
| Challenge 5 (1,700 XP) | |||||
|
Spellcasting. The pyromancer is a 5th-level spellcaster. Its spellcasting ability is Intelligence (spell save DC15, +7 to hit with spell attacks). The pyromancer has the following wizard spells prepared. Cantrips (at will): dancing lights, fire bolt 1st level (3 slots): burning hands, disguise self 2nd level (2 slots): invisibility, scorching ray 3rd level (1 slot): fireball | |||||
|
Actions Dagger. Melee Weapon Attack: +5 to hit, reach 5 ft., one target. Hit: 4 (1d4 + 2) piercing damage. | |||||
|
Reactions Flame Shroud (Recharge 5-6). When a ranged attack would hit the pyromancer, or it is targeted by a spell that has a range other than touch, the air around it bursts into shimmering, shifting flame for a few seconds. Until the start of the pyromancer's next turn ranged attacks against it, including the triggering attack, are made with Disadvantage and it has Advantage on saving throws against spells with a range other than touch. During this time it takes no damage from magic missile. | |||||
First up we calculate the Defensive Multipler. The only real trick here is estimating the impact of the Flame Shroud Reaction. Now this should kick in at least once per fight, effectively giving the Pyromancer a +5 to AC against ranged attacks for one round. Averaged out across an entire fight that's going to work out to somewhere between +1 and +2 to AC, let's be optimistic for now and call it a +2. The remaining values are readily apparent or accessible via the links provided in their section, which yields the following...
Defense_Multiplier=(12+1+5-14-5)×0.05+1=1.0
The Defensive Index can now be calculated trivially...
Defensive_Index=(77+0)×1.0×0.5=38.5
When computing the Offense Multiplier we need to ensure we use the To hit bonus from the Spellcasting Trait, but this task is otherwise straightforward...
Offense_Multiplier=(7+0-6)×0.05+1=1.05
The remainder of the Offensive Index can now be calculated pretty easily once we have referred to the example monster section of the damage analysis in this series...
Offensive_Index=(49+0)×1.05=51.45
And now we simply add these two values togethor to gain the overall Challenge Index...
Challenge Index=38.5+51.45=89.95≈90
Based on the chart above a CI of 90 is just a little above the target for CR5. This tells us that this particular monster should be a slightly stronger than average CR5 opponent.
In then near future I'll combine all of the tables publish to date into a single consolidated post. So stay tuned!
Part 9: Attack Bonus
Master Index
