Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)
Showing the Math: Proving that 4e’s Skill Challenge system is broken.
In this thread: http://www.enworld.org/showthread.php?p=4278599#post4278599
I have created a new skill challenge system to try and correct the problems with the existing one. I was kindly asked to prove that the original system is flawed, and I am happy to do so.
This thread is very math heavy. I assume you are proficient with basic probability and have the ability to understand combinations.
The Scenario: We will use a party of 1st level adventurers, and give them a Complexity 1 (Success 4 vs Failure 2), Level 1 task. Basically the easiest skill challenge such a party could face. I will assume a skill challenge where all of the DCs are medium, as this is the standard default for challenges. Then I will assume every player gets to use one of his best skills each time. For most characters, that’s probably +9 (+4 from a stat, +5 for skill training). However, that gives me a 50% success rate, and a 50% failure rate. That can get confusing in math equations, so I will give the player’s another +1 advantage, making it 55% success and 45% failure. The question is, how likely is a party to succeed at this skill challenge?
The Math: Conditional vs. Non-conditional Probability
There are two main ways to calculate the odds of this scenario. One is conditional. That means I take into account that fact that once I hit two failures, the skill challenge is over. That could happen in as little as 2 rolls, it could happen in as many as 5 (if I’m rolling a 6th roll, I would have already had either 2 failures or 4 successes, so 6 rolls can NEVER happen).
The other way is non-conditional. What I can say is I’m not going to look at when the failures happen, I am simply going to look at ALL the scenarios in which the party would beat the skill challenge, and discard scenarios where they would fail. This method is probably less intuitive right now, but I find the math easier to do, so I will choose this method. I will walk you through it as I go, and it should make sense.
Calculating the Odds:
What we are going to do is look at ALL the scenarios in which the party could win this skill challenge. The first thing to note is that the maximum number of rolls that can occur in this scenario is 5. At 5 rolls, I have already determined if the scenario is a failure or a success, every time. For example, if I had 3 successes I must have had 2 failures and the challenge has failed. If I have 4 successes and 1 failure, then the challenge was won.
What I will assume is that every skill challenge is a series of 5 rolls, and I will add up ALL of the scenarios in which the party gains a success. The reason I can do this: Because I’m looking at only the winning scenarios, the maximum number of failures I can have is 1. After all, if I have 2 failures, I would have had to fail the challenge somewhere along the way through the 5 rolls. Therefore, I don’t care when the failure happens. I could happen in the 1st roll, or the 5th. As long as I have my 4 successes in their somewhere, I will count it as a win for the party.
So in our series of 5 rolls, there are two ways in which the party can win the skill challenge:
1) 4 successes and 1 failure.
2) 5 successes and 0 failures.
We must calculate the odds of both and add them both together to get the total party success.
4 Successes and 1 Failure
First, we calculate the odds of this happening once. From our scenario above, the party has a +10 to their rolls, and they are rolling against DC 20 (which is given in the DMG pg 42). That means they have a 55% chance to make that DC 20 every time. That means they will also have a 45% failure each time.
So the probability of getting 4 successes and 1 failure is: .55 * .55 * .55 * .55 * .45 = .041.
Now we have a single probability, but we have to account for all the scenarios in which 4 successes and 1 failure can occur. For example, you could have SSSSF or FSSSS or SFSSS or SSFSS or SSSFS. We will use a combination to calculate this, because the position of the successes and failures don’t matter.
That’s C(5 rolls, 4 desired) = 5. So we will multiply our initial probably .041 * 5 = .205.
5 Successes and 0 Failures
Initial Probability: Now its .55*.55*.55*.55*.55 = .55^5 = .0503.
Our combination is C(5rolls, 5 desired) = 1. 1 * .0503 = .0503
Total Outcome: Adding both of these scenarios together gives us: .2508 = 25.08% win rate. So basically with my party taking on the easiest possible challenge, they have only a 1 in 4 chance of succeeding. And remember, I gave them an extra bonus!! If I use my original example (+9 to each roll) they only have an 18.75% chance.
Calculating Higher Complexities
So I have shown the math for the most basic challenge. But often you are going to want complexity 5 challenges, which is supposed to be the skill challenge equivalent of a battle. A battle against basic level 1 guys doesn’t sound too bad right?
I’m not going to go into the full math again, but I will lay out the ground work so you can do it yourself.
We need 12 successes before 6 failures. That means at most we can have 17 rolls (Notice how this number is always one less than the total between successes and failures, that never changes). Out of all of our 17 roll scenarios, which ones will give us success?
The answer, 12 successes and 5 failures, 13 successes and 4 failures, 14 successes and 3 failures, 15 successes and 2 failures, 16 successes and 1 failure, and 17 successes and 0 failures. Phew!! A lot of math to do. I will run the first scenario, and leave it to you all to finish the rest. I will of course provide the final answer!
12 successes and 5 failures.
Our individual probability is .55^12 * .45^5 = 1.41E-05 = .000014
Our total probability for this scenario is C(17 rolls, 12 desired) = 6188. 6188 * .000014 = .08749.
So now would repeat the process for the other scenarios. Doing that our total probability is: 14.71%, which is about 1 in 7. Again, that’s using that extra bonus I added, what would it be if it was a straight up 50/50 roll each time? 7.17%, about 1 in 14.
As a DM, do you feel that giving your party a 1 in 14 shot of handling even a basic challenge is fair?
Taking it to the Next Level. Well we’ve seen that a 1st level party can’t handle its own challenges, so let’s bring in our boys from 4th level to do the job right. Those guys have a massive +2 bonus to all skills to help them out, so a total of +11 (I don’t think these guys need that freebie bonus, afterall, these challenge is literally beneath them in difficulty!)
4th level party vs 1st level, Complexity 1 Challenge: 33.70%
4th level party vs 1st level, Complexity 5 Challenge: 26.39%
Um…maybe that party needs that freebie bonus afterall?
How about 6th level for another +1?
6th level party vs 1st level, Complexity 1 Challenge: 42.84%
6th level party vs 1st level, Complexity 5 Challenge: 41.97%
Okay okay, enough of this! Our 8th level guys come in with another +1 and their stats have been bumped again, so let’s give them a net +2 over our 6th level team.
8th level party vs 1st level, Complexity 1 Challenge: 63.28%
8th level party vs 1st level, Complexity 5 Challenge: 76.53%
Your going to look at the 8th level numbers and think it’s a typo…its not. That’s a result of how the complexity numbers are laid out. Once you get to 70% chance of succeeding on each individual roll, higher complexities actually give you a higher chance to succeed.
So…we have finally gotten a better than 50/50 chance to beat the challenge, and it only took a party 5-7 levels higher to do it.
Conclusion
I hope these examples have given some concrete proof that skill challenges are flawed as written. If you have any skepticism left, remember that I gave the party the advantage; I assumed they could use their best skills for the job. Take my original example, assume a 5 person party, and let’s say that one character only has a +7 to his skills for that challenge. The result against a complexity 1 challenge? 16.35%. Against Complexity 5? 5.17%. Better roll a 20 on that d20, because that’s all the shot you have.
However, there is hope. I have taken the time to reconstruct the model and fix the numbers. That system is in the thread above. I hope this article has been a good eye opener for everyone, and I hope that if you don’t choose to use my system you do find a new system that works for you. While the math of the skill challenge system doesn’t work, the concept is very good, and I would hate if any group gave up on it due to numbers alone.
Showing the Math: Proving that 4e’s Skill Challenge system is broken.
In this thread: http://www.enworld.org/showthread.php?p=4278599#post4278599
I have created a new skill challenge system to try and correct the problems with the existing one. I was kindly asked to prove that the original system is flawed, and I am happy to do so.
This thread is very math heavy. I assume you are proficient with basic probability and have the ability to understand combinations.
The Scenario: We will use a party of 1st level adventurers, and give them a Complexity 1 (Success 4 vs Failure 2), Level 1 task. Basically the easiest skill challenge such a party could face. I will assume a skill challenge where all of the DCs are medium, as this is the standard default for challenges. Then I will assume every player gets to use one of his best skills each time. For most characters, that’s probably +9 (+4 from a stat, +5 for skill training). However, that gives me a 50% success rate, and a 50% failure rate. That can get confusing in math equations, so I will give the player’s another +1 advantage, making it 55% success and 45% failure. The question is, how likely is a party to succeed at this skill challenge?
The Math: Conditional vs. Non-conditional Probability
There are two main ways to calculate the odds of this scenario. One is conditional. That means I take into account that fact that once I hit two failures, the skill challenge is over. That could happen in as little as 2 rolls, it could happen in as many as 5 (if I’m rolling a 6th roll, I would have already had either 2 failures or 4 successes, so 6 rolls can NEVER happen).
The other way is non-conditional. What I can say is I’m not going to look at when the failures happen, I am simply going to look at ALL the scenarios in which the party would beat the skill challenge, and discard scenarios where they would fail. This method is probably less intuitive right now, but I find the math easier to do, so I will choose this method. I will walk you through it as I go, and it should make sense.
Calculating the Odds:
What we are going to do is look at ALL the scenarios in which the party could win this skill challenge. The first thing to note is that the maximum number of rolls that can occur in this scenario is 5. At 5 rolls, I have already determined if the scenario is a failure or a success, every time. For example, if I had 3 successes I must have had 2 failures and the challenge has failed. If I have 4 successes and 1 failure, then the challenge was won.
What I will assume is that every skill challenge is a series of 5 rolls, and I will add up ALL of the scenarios in which the party gains a success. The reason I can do this: Because I’m looking at only the winning scenarios, the maximum number of failures I can have is 1. After all, if I have 2 failures, I would have had to fail the challenge somewhere along the way through the 5 rolls. Therefore, I don’t care when the failure happens. I could happen in the 1st roll, or the 5th. As long as I have my 4 successes in their somewhere, I will count it as a win for the party.
So in our series of 5 rolls, there are two ways in which the party can win the skill challenge:
1) 4 successes and 1 failure.
2) 5 successes and 0 failures.
We must calculate the odds of both and add them both together to get the total party success.
4 Successes and 1 Failure
First, we calculate the odds of this happening once. From our scenario above, the party has a +10 to their rolls, and they are rolling against DC 20 (which is given in the DMG pg 42). That means they have a 55% chance to make that DC 20 every time. That means they will also have a 45% failure each time.
So the probability of getting 4 successes and 1 failure is: .55 * .55 * .55 * .55 * .45 = .041.
Now we have a single probability, but we have to account for all the scenarios in which 4 successes and 1 failure can occur. For example, you could have SSSSF or FSSSS or SFSSS or SSFSS or SSSFS. We will use a combination to calculate this, because the position of the successes and failures don’t matter.
That’s C(5 rolls, 4 desired) = 5. So we will multiply our initial probably .041 * 5 = .205.
5 Successes and 0 Failures
Initial Probability: Now its .55*.55*.55*.55*.55 = .55^5 = .0503.
Our combination is C(5rolls, 5 desired) = 1. 1 * .0503 = .0503
Total Outcome: Adding both of these scenarios together gives us: .2508 = 25.08% win rate. So basically with my party taking on the easiest possible challenge, they have only a 1 in 4 chance of succeeding. And remember, I gave them an extra bonus!! If I use my original example (+9 to each roll) they only have an 18.75% chance.
Calculating Higher Complexities
So I have shown the math for the most basic challenge. But often you are going to want complexity 5 challenges, which is supposed to be the skill challenge equivalent of a battle. A battle against basic level 1 guys doesn’t sound too bad right?
I’m not going to go into the full math again, but I will lay out the ground work so you can do it yourself.
We need 12 successes before 6 failures. That means at most we can have 17 rolls (Notice how this number is always one less than the total between successes and failures, that never changes). Out of all of our 17 roll scenarios, which ones will give us success?
The answer, 12 successes and 5 failures, 13 successes and 4 failures, 14 successes and 3 failures, 15 successes and 2 failures, 16 successes and 1 failure, and 17 successes and 0 failures. Phew!! A lot of math to do. I will run the first scenario, and leave it to you all to finish the rest. I will of course provide the final answer!
12 successes and 5 failures.
Our individual probability is .55^12 * .45^5 = 1.41E-05 = .000014
Our total probability for this scenario is C(17 rolls, 12 desired) = 6188. 6188 * .000014 = .08749.
So now would repeat the process for the other scenarios. Doing that our total probability is: 14.71%, which is about 1 in 7. Again, that’s using that extra bonus I added, what would it be if it was a straight up 50/50 roll each time? 7.17%, about 1 in 14.
As a DM, do you feel that giving your party a 1 in 14 shot of handling even a basic challenge is fair?
Taking it to the Next Level. Well we’ve seen that a 1st level party can’t handle its own challenges, so let’s bring in our boys from 4th level to do the job right. Those guys have a massive +2 bonus to all skills to help them out, so a total of +11 (I don’t think these guys need that freebie bonus, afterall, these challenge is literally beneath them in difficulty!)
4th level party vs 1st level, Complexity 1 Challenge: 33.70%
4th level party vs 1st level, Complexity 5 Challenge: 26.39%
Um…maybe that party needs that freebie bonus afterall?
How about 6th level for another +1?
6th level party vs 1st level, Complexity 1 Challenge: 42.84%
6th level party vs 1st level, Complexity 5 Challenge: 41.97%
Okay okay, enough of this! Our 8th level guys come in with another +1 and their stats have been bumped again, so let’s give them a net +2 over our 6th level team.
8th level party vs 1st level, Complexity 1 Challenge: 63.28%
8th level party vs 1st level, Complexity 5 Challenge: 76.53%
Your going to look at the 8th level numbers and think it’s a typo…its not. That’s a result of how the complexity numbers are laid out. Once you get to 70% chance of succeeding on each individual roll, higher complexities actually give you a higher chance to succeed.
So…we have finally gotten a better than 50/50 chance to beat the challenge, and it only took a party 5-7 levels higher to do it.
Conclusion
I hope these examples have given some concrete proof that skill challenges are flawed as written. If you have any skepticism left, remember that I gave the party the advantage; I assumed they could use their best skills for the job. Take my original example, assume a 5 person party, and let’s say that one character only has a +7 to his skills for that challenge. The result against a complexity 1 challenge? 16.35%. Against Complexity 5? 5.17%. Better roll a 20 on that d20, because that’s all the shot you have.
However, there is hope. I have taken the time to reconstruct the model and fix the numbers. That system is in the thread above. I hope this article has been a good eye opener for everyone, and I hope that if you don’t choose to use my system you do find a new system that works for you. While the math of the skill challenge system doesn’t work, the concept is very good, and I would hate if any group gave up on it due to numbers alone.
