D&D 4E Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

[gribble]

Let's check the implications of this. If the average player has, say, a +2 to his un-trained skills, then he has a 65% chance of succeeding, and thus the expected number of successes from 4 people doing Aid Another is 0.65 * 4, or 2.6. So you would get an average of a 5.2 point bonus on your skill check from Aid Another checks... which offsets the +5 DC penalty. Hm. Personally, I would prefer if you simply drop the 5 DC increase in that event and stress individual actions.

I think you're on to something. This, combined with Morrus' (quite correct) assertion that for non time limited skill challenges everyone can assist, makes the math a lot more even. I could see PCs actually succeeding more than failing in this system (though it does seem to unfairly penalise parties smaller than 5).

And I wouldn't have a problem with assisting like this - it just means you have to look at rounds a bit more abstractly. Everyone would still get to choose and roll their skill, it just means that they automatically get 1-4 assists.

I'm no stats whiz, but I don't think it solves the problem of high complexity (= more XP) challenges being easier to succeed at though (which by my understanding is at least relatively independant of the probability of succeeding at an individual check).

 

[Ulthwithian]

Okay, here's something very odd.

***HUGE DISCLAIMER -- I am a beginning programmer***

I just wrote in the past hour or so a simple little program to simulate a given number of trials for each complexity. It's very simple, but I wanted something simple before I started complicating it.

Here's the result, for a +9 check, DC 20 check (moderate difficulty), and 100,000 trials.

Comp Prob.
1 0.187
2 0.142
3 0.113
4 0.088
5 0.072

While the probabilities are very small, the chance to succeed at the challenge goes down as the complexity of the challenge goes up.

Now, here's what happens if you need to roll a 7+ (i.e., bonus of +8, DC 15):

1 0.527
2 0.552
3 0.567
4 0.586
5 0.596

Once you have a better-than-even chance of succeeding at any given check, the longer the challenge, the better chances you have of completing it. This does make a certain amount of intuitive sense, as two unlucky rolls in a complexity 1 challenge is 'game over' while the same in a complexity 5 challenge might be able to be overcome.

Edit: Correction - It is precisely at this point that the numbers rising with complexity in my simulation. If you need an 8 or 9+, it's fairly stable, but still probabilities go slightly down with time.

Now, I wouldn't be surprised if I have some sort of logic error in my code which is throwing off this simulation. I would be more than happy to send my source code (it's written in ANSI C) to anyone who wanted to verify that my logic is correct.

 

[Wulf Ratbane]

Two words: Gambler's fallacy.

Two words: Not applicable.

 

[Wulf Ratbane]

Okay, here's something very odd.

When the probabilities are very small, the chance to succeed at the challenge goes down as the complexity of the challenge goes up.

Once you have a better-than-even chance of succeeding at any given check, the longer the challenge, the better chances you have of completing it.

You didn't do anything wrong-- you're correct.

This was been brought up earlier in the thread (perhaps even in the first post) and regardless of any fix with regards to setting the DCs (whether the +5 is included or not) there is no getting away from this "flaw" in the system as written.

There is a "break point" in the difficulty/DC setting whether the probability of success flip-flops, such that the longer the challenge takes (greater complexity, more skill checks, more iterations) the more likely you are to ultimately succeed at the task.

I'm not sure it's a flaw, it might be a feature-- but it does seem rather counter-intuitive to give more XP for what is ultimately an easier challenge.

 

[Tervin]

Now, I wouldn't be surprised if I have some sort of logic error in my code which is throwing off this simulation. I would be more than happy to send my source code (it's written in ANSI C) to anyone who wanted to verify that my logic is correct.

Your program seems to be working correctly. If you look at the probability calculations that people made earlier in the thread (including my ugly little Excel sheet) they point to exactly that.

If you look closer at the formulas used it will also make sense mathematically.

 

[Ulthwithian]

... but if that's the case, then people are objecting over nothing. Granted, the probabilities are low, but unless you need a low number (IOW, a 0.7 probability of winning any check), the paradigm holds.

I guess what I'm trying to say is if WotC expects you to need to roll an 8 to succeed on a skill challenge's skill check, there's nothing wrong with the trend in the probabilities. (Note: At first level, this is +5 stat modifier, trained, and racial modifier to skill.)

Hrm. I guess it's time to focus on increasing the probabilities.

 

[Wulf Ratbane]

... but if that's the case, then people are objecting over nothing. Granted, the probabilities are low, but unless you need a low number (IOW, a 0.7 probability of winning any check), the paradigm holds.

If it is written correctly and working as intended, then I think you will find the math nuts here converging around two final points of contention:

1) How does one go about increasing your bonus to skill checks?

a) Is it possible through "normal" play, or does it require "optimal" play?

b) If it requires optimal play, is this optimal play fun?

i) That is to say, will my character be relegated to using Aid Another in the shadow of the party skill monkey, or

ii) Will my character be forced to invest in skill bump powers, feats, or magic items?

and

2) Even assuming that the typical character can achieve better-than-average bonuses on typical skill checks, why does the system hand out more XP for a "challenge" that actually becomes less challenging as complexity increases?

 

[FireLance]

Perhaps the 'saving grace' that the designers anticipated for the system is that 'failure' of a skill challenge may mean that you succeed the task but with future implications?

e.g. in Escape from Sembia, there was a skill challenge to escape. Win or lose at the skill challenge, you still escaped... it is just that if you lose the skill challenge a future encounter gets the jump on you, rather than vice versa.

Interesting that you should mention Escape from Sembia, because Festivus posted in this thread about the revised version of the skill challenge in that adventure.

Apparently, the skill challenge DCs go as low as 12 (lower than the recommended DC for a 1st-level skill challenge in the DMG), and that may even be the standard DC for level 1 skill challenges (given the statement that "level 1 is 12"). If it is the standard, it makes it easier for the average 1st-level party without access to utility powers or magic items to succeed at a skill challenge. In addition, he stated that "often you get a second save at a much higher DC before it's a fail, if you succeed by above another given DC you get a bonus". Both of these will help swing the probabilities in the PCs' favor.

I can imagine that it might be quite difficult to get into the mindset of creating skill challenges where the results are not "succeed/fail" but instead are "degrees of success".

This is a good idea, and it would help mitigate the somewhat binary nature of the current skill challenge system.

 

[hong]

This is a good idea, and it would help mitigate the somewhat binary nature of the current skill challenge system.

I wonder how ppl's opinions would change if the idea was not that if you failed the skill challenge, you got a penalty, but if you won the challenge, you got a bonus...?

 

[Mustrum_Ridcully]

If it is written correctly and working as intended, then I think you will find the math nuts here converging around two final points of contention:

1) How does one go about increasing your bonus to skill checks?

a) Is it possible through "normal" play, or does it require "optimal" play?

b) If it requires optimal play, is this optimal play fun?

i) That is to say, will my character be relegated to using Aid Another in the shadow of the party skill monkey, or

ii) Will my character be forced to invest in skill bump powers, feats, or magic items?

My conclusion so far: The game rules assume that you use aid another (=> optimal play) to complete a skill challenge. I don't really like that. We've abused the hell out of Aid Another in 3E (Diplomacy & Search checks primarily), I don't feel like doing that again in 4E. So my solution is: No simple aid another in 4E. Reduce all DCs to the base amounts (no extra +5.). The magical items and extra feats I presume to be already calculated in.

2) Even assuming that the typical character can achieve better-than-average bonuses on typical skill checks, why does the system hand out more XP for a "challenge" that actually becomes less challenging as complexity increases?

That is a good question. It might just be a question of the "feeling" of such a skill challenge. If you roll a lot of dice, it's worth more XP, especially if each die roll is also associated with a lot of time spent at the game table, possibly role-playing the before and after of the roll. Therefor, it's worth more XP.
A further idea might be: High XP skill challenges are usually very important to advance the story of the game. You want the PCs to succeed. More difficult skill challenges (low complexity ones) not only make it hard to advance the story in the direction you wanted it go (either the DM or the PCs), and such a failure should not also be penalized by not getting a lot of XP.

Personally, I'd love to read a comment of the designers on the topic.