I want to thank ImperatorK and weenog from the Brilliant Gameologists forum. I stole this exchange because I think it brings up a good question and also answers it very well. The thread was started in July this year. I changed up the formatting a little bit to break up the paragraphs and make it easier to parse.

It starts with this post:

ImperatorK
Does imagination play a role in D&D? Do rules limit imagination? Are rules supposed to limit imagination? What other thoughts do you have on this subject?

This thread was inspired by [a statement made] in a discussion about Immutability of Fluff:

“D&D isn’t just a game of “imagine”. Not just anything is supposed to be possible. The rules are in place to limit your imagination. Within those limits, there might still be considerable leeway.

For example, you can imagine infinite amount of different numbers between 1 and 2. But again, not just anything is supposed to happen in the game because you can imagine it.

If you don’t want to impose limitations on your imagination, why use a limiting ruleset? Why bring chessboard and pieces to the table if you aren’t going to play chess? Again, not a rhetorical question; you can answer it.”

And concludes with this post

weenog
Certainly imagination plays a role in D&D. Even if you’re not a Real Roleplayer and live for the wargame aspect of D&D, you’re not actually fighting or even planning out fights, all the action is strictly imaginary (excluding dice-throwing after a bad call). The rules do serve to limit imagination, and rightly so.

See, not everyone imagines the same thing at the same time, and nearly everyone is selfish to some degree or another (not a bad thing, just a survival trait). However, when you combine mismatched imaginations with selfishness in a collaborative project to imagine struggle and conflict, you eventually wind up in a little kid shouting match along the lines of:

“I shot you!”
“No you didn’t, you missed.”
“But I’m the best shooter in the world!”
“I have super speed and can dodge arrows and bullets!”
“I never miss, you’re dead!”
“No, I ran over to you and kicked you a hundred times before you could finish drawing your weapon, you’re dead!”

I don’t believe fluff should serve as a limiter, however. If it has no bearing on the mechanical outcome of a conflict (not always combat, the struggle between a puzzle’s designer and the guy that needs to solve it to move on is conflict too), it’s not necessary.

Sure, it can serve to enhance the experience, but it can also degrade it. If the fluff is interfering with the players having a good time, it’s having a negative effect, not a positive one, and it needs to butt out.

The thread actually goes on for two pages, but I think everything that needed to be said can be found in these two posts.