Before we get to the numbers, let's give a recap of what these four things do for a tank. Fight examples will be drawn from heroic instances and Karazhan, as I have not yet progressed beyond that.
Hit points: Obviously, the more hit points you have, the less likely you are to die. However, in long raid boss fights, the issue becomes one of throughput; you only get the value of your max HP once, while increased mitigation happens all fight. Nonetheless, higher HP is still good in these fights (in a fight-dependent way) for some potential non-mitigation-related reasons: 1) in fights against hard-hitting bosses, if you fail to avoid several hits in a row hit points can be a guard against the law of small numbers; 2) if you or your healer(s) are stunned or feared or whatever, you need to be able to survive a period without heals; 3) the more HP you have, the more consecutive seconds a healer can spend ignoring you, meaning that 3a) healers can spend time outside the 5-second rule and regenerate mana, and 3b) healers can afford to have their mental focus off of you for enough time to heal someone else without worrying that you will die during their mental switch time or GCD.
These are not negligible considerations, especially large in some fights (e.g. Maiden of Virtue (Kara), Blackheart the Inciter (SL), Terestian Illhoof (Kara)).
Armor: Armor mitigates every physical attack by a certain percentage, and is applied before block value. This makes it good to have against hard-hitting mobs. Also, armor as opposed to avoidance makes for a dull, less bursty wave of incoming damage. This is particuarly useful when your healer is a priest or especially druid, since if their HoTs are enough to keep you up consistently, they can slap them on you and then worry about other targets. Less burstiness is also better when you have to tank either single really hard-hitting mobs, or multiple hard-hitting mobs, where survival is important and burstiness can kill that. This is also useful in cases where you are potentially incapacitated, since your avoidance and block is not helping you there. Some good fights for high armor are Nightbane (Kara), Aeonus (BM), Warlord Kalithresh (SV), Warp Splinter (Bot).
Block value: Block value mitigates every unavoided physical attack by a certain number, and is applied after armor. This makes it good to have against fast, weak-hitting mobs (e.g. rogue-type bosses), essentially because it procs more (your incoming damage is reduced by a constant every time you block), or against multiple mobs which presumably are not as fast-hitting. Examples of fights where block value is great are Moroes (Kara), Mekgineer Steamrigger (SV), Kargath Bladefist (Shattered Halls).
Avoidance: Having higher miss/dodge/parry chance allows you to escape attacks unscathed. This is obviously the best-case scenario for an attack, and is especially useful against mobs which either apply a debuff on hit or have a debuff applied physically, of which the most noxious is Mortal Strike. Examples of fights where this comes up are from adds to Priestess Delrissa (MgT) or Moroes (Kara). Also, note that (importantly) putting stat points into here instead of any of the other categories frees up other stat points from block rating (the cheapest way to become uncrittable), allowing you to recoup a secondary benefit.
Finally, while specialization is mathematically good (see below), there are two advantages to being well-balanced. One is being able to tank a variety of fights well, because all of these factors are more relevant at times. Another is that items with bonuses to more stats give more total bonus (due to Blizzard's itemization formula) than bonuses to fewer stats. However, there are still cases where one has an outright decision to make (e.g. gemming, or deciding which badge reward to get, or whatever.) That's where my mathematician will take over.
Assuming an uncrushable tank, the formula for (physical) damage is simple: f(a,r,b,x) = (1-a)(g(r)x - b) per attack on average, where a = avoidance (miss + dodge + parry), b = block value, and r = armor. g(r) is the amount of damage that gets through given r armor; against raid bosses this is equal to 11960/(r+11960). Let's start with the sign pattern of the Hessian (matrix of partial second derivatives) of this function:
a r b
a 0 + +
r + + 0
b + 0 0
Assuming that all three stats start at zero (and this is true for any other "minimum base values" of stats), the possible options to spend some number of remaining itemization points comprise choosing some avoidance and armor values, and spending the rest on block. This makes the feasible region representable as a right triangle in the first quadrant:
Holding x (how hard the mob hits) constant for a second, our goal is to minimize f(a, r, b, x). Our first result is that holding armor constant, it is always right to go either all-avoidance or all-block. This is implied by the submatrix of the Hessian given by avoidance-block:
a 0 +
b + 0
For any function with this sign pattern, the minimum is either achieved at the maximum possible value of a or the maximum possible value of b (this is easy to show using rudimentary calculus or common sense.) Intermediate values in fact do strictly worse than one would expect given linear interpolation. This is easy to see with a real world example. Suppose that Arthur the Paladin is tanking a boss who hits for 100 damage per hit post-armor. He has three choices: 20 percent avoidance and 0 block, 10 percent avoidance and 10 block, or 0 percent avoidance and 20 block. Then the first choice will give him an incoming dps of 80 percent times 100, or 80; the last choice will give him an incoming dps of 100 percent times 80, or 80. The hybrid choice actually underperforms either monomaniacal strategy! It has an incoming dps of 90 percent times 90, or 81.
Note also that the dominant strategy depends on how hard the boss hits. If the hits go up to 150 per hit (either by facing a harder hitting mob, or by having worse armor), suddenly the avoidance strategy performs better: 120 dps as opposed to 130 for the block strategy (or 126 for the hybrid strategy, again worse than the average of the other two.) If the hits go down to 50 per hit, the block strategy wins: 30 dps as opposed to 40 for the avoidance strategy (or 36 for the hybrid.) So having more armor swings the pendulum towards block value. We will return to this observation later.
At any rate, given a fixed value of r, we know that the optimum value of either a or b is zero. This reduces the region we need to consider to the union of the hypotenuse and bottom edges of the triangle shown above.
Consider the hypotenuse first, where block value is zero. The formula for percent damage taken is now equal to (1-a)(11960/(r+11960)). Suppose that 1 percent of avoidance is equal in item points to 265 armor (this is approximately correct.) For simplicity, we will let r from now on denote the number of 265-points allocated to armor, making this formula: ((100-a)/100) (45/(r+45)). We can minimize this function along the hypotenuse by setting the a-partial derivative and r-partial derivative equal to each other:
(-1/100) * (45/(r+45)) = ((100-a)/100) * (-45/(r+45)^2), or r+45 = 100-a, or r+a = 55.
This is a surprising magic formula. What this says is that for fixed a+r and no block, the values of a and r are irrelevant when determining your best choice! If r+a > 55, then avoidance wins; if r+a < 55, then armor wins. Essentially, the better gear you have, the better you are putting it into avoidance. Note that in all real-world situations, r+a should be greater than 55.
Of course, this is not quite accurate because of block value in the case where we don't start with zero item points, but rather with some prescribed set of gear (like, I don't know, a shield.) This extra term adds a correction term of +(1/100)(kb) to the left side where k is some factor dependent on how hard the mob hits. However, as r gets lower (aka mob hits harder), note that this term decreases in importance. This means again that the minimum damage taken is always achieved at one endpoint of the hypotenuse. Since the point with zero avoidance will be discussed in the discussion of zero (or min) avoidance, we are left with only having to compare the max-avoidance (min-armor, min-block) point with whatever the maximum from the armor-block-only strategy.
For this strategy, the value of avoidance is obviously irrelevant in the maximization, so we need only consider the damage taken per hit, (45/r+45) * x - 29 * b (in itemization points, 1 point of avoidance rating is approximately equal to 29 block.) However, the value of a mob's hit, x, is relevant. Using partial derivatives again and setting them equal, we obtain:
-45/(r+45)^2 * x = -29
or r = sqrt(45x/29) - 45.
Essentially, the number of itemization points dedicated to armor increases with how hard the mob hits. The interesting thing is that this does not at all scale with the total number of itemization points available, although if x is small enough (less tha 1315) this will be negative (meaning that we should pursue as much of an all-block strategy as possible.)
Here is the optimal value of armor under the armor-block strategy, depending on how hard the mob hits:
x / armor (265r)
2k / 2840
3k / 6160
4k / 8950
5k / 11420
6k / 13640
7k / 15690
8k / 17600
9k / 19390
10k / 21090
Finally, we have to compare the armor-block strategy to the avoidance strategy. This result will depend on how hard the mob hits. Let's consider the following (fairly realistic) parameters:
-- We start with 40 percent avoidance, 12k armor, and 200 block. We have 10 points to itemize. What is the value of x at which the armor-block strategy wins? It turns out to be at or below 3885. So if the mob's unmitigated hits are less than this, prefer the armor-block strategy; if they are more, prefer the avoidance strategy. Note that all raid bosses hit for (much) more than 3885, but the armor-block strategy generally has the benefit of having less burstiness. A typical raid boss might hit for 10k unmitigated; the avoidance strategy will result in 1920 incoming damage per hit while the armor-block strategy will result in 2330 incoming dph. This is a big (20 percent) different, but perhaps not insurmountable.)
-- Let's consider my real-world tank, Abdelnaby. He currently stands at 53.50 percent avoidance, 15925 armor, and 336 block. What should I be focusing on? It turns out that Abdel's break-even point is 3927, meaning that if I plan to fight bosses with more than 3927 per hit, I should stack avoidance, while if I plan to fight things with less than 3927 per hit, I should stack block rating. Again, considerations of real-world issues may shift the spectrum towards armor/block, but it looks like my general strategy of focusing on avoidance is a pretty good one.