No beta, yes risk

In direct marketing texts very little is discussed about sample sizes, and even less about Type I and Type II errors. Admittedly, the latter is somewhat complicated, and business statistics books often recommend that readers go over sections covering it carefully and repeatedly.

‘Complicated’ is, unfortunately, not synonymous with ‘of theoretical value only’. In fact, the opposite is true in this case, because the Type II error is of fundamental importance, as becomes apparent if we step back and ask why we bother about test and sample size in the first place.

We do so, basically, for two reasons. First, we don’t want to throw the baby out with the bathwater. We don’t want to take a test result that is somewhat less than our expectation on its face value. We’d rather use the test to estimate the list characteristic leading to the result, and if that turns out to be acceptable, we’d like to scale up. This is why we worry about a (the probability of rejecting a true null hypothesis).

At the same time, we don’t want to lose money by scaling up when we shouldn’t have. This means we should worry about the minimum response that would be ok. For this we must worry about b (the probability of accepting a false null hypothesis).

Yet, the commonly used formula for sample size completely ignores beta (not only in direct marketing books and online calculators but also in most of the business statistics texts I’ve come across)!

The formula goes like this:

N =	za/22p(1-p)
	E2

Where

z is the value used for the specified confidence level

p is the estimated response (population proportion) and

E is the ± sampling error allowed.

Let’s say estimated response is 2%; the allowed error is ±0.25%; and confidence level is 90%.

Putting these into the template yields a sample size of 8,485.

Let’s see what this means in terms of b if the true response (which we’ll get to know only if we scale up to the entire list) is, say, 1.65%. b turns out to be 23%, that is, 1 in 4!

Humm, that’s bad. There’s a 1 in 4 chance that while accepting a figure between 1.75% and 2.25% as ‘as good as’ a 2% response rate, we’ll actually accept a list with only 1.65% response.

No wonder one of the well accepted rules of the thumb in direct marketing goes: “As a rule, the response rate from a rollout to the balance of the list after a successful test mailing will usually be lower than the response from the test.”

This may be because of a variety of differences between test and rollout conditions. But one thing needs to be kept in mind: If one ignores Type II errors, the success of the test can be very suspect indeed.

A way out could be to use an alternative formula:

N = (	|z0|(p0(1-p0))1/2 + |z1|(p1(1-p1))1/2	)2
	p0 – p1

Where
p₀ is the estimated response

p₁ is the value for which Type II error will be monitored

z₀ is z_a or z_a/2 depending on whether the test is one- or two-tailed and

z1 is z_b where b is the limit on type II error probability when p = p₁.

Let’s see what happens if estimated response is 2%, the allowed error is ±0.25%, and confidence level is 90% (as before); while p₁ is 1.65% and b is 10% (i.e., there is only a 1 in 10 chance – not 1 in 4 as earlier – that we’ll accept a list with a real response rate of 1.65% by performing the test).

We get a sample size of 12,643.

Sure, it’s a 50% increase in sample size. But it may be well worth it if the roll out numbers and costs are far higher than the test’s.

In any case, won’t decisions be better if they were taken with a clearer idea of the risks?

PS: Please excuse the ungainly appearance of the formula. It's the best I could manage using a Word file. Both formulas can be found in the useful templates at http://highered.mcgraw-hill.com/sites/0070620164/student_view0/excel_templates.html

Reader’s Digest list

Reader’s Digest is often talked of as a good magazine to learn English from. Shouldn’t they target people buying English courses?

Would you tell your plumber how to do it?

It’s your kitchen, your money, your headache. Yet you won’t tell your plumber how to fix your kitchen. You may try, but if he thought it won’t work, he won’t do it your way. If you insist, he won’t do the job. He’ll tell you, in no uncertain terms, to go to someone else. Chances are, you’ll back off. He know best.

Yet you’ll rewrite your agency’s copy, and redo their art. Chances are they don’t know a thing. Then why did you hire them in the first place?

Yet you pay them a lot more than you pay your plumber.

Call to enjoy

If we want data from a sales promotion form we have to enter it from a filled form. This takes time and can lead to errors. Also, all participants may not fill their forms entirely. What can be a way out? What if we asked participants to fill the form online, after which their coupon (number) will be emailed. Or fill it over the phone (single entry, by the tele-operator) after which the coupon number will be SMSed?

This work is not important

We’ve heard this often.

Another one we hear, somewhat less often, is: “We’re doing it for billing. Don’t get involved.”

The second is downright idiotic. Unless we are doing charity, we all work for money. For billing.

By that logic, only those who work for charity can get emotionally involved.

Ironically, they’re the ones who can’t get too involved. Because they frequently have to make very hard choices. About who should get aid first and who’ll have to wait. They have to take enormously risky decisions, on whether A will work or B. Because if A doesn’t work, it’ll be too late for B.

So it’s they, not we, who have to harden their hearts. It’s they, not we, who have to be ‘professional’. It’s they, not we, who have to keep one eye on the money at all times. If they go wrong, other people lose lives; if we go wrong, people – we – at most, lose jobs.

That said, working for money shouldn’t be shameful. It doesn’t mean ‘disgrace of talent’. Maybe communists think like that. Are we communists? If we aren’t, why should we feel so bad about being paid for an honest day’s work, more so when the day so often drags out into the evening and night?

Getting money is not a good enough reason for not taking our work, and, by extension, ourselves, seriously. Perhaps a prostitute shouldn’t hope for much because she’s getting paid. The same logic needn’t apply to us, unless we wish so.

Also, there’s an important difference between her work and ours. Sex is normally not bought; our work is always bought.

There is nothing wrong in selling it. And there is no reason why professional pride should come as a free gift with that sale.

Anyway, let us return to the title: This work is not important to him.

Let’s rethink this one by asking three questions.

First, what is his – the client’s – salary? Is it not, as a minimum, 50% more than yours, even when he has the same experience as you? If the work was not important, why is his company paying him so much to get it done? Why are more and more companies adding direct marketing and CRM managers and paying them far more than agencies pay their employees?

Ok, client companies often have very different economies than agencies, so some comparisons will be erroneous and misleading. Still, the client’s employer is paying him to get work out of you. That should be good enough for you to believe that your work is important.

Second, what will happen if you, that is, the agency, goofs up? Will he, that is, the client company, say, “It’s alright, don’t bother”? Or will he bite your head off? He’ll do the latter.

Possibly people react badly when anything they pay for goes wrong, regardless of whether it’s important or not. Yet, we may suspect that whatever you’re doing is important enough for the client to be bothered.

Third, how does it matter if the work is important to him or not?

It’s very important to you. It’s your job. It’s your source of livelihood.

A restaurant is not important to you. Even your favourite restaurant isn’t. If it shuts down tomorrow, very little will change in your life. You’ll find some other favourite restaurant, and, perhaps, remember the old one once in a while.

But the restaurant is very important to its owner, to its waiters and cooks and cleaners. To its suppliers. To its creditors.

For them it does matter whether it’s doing well or badly.

Or take your maid. Her work is unimportant. If she doesn’t show up one day, you can do the work yourself. Or not do it at all. If she works sloppily, or is absent too often, you can easily replace her.

But her work is very important to her.

One wage less may be the difference between a full stomach and hunger. Or keeping her children in school and dooming them to a life as bleak as her own. She better take her work very seriously.

Just as I must take my work, and myself, seriously. Regardless of its worth to my client, to the economy, to humanity, and to history of the universe.

None of them are responsible for me.

Only I am.

Besides, if it has to be done, why not do it well? If you’re going to spend lots of time and energy doing this job, and you can’t immediately switch to saving the world, why not save your spirit by taking your work seriously.

Look at it this way: If you take up an account where your client feels your work is not important, you’re looking for trouble; but if you think your work is not important, you’re staring at disaster.

Respond and be damned

My colleagues got a very fancy invitation from HP about their new cost-effective technology. Beautiful graphics, amazing personalisation.

But when the got there, on the evening of the 20:20 final, no less, they met reality.

The presentation was ok. No problem there.

But the sales representative was most uninformed and unhelpful, though my colleagues were keen to know more. He didn’t even take their business cards. Wonder why they bothered to spend the thousands?