Tom Webster, writing and speaking

Quick And Dirty Email Product Testing The Right Way

Added on by Tom Webster.

If you are thinking about introducing a new product, or making some tweaks to an existing product, there are a variety of ways to use market research to help hone your offering and optimize pricing. You might want to use qualitative research to determine why a customer might or might not be attracted to your product, and you might use quantitative models to determine pricing, or feature sets. If, however, your goal is simply to determine whether or not your customers will actually buy a given product, there is no better predictor of behavior, than behavior. In other words, if you want to find out if there is interest in a given offering, one of the best things you can do is just offer it to a sample of your customers, and see if it works, or if it needs to go back to the drawing board. If you are fortunate enough to have a reasonably large email database, then testing a product with a sample of that database is a pretty sound way to proceed, especially if you have multiple offerings that you wish to compare. This allows you to try different combinations of price and features with various segments of your database and see which ones have legs, and which ones need to go back to the drawing board.

Some of you may have access to robust email marketing suites or other marketing automation tools that can help here, but if not, here is one way to get the most out of this kind of quick and dirty email marketing "research." The key is in proper sampling (isn't it always?) You will want to ensure that you are getting a truly random sample of your email database for this to really be of use as decision support - if you take the first 100, or the last 100 names, for instance, you are imposing a bias on your efforts by using a disproportionate number of early (or late) adopters to your product or brand. Truly random sampling means that every member of your database has an equal, non-zero chance of being selected.

So here is the official BrandSavant way to do this yourself:

  1. First, you need to have your email database dumped into Excel, so that every record (person) is on a different row. Pretty much any list manager you might use probably exports to Excel, or at least to .csv, which Excel reads nicely.
  2. Insert two columns at the beginning of your database (in other words, add blank columns "A" and "B" to the left of your first column, shifting the first column of your database to "C".)
  3. In A1, enter RAND() into the formula box.
  4. Highlight A1 and then click and drag your cursor straight down column A until you get to the bottom of your database. Every cell in column A should now have a random number between 0 and 1.
  5. With the random numbers in Column A highlighted, click Edit->Copy (or control-c) and then click in cell B1. Under Edit, select Paste Special. Paste as "Values." This puts a static copy of each random number in column A into a corresponding cell in column B. This is important, because the numbers in Column A are not fixed and will regenerate every time you do anything. The numbers in Column B are static and can be sorted, which is what you are going to do next.
  6. Under Data, select sort, and then sort by column B, ascending (alternatively you can click in cell B1 and click the A->Z button, if your toolbar has that.)
  7. Voila. Your database has now been randomized. You can select the first 100 names and send them your test offering, and send other versions to records 101-200, 201-300, etc.

Now, 100 is a smallish sample by academic standards, so you are really looking for big disparities in response. But if you are doing A/B testing, at least you can tell yourself that you've made a solid effort to equalize the samples for Offer A and Offer B. And you get a BrandSavant official stamp of approval. Attaboy/girl.