I’ve written about it before, but PIER is an acronym for “Priority, Importance, Ease, and Result.” What do these mean?
The scoring scale for PIER decisions is 1 – 5—although Priority has a 6 point scale to help weigh the urgency of a task a little higher than the other elements. Fill out all of the scores for each element in the matrix and then multiply them together to get the end result. You will have a score from 1 to 750.
When you’re scoring an experiment on the PIER system, you should get more than one person to submit scores for each experiment, add everyone’s score for each element together, and then divide by the number of people who scored the experiment.
“Ease” is the key in ICE scoring, but it’s a little bit squishy. How do you estimate Ease? Everyone would score it a bit differently. I came up with a good standardized system for scoring that works well for growth or marketing teams.
For every team outside of the team who needs to touch a task, I subtract 2 points. I also deduct a point for every 10 hours of work I figure it will take to execute. If an experiment requires no dev work and less than 10 hours of work, score it a 5 (really easy!). If it requires a developer, a sales engineer, and someone from the Finance team and will take more than 40 hours of work to pull off overall, well, that’s probably not a good experiment to undertake. Descope and try again.
It’s almost impossible to be consistently correct about the positive impact of a test. On the other hand, it’s pretty easy to know if you’ve got instrumentation prepared that will be able to detect if you had a result. You can get a good estimate if you have enough traffic to provide any clear signal of which side of the test performed better.
There are times you need to run a test even when you don’t have all of the analytics or reporting that you’d like to have to be sure that you will know what the outcome of that test is—that’s what Priority and Importance are for—but in general you should avoid tests where you might not be sure just exactly what the outcome was.
When you’re ready to use either ICE or PIER, it’s good to open up your task tracking tool and drop all of the experiments that you’re considering directly in there: