Title

Direct & Integrated Marketing Roundtable

Monday, November 5, 2012

Learn from the NYC Marathon's PR Disaster--Develop your Social Media Strategy before its too late!


We all know at some level that social media has had a trans formative influence in our world.  What we may not have experienced, is the power of that influence against one of our own interests.

We have marveled at the power of social media to highlight and move forward trivial cultural issues, such as a funny video or picture, and on a more serious note, social media has been credited with triggering the Arab Spring and consequently toppling governments.

Other organizations can learn a lesson from what has played out this week for the City of New York, The New York City Marathon and it's sponsor, the New York Road Runner's organization.  That lesson is: Build your social media strategy before you need it!



The power of social media was unleashed against the City of New York, the New York Road Runners organization, and would be marathoners.  Following the epic storm (Hurricane Sandy) that hit New York on Monday, October 29, the Mayor of New York, Michael Bloomberg together with the NYRR CEO Mary Wittenberg announced boldly on Wednesday, October 31 that the premiere running event for New York, the New York City Marathon, would continue as planned; only to announce Friday evening November 2, that the event would ultimately be cancelled.

While the initial decision (moving ahead) was surprising, both Bloomberg and Wittenberg defended the decision for myriad reasons.  Among the reasons to move forward were that the Race would focus attention on the city, and thus relief efforts would result.  More directly, Wittenberg would donate $1 million for relief and recovery efforts.

As the week wore on however, the mainstream media and social media weighed in about the timing of the NYC Marathon relative to the hurricane and the concern that resources (police, fire fighters, water  and generators) would be diverted from those in need and to the running event.  Particular mention was made of the bridge closures at a time when some of the tunnels were still flooded, making navigation on and off Manhattan more difficult.  A congressman from Staten Island, Michael Grimm appeared on the evening news on Thursday, November 1 making an impassioned plea to call off the Marathon as Staten Island was making grim discoveries in the wake of the hurricane.  Meanwhile airports had reopened, and international runners were arriving in New York, some evidently vying with New Yorkers made homeless by the hurricane for hotel rooms.  All of these stories (and more) played out in mainstream media and were amplified in Social Media.  Finally, 36 hours before the marathon was to kick off, an announcement of cancellation of the event came from the Mayor's office, and the NYRR was silent, for at least three quarters of an hour.

Certain threads of the story took on a life of their own in social media. A few of the threads which had no basis in truth but were nonetheless played out in social media were:

  • Rumor:  The NYRR has generators for runners which have been diverted from people who actually have no power and need them.  Truth:  The NYRR has a great deal of their own equipment and know exactly what is required to put on a marathon having hosted the premiere U.S. marathon annually for the last 40 years.
  • Rumor:  The runners will take water away from people who have no water.  Truth:  This notion is as ridiculous as the first one for the same reason.  You don't stage a marathon annually for 40 years and not have a concept about the requirements of water for the runners.
  • Rumor:  The runners could just as easily run circles around Central Park.  Truth:  This was not a viable solution due to the sheer number of runners involved.  In the early years of the marathon when competitors numbered in the hundreds this was done, but how could the park accommodate 44 thousand runners?
  • Rumor:  The race can be postponed for a week.  Truth:  Well this sounds like a good idea, but in a city like New York where the staging of the marathon requires street closures, and associated logistics, it is not a given that all conditions will be "right" for just moving it by a week.  This isn't a school picnic after all, thousands of volunteers, and support people are involved.  There is no guarantee that all of the venues would be available a week in advance, and not in conflict with some other event.
Some of the comments that were made in Social Media were not only not intended to be helpful at all but just be rude and directed at runners:
  • The runners want only to complete an item on their bucket list.
  • Anyone who watches or participates needs their head examined.
  • No human being should run that far anyway.
  • Runners are out to impress people in their fancy running clothes.
In the compressed time period between the announcement that the Race was on as of Wednesday morning, until the time that it was announced as cancelled on Friday evening, one fact was painfully obvious to me.  The voice of the runners was missing, I would have thought that the NYRR would have the wherewithal to launch an effort to be the voice of the runners.  I am sure that they have never been in such a role before, but it was rather eye opening that their viewpoint was completely absent in the dialogue.  

In the final analysis, it is certain that even if the NYRR had engaged with the critics in social media dialogue the result would have been the same -- the event would have been cancelled. But had the NYRR been a more active participant in social media, perhaps the NYRR could have maintained some level of good will among their core constituency, the runners.  Or alternatively, had the NYRR been a skilled user of social media, perhaps they would have made an announcement to cancel the race at the outset.  For example, the graph below created using Lithium (a social monitoring tool) clearly shows an explosion in conversation involving the words "NYC Marathon" and "cancel" in the social space (twitter, Facebook, blogs, news feeds, etc.)   In addition, we note that the sentiment regarding these conversations is more heavily skewed negative (red) versus positive (green).  



From the NYRR Facebook page we can see only 16 posts were made between October 25th and November 2nd.  However the 16 posts by the NYRR generated nearly 4,000 comments from other Facebook users.
  • October 25--First mention of a large storm that is being monitored.  A second post about "Mighty Milers" is also posted on this date.  Total of two posts.
  • October 27--Poland Spring Kickoff Race two posts.  One cautions about the storm and directs attention to social media and a special hotline phone number.
  • October 28--Monitoring Sandy one post.
  • October 29--"Still working through hurricane Sandy" one post.  This post seems to indicate that an assessment of some kind is going on but the main emphasis in the post is to communicate the Half Marathon registration in 2013.
  • October 30--"Keeping all Options Open" two posts.
  • October 31-- Single post requests runners to stay out of Central Park as clean up proceeds. Mayor's announcement regarding going forward was made on October 31, but was not posted to Facebook immediately.
  • November 1--"Race goes forward"  references announcement from the Mayor; Two posts "Race is dedicated the victims of the hurricane" soliciting donations.  Total of three posts for the day.
  • November 2--NYRR changed cover photo to "Race to Recover" then at 6:28 pm posts the news that the race is canceled. Two more posts appeared later in the evening.  Total three posts for the day.

What could NYRR have done differently to better manage the dialogue?  

More monitoring of the social media, and in a proactive manner.  It seems that they were either not monitoring the social media or if they were they were not responding in a manner to diffuse the criticisms and misconceptions.  Is it possible they did not fathom that the decision would be controversial?  They could have tested the public sentiment by asking questions on their Facebook page as an attempt to gauge public sentiment.

Regarding the timeliness of the social media communication, often announcements were made in other media first, leaving the social channel as a place where core constituents, learned the news second hand after it was tweeted out or communicated by others.  

Provide a platform for the runners to communicate to each other. The NYRR was not a heavy user of social media before this Marathon, but perhaps that will change after this experience.  Had the NYRR promoted a special hashtag to marathon entrants, then perhaps as the social media lit up against the Marathon, the runners could have found their own collective voice.  It is unfair to characterize all runners as selfish, insensitive, etc. and having a voice and feeling part of a community against all of the critics would have helped to minimize the damage to the relationship NYRR has with their constituency of runners.

Had the runners been provided a platform for communication they may have come together and made the suggestion to cancel the event to the NYRR, thus making the runners part of the process instead of hapless pawns in the decision.

NYRR needs to provide more information about their efforts to help the community forward going and cleanup and recovery efforts right now.  As noted based on their Facebook page there was granted not a lot of time to socialize the vision for helping with the recovery efforts, however, the organization runs races (shorter in distance) all throughout the year, and the NYRR is involved in supporting many charities in the city.  I would venture to say that most critics of the Marathon do not know that the NYRR raises money for prostrate cancer screenings, lung cancer research, victims of domestic violence, and food programs for people with AIDS to name a but a few of the worthwhile causes.  In addition the NYRR is an organization dedicated to fitness promoting running to children and having mentoring programs where adult runners teach children and teens running techniques.

Had NYRR's good works been better promoted throughout the year, it would have been much more difficult to cast them in some kind of a villian's role.  Then the most they would seem guilty of would be being overly optimistic on how quickly the city would be ready to have the race.

NYRR needs a comprehensive social media strategy.  For a not for profit organization a comprehensive social media strategy consists of:
  • A clear definition of how the social media strategy will support the NYRR brand and mission.
  • An editorial calendar.
  • Ensure that the social media strategy integrates fully with all of the other marketing strategies, this means that the NYRR twitter handle and Facebook page is mentioned on all promotions of races, on the web pages, and in the signage, etc used at races throughout the year.
  • Read and measure key metrics and synthesize them throughout the organization.  Social media metrics should be aligned with the measurement of new members to NYRR, donations, and mentions in social and other media all of the NYRR's key metrics should be easy to follow in a single dashboard.
  • Track results routinely and refine strategies based on what works well in terms of engagement.  Feed back results throughout the organization.
  • When all of the important groundwork has been laid to manage and read social media metrics and other important organizational Key Performance Indicators (KPIs), then NYRR should begin to understand and reach out to key influencers to develop a strategy to further their brand message via this important group.
Don't forget, success in social media requires:
  • Defining the strategic purpose of the medium.  For a not for profit, are you using the medium to acquire new members or communicate with current members?  Or are you trying to communicate your mission to potential donors?
  • Defining the nature of your relationship to all of your constituents.  Are you a cheerleader, advisor, curator, mentor, friend, etc?
  • Establishing the narrative, what is the message you want to convey to your followers?  Can you structure your message to work with other channels?
  • Paying attention to this medium, monitoring the dialogue and commenting appropriately.
  • Establishing in advance KPIs that will identify the effectiveness of the strategy.  (If you're not measuring it you're not managing it!)
In the spirit of full disclosure I am a member of the NYRR organization but have never run a marathon.  I have great admiration for those who have the discipline and strength to train for and  run a marathon.  My husband Perry is an avid runner, and I am a sometime runner participating in shorter races throughout the year.  It is because of my association with NYRR that I know what a terrific organization they are, and it is my hope that they adopt a social media strategy so that they can control the conversation about their organization.

What are your thoughts about the social media strategy (or lack thereof) surrounding the NYC Marathon?  I would like to hear from you.

Rhonda Knehans Drake


      

Obama vs. Romney 2012 Social Media Barometer Dashboard (Week ending 11/04/2012)


As this is the last report before the election, let's keep in perspective social media strategy is but one of the many tactics available to the candidates.  It will probably take several election cycles to understand how much influence social media has on the electorate relative to endorsements, TV ads and the candidates positions on key issues.  The election of 2012, however is our first data point in what promises to be a new kind of campaigning where citizens can weigh in with their opinions immediately via social media.

We will soon know who's tactics were more effective.  Remember to exercise your right to vote!

This week events surrounding Hurricane Sandy, which hit the east coast on Monday, October 29, impacted the content of the conversations as can be seen below.


The highlights of the week for the period ending 11/04/2012 are shown below:

  • Obama had a higher share of voice with 56.5% of the conversations mentioning Obama.
  • The ratio of positive to negative sentiment for Romney reached an all time high since reporting began.  The ratio of positive to negative sentiment was 90.1% for the week.
  • All web metrics were down for Romney indicating that both traffic and engagement were down for Romney.
  • Word clouds indicate a conversational focus on Hurricane Sandy for Obama, and an election victory for Romney.
  • Romney continued to have a high level of engagement on Facebook relative to Obama.
  • Google search trends revealed searches for Obama and Christie and Obama and Bloomberg indicating an interest in the President's impact in the hurricane relief.  Romney searches included FEMA and Jeep indicating recent positions that he articulated in the role of FEMA, and the strength of the automobile manufacturing industry.

Rhonda Knehans Drake

Tuesday, October 30, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard (Week ending 10/28/2012)


This week Obama has taken back the lead once again in share of voice, however, not all is positive for Mr. Obama.  The highlights of the week for the period ending 10/28/2012 are shown below:
  • Obama took back the lead on share of voice for the week standing at 53.5%
  • Ratio of positive to negative sentiment for the week hit an all time high standing at 80% for Romney.  The ratio for Obama was less at 68%.
  • All website metrics for both candidates were positive for the week, stronger for Obama.
  • Romney's Facebook fans exceeded 11 million this week
  • Both candidates had a record number of tweets for this reporting period.
  • Donald Trump was trending with Obama and Gloria Alfred for Romney.
  • Klout scores remained unchanged for both candidates.



Next week's update will be published the day before the election.  We are very excited to see how things shape up.

If you need assistance in developing or executing social media dashboards for your brand, contact the Marketing Data Scientists at Drake Direct (www.drakedirect.com).

Rhonda and Perry Drake

Monday, October 22, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard (Week ending 10/21/2012)

This week Romney has taken the lead once again in share of voice, however, not all is positive for Mr. Romney.  The highlights of the week for the period ending 10/21/2012 are shown below:

  • Share of voice for Romney is up and standing at 51.38%
  • Ratio of positive to negative sentiment has strengthened for Obama and weakened for Romney
  • Word clouds are focusing on hot topics such as gay marriage and planned parenthood
  • Romney added more than one million Facebook fans this week.'
  • The re-tweet rate for Romney has fallen since last week.
  • However, Mr. Romney's Klout score has increased.


Next week's update should prove interesting following the Monday night third and final round of debates.  Check back next Monday to see how the debates shaped the social media narrative.

If you need assistance in developing or executing social media dashboards for your brand, contact the Marketing Data Scientists at Drake Direct (www.drakedirect.com).

Rhonda and Perry Drake

Tuesday, October 16, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard (Week ending 10/14/2012)

This past week the analysis of social media conversations has shown a shift in the sentiments for Romney.  It is not clear what is driving this shift, whether it is the first presidential debate, the vice presidential debate, or a combination but there is definitely more positive sentiment towards Romney as of the past week.
The highlights of the week for the period ending 10/14/2012 are shown below:

  • Share of voice for Obama is up slightly from 49.9% last week to 52.4% this week.
  • Positive sentiment held steady for Romney at 73% positive sentiment to negative, but the positive to negative sentiment for Obama dropped to 51% this week versus 64% last week
  • All of Obama's web metrics declined in the last week.  Romney however saw declines in Traffic Rank and average time on site, but increase in % reach (up 2%) and page views per visitor (up 8%)
  • Word clouds continue to highlight conversations focused on winning, with "Win debate," and "Win election" figuring prominently in the discussion.
  • Both candidates are adding Facebook fans rapidly, and Twitter followers increase as well albeit at a slower rate.
  • Most search trends have focused on the candidates themselves, for the most part other personality mentions in conjunction with the presidential election have ceased, the notable exception being Big Bird in owing to Romney highlighting him at the first debate and subsequent mentions in political advertising.




Monday, October 8, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard (for week ending 10/07/2012)

The latest update of the Obama/Romney Social Media Dashboard is ready for viewing. Highlights through October 7, include the following observations:

  • Share of voice was equally split for the two candidates at 50% each.  A shift from last week when Obama had the lead.
  • In general, mentions of both candidates is at record levels and is up 123%.
  • Romney's ratio of positive to negative sentiment for the week was observed to be 14% higher then Obama's 
  • At the end of the reporting period, traffic spiked to both candidates websites due to the debate.
  • Additionally, both candidates had a significant increase in Facebook fans and engagement this week due to the debates.  However, Romney was observed to have record engagement at close to 33% of  his fan base. 
  • This week Romney's use of Twitter increased three fold.
  • As prior weeks, Romney leads in retweets per tweet but still trails behind Obama in usage.
  • Trending terms revolved around who won the debate.


Next week should be very exciting with the first Vice-Presidential debate of the election season occurring on Thursday.  Check back next Monday to see how the debates shaped the social media narrative.

If you need assistance in developing or executing social media dashboards for your brand, contact the Marketing Data Scientists at Drake Direct   ( www.drakedirect.com).

Rhonda and Perry Drake

Monday, October 1, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard (for week ending 9/30/2012)

The latest update of the Obama/Romney Social Media Dashboard is ready for viewing.  Highlights through September 30, include the following observations:

  • Obama has regained a slight lead in share of voice (last week the share of voice was virtually even).  This week Obama social mentions represented about 58% of all mentions between the two candidates.
  • Obama's positive sentiment fell to 40% of negative (down from last week where positives were 53% of negatives) and Romney's positive sentiment improved slightly to 50% (last week positives were 48% of negatives.
  • At the end of the reporting period, traffic spiked to Romney's website, and the reason is not immediately apparent, but the increased traffic improved both Romney's Site Rank, and his Reach.  Consequently, the increase in traffic to the Romney website has impacted engagement metrics both average pageviews and average time on site were lower this week by about 25%.
  • Romney continues to have higher engagement among Facebook fans, although engagement as of week ending September 30, was at 22% down from nearly 30% last week.  Romney continues to build his fan base on Facebook steadily adding 6.5% new fans in the past week.  His fan base is still less than one third of Obama's (29.0 million for Obama vs 7.7 million for Romney) but if Romney keeps this pace he should exceed 8 million fans by next week.
  • The two candidates make vastly different use of Twitter.  In the past week, Obama issued 162 tweets versus Romney's 13.  This level of communication is fairly constant in comparison to the prior week.  Retweets (last five tweets retweeted) as a percent of followers however has fallen a bit for Romney.  This week's retweet rate is only 57% of last weeks number.  Romney still has much higher engagement in Twitter than Obama, which is a consequence of the smaller, more engaged fan base.


Next week should be very exciting with the first Presidential debate of the election season occurring on Wednesday.  Check back next Monday to see how the debates shaped the social media narrative.

If you need assistance in developing or executing social media dashboards for your brand, contact the Marketing Data Scientists at Drake Direct   ( www.drakedirect.com).

Rhonda and Perry Drake

Monday, September 24, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard Update (week ending 9/23/12)


The next update of our Obama Romney 2012 Social Media Dashboard is complete.  Highlights through 9/23/12 include the following observations:
  • Obama's share of voice has slipped from 61.5% for the week ending 9/16 to 49.6% for the most recent week.  
  • Negative to positive sentiment remained high for both candidates with a slight increase in positive for Obama.  
  • Searches on Google spiked for Romney coincident with his publicized remarks about "47% of America".
  • Romney's fan base on Facebook and Twitter still remain more engaged than Obama's as we have been consistently observing over the past few weeks.


Please feel free to comment, and of course check back next week!

Rhonda and Perry

Monday, September 17, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard Update (week ending 9/16/12)

The next update of our Obama Romney 2012 Social Media Dashboard is complete.  Highlights through 9/16/12 include the following observations: 
  • Obama's share of voice has slipped from 68% last week to 61.5% this week.  
  • Negative to positive sentiment has increased for both candidates in light of the turmoil in Libya and Egypt.  
  • This global event has also shaped the conversations as can be seen in our word clouds produced using NetBase.
  • Romney's fan base on Facebook and Twitter still remain more engaged than Obama's as we have been consistently observing over the past few weeks.
  • Nicki Minaj is trending with Romney given her "endorsement" statement last week.
The next update will be 9/24/12.  Until then enjoy the metrics.

Perry & Rhonda Drake

Friday, September 14, 2012

Measuring Social ROI...A Social Media ROI Calculator

Are you measuring your social ROI?  You know what I say

              .........just do it!



You have to start somewhere.  You have to start sometime.  So it might as well be now.  To be honest, it is really not that difficult to put some numbers against your social marketing efforts.  Trust me.  And, it becomes a fun exercise in doing what I call "What if" analyses.

I have been doing ROI calculations for a long long time. And I thought, why aren't more marketers trying a bit harder to determine the ROI of their social marketing efforts.

When I went to the Social Media Benchmark Conference on May 9th in NYC this year I expected to hear all about Social ROI and how various companies are doing it.  To my surprise very few numbers were revealed.  I thought wow.  How can this be?  See my post titled Beer, Sex and Social Media regarding this conference for more details.

Another conference is coming up titled Social Media ROI sponsored by Business Insider in NYC on September 27th.  I plan on attending this one as well.  And, I am curious to see if I will get some real numbers this time.  I will find out soon and report back of course.  But in the mean time lets you and I discuss social ROI and how we can do it.

Of course let me be very upfront, clear and honest...an ROI for your social campaign is not going to be perfect.  By no means.  There will be a lot of assumptions you will have to make.  But with a little thought, and the use of preexisting research, some real meaning can result out of your efforts.  It is actually kind of fun.  A challenge.  I love challenges!

So let me take you through the thought process and the steps to conduct a social ROI analysis.

First of all there is a ton of great research out there that we can use to help us do our ROI analyses.  I consolidated some of the best studies from ComScore, Forrester and others available today to help us.  They are summarized below. These will become inputs into our calculations later.

Facts from "The Power of Like" by Comscore & Facebook, May 2011
  • 16% of fans are reached by branded content by a brand that posts 5 of 7 days and roughly 22% are reached for brands that post every day as shown below in Figure 1.  Pretty useful data for our ROI calculations.
Figure 1
  • Results are similar for friends of fans - a given status update from a user will result in about 12 % reach among friends.
  • Fans and friends of fans for Starbucks were found to spend 8% more and transact 11% more in store than non-fans and non-friends of fans as shown below in Figure 2.  I think we can use this too.
Figure 2
  •  Southwest airlines exposed fans were 5 times more likely to visit the web site than non-exposed fans. And, Southwest exposed friends of fans were 2.5 times more likely to visit the web site than non-exposed friends of fans
  • Bing fans conducted 68% more searches than non-fans.  Bing friends of fans conducted 27% more searches than non-friends of fans               

Facts from "The Facebook Factor" by Forrester, April 9, 2012
  • Between 55% and 95% of fans are likely to purchase in the next 12 months for the companies considered in this study (BestBuy, Coke, BlackBerry, Walmart).  Very interesting.
  • Between 10% and 74% of non-fans are likely to purchase in the next 12 months
  • Fans were between 5.3 and 11.4 times more likely to purchase (see Figure 3 below)
 
Figure 3

Facts from "The Value of a Facebook Fan" by Syncapse, June 2010 
  • Fans spend 84% more than non-fans based on the companies studied.  Now that is a great fact.
  • Fans are 28% more likely to stay with a brand
  • Fans are 41% more likely to recommend a brand to a friend
  • 68% of fans in general recommend a brand to a friend
  • 48% of non-fans recommend a brand to a friend               

Facts from “What is the Typical Virality Rate for a Facebook Post?” by EdgeRank Checker, March 2012
  • The median "virality" rate for Facebook pages is 1.92%.  A valuable number also for our consideration.

The ROI Calculation
Below is an example ROI analysis I conducted for a client not that long ago.  To protect the client and my work for them, I have removed reference to their name and some of the calculations.  

The main goal of the analysis was to help them understand the benefits of obtaining 100,000 fans and if those benefits would offset the costs.  The screenshot of the Excel spreadsheet I created to conduct this analysis is below and requires the following data:
  • The lift in spend for fans and non fans (using ComScore study)
  •  An estimate of one year customer value, or some other proxy.  For this client we used $100.
  • What percent of those fans you obtain will even see your posts if you do them daily.  Again, not all will see them.
  • The virality rate based on research findings
  • Reach for your friends based on comments and shares.  Not all will see them
  • Estimate of average friends per fan.  130 was based on a study I found somewhere but you can use whatever you think.
  • % of fans and friends that are exposed that will buy again based on the research available..
Putting it all together in a spreadsheet was the next step.  Calculations were done to determine:
  • How many of your fans will see a given post
  • How many of those will share it
  • What the true reach of the friends of the fans will be
  • How many of the exposed fans and friends are already your customers
  • The resulting increase in spend.  
 


Using the facts from the various studies cited above and assumptions obtained from the client I input the data into my spreadsheet and determined the increased value to the brand by obtaining 100,000 fans would be approximately $211,000.  And, this is just based on a one year value.  So the question posed to the client was:

What will it cost you to get these fans and will it be less than $211,000.

Keep in mind, as good as these numbers look, we have not even taken into consideration other benefits like conversion of fans that are not customers to customers, increased customer retention, increased advocacy among fans, conversion of friends, etc.  So many other benefits.

Give it a try.  You might as well start now.  It will never be perfect. Never.  So no need waiting.

If you need help and guidance in assessing the success of your various social campaigns (or any campaign), please do not hesitate to reach out to me.  This is my specialty.  Note I do have some ROI calculators available for FREE and are located on my web site dedicated to marketing test design and analysis at www.confidenttest.com.  Check them out.

I hope you enjoyed the post and that it gave you some great ideas as to just what can be done here.

Perry
PS  the studies cited in this post are listed below.

Resources:

Monday, September 10, 2012

Obama vs. Romney 2012 Social Media Barometer Dashboard Update (week ending 9/9/12)

It is official.  We have updated the 2012 Election Dashboard Infographic.  It is for the week ending 9/9/12 and includes all social discussions shaped by the Democratic National Convention last week.

Our first dashboard presented one week ago (http://www.drakedirect.blogspot.com/2012/09/election-2012-social-media-barometer.html)  was released right after the Republican National Convention.  In that release, Romney had almost 50% of the share of voice and more favorable sentiment vs Obama. In the release of the new dashboard, shown below, we see just the opposite.  Highlights include:
  • Obama now holds 68% share of voice compared to last week at 54%
  • Obama's positive to negative sentiment ratio is now favorable vs last week where Romney's was favorable.
  • However, Romney's fan base and twitter followers are still more engaged albeit smaller in size.



The next update will be 9/17/12.  Until then enjoy the metrics.

Perry & Rhonda Drake

Thursday, September 6, 2012

Baselining--Your Key to Measuring Successful Engagement in Social Media



Baselining--Your Key to Measuring Successful Engagement in Social Media

So, now you have your social media strategies set.  You have a playbook developed and an editorial calendar established for the next four weeks.  Your social media manager knows to stay on message and post at specific times throughout the day. 

How do you look at your data to begin to build the knowledge base to really identify what is engaging your followers?


This question is far from trivial.

Your have two goals.  The first is to attract followers or friends:
  • Fans on Facebook
  • Followers on Twitter and Pinterest
  • Connections on LinkedIn and Google+
While you get new followers and fans, you want to share content that will keep them engaged.  This is your second goal.  When you’re measuring the success of individual posts, one key challenge is to continually grow the current fan base.  Success of individual posts will be impossible to read in raw numbers because of the continuing growth in the fan or follower base.

It does make sense to think about creating metrics that give you the ability to understand the strength of individual posts in the context of your fan base.  The fan base is variable and growing, thus making total likes, comments, and shares numbers difficult to interpret, especially if your fans are growing at a fast pace. 

As a marketer, however, you would like to distinguish stand-out campaigns both from the perspective of getting new fans or Facebook page “likes”, as well as particularly engaging individual posts or tweets.  Baselining your data and measuring the contribution of current campaigns to historical levels, is key to identifying what campaigns or individual posts are truly exceptional.

These two practices – (i) comparing engagement to baselines and (ii) maintaining a history of posts and attributing characteristics – will  help you   develop increasingly engaging content for your followers and fans.  (See the graph below to understand the concept of Baselining.  The data shows weekly Facebook ‘likes’ for a brand page.  The magenta line shows the incremental ‘likes’  by week.  The two dotted lines represent the baseline and two standard deviations above the baseline. The higher one is two standard deviations above the baseline.)

Let’s consider a baseline in the context of brand likes or followers.  This case is easier to consider first.  The baseline is computed by looking at all of the weeks when you don’t have an active campaign underway to increase likes.   This represents the base increase of the consumers to engage with your brand socially when you’re not promoting your social presence.  The actual week-to-week ‘likes’ may fluctuate over time and the baseline will reflect metrics based on a rolling number of weeks (as many as you can, but preferably 30 or more).   We arrive at the baseline by computing the average number of brand likes over the non-promotion weeks ; that is where non promotion is defined as a week when no particular campaign is in place.  Overtime, the baseline may increase or decrease.  Increases mean there is a social momentum about your brand and it is a sign that your brand has good equity or a good reputation with consumers.  Your current fans could be commenting favorably about your brand and some ongoing stream of consumers ‘likes’ your page.  If your baseline begins to decline over time, it could be a sign that your competitors in the market place are attracting fans or followers who might follow your brand but that competitor’s voices or profile has overshadowed yours.  

Next, let’s consider the two standard deviation benchmark.  The two standard deviations rule is a statistical concept.  If we look at all of  weeks when we don’t have an active campaign going to increase likes, we will still see variation in the weekly number of page likes.  Any week’s likes on a brand page is 95% likely to be within two standard deviations of the mean.  So, the probability that the number of likes on any given week is more than two standard deviations from the average number of likes is only 5%.  If the number of likes rises to this level or higher, it could be considered as a direct result of a campaign (or some other influence) that has caused the number of likes to increase to such a level relative to the mean.  The trend below illustrates the two standard deviation rule.  The highest spikes occurred when a ‘like’ campaign was launched but, as you can see from the data, the campaign’s impact lasted longer than a single week.  The activity trended down at the end of April and, then, spiked again in September.



Now, let’s look at Baselining in the context of individual posts.  The process for baselining for individual posts from a particular brand is similar.  Because posts are likely to have various attributes, the baseline can be approached in two ways:
  • Develop a baseline of ‘likes’ across all posts
  • Develop a second baseline of ‘likes’ for posts with similar attributes.
The first will provide insights into the strength of a post relative to all posts to your brand page.  The second will provide insights into the strength of a post relative to others with the same attributes (pictures, surveys, videos, etc.)

For individual post ‘likes’, consider using a metric that takes into account the proportion of current fans responding or ‘liking’ a post. You might do this because as you are building your fan base, the proportion of fans liking any given post will change.  The proportion either liking or commenting or sharing should all be taken into consideration since each delivers different returns for your brand.

The key to assessing the success of individual posts is to baseline the proportion of ‘likes’, comments, and shares relative to the overall fan base.  When you find posts that are associated with a proportion of your followers “liking” in numbers more than two standard deviations above the average proportion of fans, ‘liking’, commenting, or sharing a post, then you will know you have discovered content that has risen above your norm, and has really engaged a higher than average proportion of your followers.  Over time, you will find what truly engages your followers.

The marketing data scientists at Drake Direct would be happy  to help you establish baselining for evaluating the success of your brand's social media programs.  Please contact us if you need any assistance in your evaluation efforts.

Rhonda Knehans Drake
Rhonda@DrakeDirect.com


Monday, September 3, 2012

Election 2012 Social Media Barometer Dashboard for Candidates Obama and Romney [INFOGRAPHIC]

As the 2012 election season ramps up, we find ourselves viewing various polling data which purportedly convey who is dominating the race.  Rarely do the news outlets report on all the social data available to help interpret other underpinnings of the race dynamics regarding the public perception.  In other words, the narrative on the discussion of the campaign is largely devoid of social media metrics.

We, at Drake Direct, have constructed what we think is the first Social Media Infographic Dashboard for the Obama - Romney 2012 Election as shown below.



The following are thought to be the most important metrics for the upcoming election, from a social monitoring perspective:
  • Share of Voice and related changes over time.  Share of voice is the percent of the total conversation each candidate dominates.
  • Changes in Sentiment for a candidate and comparisons to each other.  Sentiment, on its own, can be inaccurate. However, in relation to the prior week, for example, or a competing brand or candidate,  it can be more reliable.
  • Traffic Patterns to their political web sites and how those patterns are changing over time.
  • What topics are trending for each candidate via word "clouds."
  • How active each candidate is on Facebook and Twitter and, more importantly, how engaged their fan base and followers are in spreading the word which can involve the creation of a few new metrics.
  • An overall measurement of social influence using Klout as a proxy.
  • Understanding how people are using Google to search for more information on a candidate and where those searches are occurring geographically .
  • Monitoring recent quotes and posts.

There are many tools available for monitoring social data and conversation.  Some are free; some are licensed.  We used a combination of both to construct the above dashboard and included:
  • Lithium -  paid social monitoring software
  • Netbase -  paid social monitoring software
  • Tweetfeel - provides live tweet streaming on any give topic
  • Socialmention - reveals various social metrics such as reach, passion and sentiment
  • Google Insights - Google search trending tool 
  • Alexa - tracking of web traffic and engagement metrics on competing web sites
  • Compete - tracking of web traffic and engagement metrics on competing web sites (paid)
  • Facebook Insights - for any brand page, reveals basic fan and engagement metrics
  • Twitter - number of retweets per post
  • Klout - social influence score

While there is little historical data on the relative strength of social media's impact or predictive power to guide us, we can still follow each candidate's campaign execution to see how his actions fuel our emotions and conversations.  Social media can enhance or detract from our perceptions of the one seeking office. Drake Direct does not endorse either candidate.

We will update this dashboard weekly, right up to the election.  So, subscribe to our feed to make sure you don't miss an update.

Hope you enjoy our Election 2012 infographic dashboard.  To download this week's infographic, just right click on the image and save.

If  Drake Direct's Marketing Data Scientists can help your business establish social media baselines and measurement standards, using similar dashboarding, please contact us for a free consultation.  We'd be happy to help. 

Best,
Perry and Rhonda

Note:  Before embarking on any social monitoring campaign, make sure you understand the tools, the metrics, and the issues surrounding them.  Many of the metrics in the social space are prone to error, such as sentiment.  In particular, looking at sentiment on its own and not, for example, comparing current to prior week or your competitor's figures can be misleading. These metrics must be indexed.  You just have to be careful.  We will have more to say on this in a future blog post.

Monday, August 20, 2012

Commonsense Approach to Integrating Social Media into Your Marketing Mix

In the course of getting a Social Media strategy up and running for any brand, a myriad of questions will undoubtedly arise.  Many marketers approach the Social Media channel timidly, for fear of making a mistake. 

 
Other marketers assume that since the cost of developing a Facebook page lacks a subscription fee, (Facebook, Twitter and Pinterest accounts are free) that the social media channel is a kind of “throw away” and delegate the responsibility of establishing a social presence to a relatively junior member of their staff, and then just kind of forget about it.

Needless to say, both of these approaches fail to fully leverage the power that is inherent in the social media channel.  The power of the channel is, of course, the traffic.  With so many consumers spending so much time in social media, the traffic generated makes a compelling case that all businesses need to explore the social channel and define how it can work for them.  This isn’t to say that the answer will be apparent immediately, but, like the early days of the World Wide Web, there was a huge learning curve on the design and utilization of websites.  There is similarly a learning curve to leveraging social, and it requires a bit of common sense, and some ongoing attention.

Probably the most logical way to think about the social channel, and perhaps the most straightforward way to use it, is to think about it in the context of other marketing channels that you are using, and design your social activities to synergize with your other activities.  While this approach is logical, companies who employ a junior member of the marketing team to man the social media responsibilities alone, will probably not see adequate thought or effort into making sure each Facebook post, or Tweet has a clear purpose in the context overall marketing communication.

What follows is a set of commonsense rules to Integrating Social Media into your Marketing Mix.

Rule 1.  Align your Social Media editorial calendar with your promotional calendar.    
Most marketers are aware of the need to have a promotional calendar to manage key marketing related activities: media buys, product launches, management of inventory and fulfillment, but many fail to see that Social Media should be in part informed by these promotional activities as well.  Why? Because  the consumer expectation, in the Social Channel, is that there should be some recognition or knowledge from the channel that other activities or promotions are occurring.  Messaging upcoming events to consumers in social provides your most loyal consumers with the opportunity to be on the alert for a key event, and gives them the benefit of knowing about something ahead of less loyal consumers, thus rewarding their loyalty.  The benefit to the marketer is that when the promotional period does arrive, it will be more successful due to the advance messaging directly to those loyal fans and followers.

Rule 2. Establish the goals for the promotion and for the Social Channel.   
For each promotion a marketer typically develops a forecasted outcome of what the promotion will do.  The social channel is really no different.  What is different about social, is that, depending on how long you have been at it, and what tactics you have tried, there may be little baseline from which to gauge what a particular tactic will deliver.  Thus a marketer is in need of careful measurement, both in social, and across the entire online presence.

Rule 3.  Integrate social fully with all of your other marketing strategies.   
This may appear repetitive, but a full integration of social with other marketing strategies means:
  • In each email communication to consumers, Highlight your brand’s presence on Facebook, you give a consumer a chance to “Like” you in the email.  

  • On the brand’s website remind a consumer to like you on Facebook.  If there are sign up pages for coupons or other information, give a consumer a chance to connect with Facebook from your website.  Give consumers a chance to follow you on Twitter.  Be prepared to engage in a dialogue with them especially if they tweet to your handle directly.
  • On your Facebook page and Twitter page, provide your website URL.  If a Facebook post or a Tweet mentions an opportunity or offer at the website, include a link to the appropriate page to facilitate the traffic.
  • On print and broadcast campaigns indicate your presence in the social channel with the Facebook, Twitter, and Pinterest logos. 
  • Include details about website, Facebook, Twitter, Pinterest, etc. on includes product packaging if possible.
  • Consider the value of a You Tube channel for your brand.  Videos provide high engagement and are among the most popular content on the web. 
In summary, a consumer will expect that all of the channels work together, and the messaging, and branding will be consistent no matter where they encounter your brand.  To encourage engagement with a brand, social media is an excellent way to foster loyalty among consumers.

Rule 4.  Read your results routinely, methodically, and comprehensively. 
Every marketer has Key Performance Indicators (KPIs) that they read to assess the effectiveness of their tactics.  Success in social media is more than the number of “Likes” or the number of re-tweets.
Going back to the goals that you established for social, think about if over a particular promotional period whether the goal was to get likes on the Facebook page, or increase sales, or web traffic. 
If our goal is to get likes on Facebook, presumably the result can be read in the social KPIs, but the drive to get likes may have been in an email campaign.  To understand how the effort worked, we need to look at the email campaign that was facilitating the drive to Facebook, and the increase in likes coincident with the campaign.  

If we are using Twitter to increase sales, we will want to read the number of web visitors who were referred by our Tweets, and made purchases.  So the appropriate metrics here would be re-tweets, and the amount of traffic referred by Twitter.  In other words, since consumers are moving in and out of Facebook and Twitter, and between your website and Google, you need to be more expansive in your view of your KPIs especially when a campaign is in process.  Google Analytics makes it so easy to see these social referrals out of the box as the example report shows below.


Rule 5.  Organize results into a consolidated dashboard.   
It is not productive for every functional area to build a dashboard for the measurement of their own departmental goals.  Marketing is a team effort in the sense that a website, email campaign, social media, and offline efforts may all be handled by different groups.  It may be the case that not all of these groups are even organized under the marketing function.  If all marketing efforts are to be maximized then they must work together.  It is difficult for the efforts to synergize if each person only has visibility into their own component of the marketing communications.  That’s why a consolidated dashboard is a must.    A consolidated dashboard is an effective way for everyone to adopt a common vision for the business.  It shows the folks sending out emails how they influence web traffic, and how Facebook “likes” spike when all of the messaging is going on at the same time.  See the example below for a ficticious marketer of movies.

Rule 6.  Synthesize results and build knowledge.   
This seems really basic, but unfortunately as the new marketing communication strategy, many organizations are operating under the assumption that Social Media should be spontaneous, in-the-moment, unplanned, unfettered by other controls that are put in place for other forms of customer communication.  This is simply not true.  When you begin aligning all trends in a common dashboard, and the social media team begins studying the events that led to last week’s huge spike in likes, shares or retweets, the natural instinct should be to internalize the technique and try to replicate it.  A knowledge base should build to inform ongoing strategies.  This build of the knowledge base should develop into a process that allows for the social media manager to build on successes and expand communication strategies cautiously over time.  

Think of attributing each communication so a database can be built whereby you can assess the time of day, the call to action, the type of post, etc.  So that you can begin to read what your loyal followers are most responsive to.  In addition, begin to understand the cadence so you aren’t boring your fans with the same kinds of posts all the time.  

When you have developed your social media strategies, and are operating it with the care and discipline that is attendant to all other marketing communications, you will be in a position to dive deeper into the consumer relationships and even develop strategies to leverage influencers.  But before you can harness the influencers, make sure you have your infrastructure set up, a good dashboard, and a method to track the response based on the attributes of your communications.

I hope this commonsense approach has been helpful.  I am always happy to assist clients in getting their social campaigns on track.  Please feel free to contact me in the event I can help you.(Rhonda@drakedirect.com).

Rhonda Knehans Drake & Perry D. Drake

Monday, August 13, 2012

Why Don't My Double Click Impressions Match My Google Analytics Numbers?

I get asked this question all the time. Not to worry, there is an easy explanation and a fix.


Here is the main reason.  DoubleClick is reporting impressions served or delivered to ad tags.  Analytics packages like Google Analytics of Adobe SiteCatalyst are counting the execution of page tracking code.

If your Google Analytics tracking code is at the bottom of your web page and placed right above the closing of the body section (see Figure 1 below) then you are definitely at risk for reporting discrepancies.  And, the bigger and heavier the page the more at risk you are. 



Figure 1:  Google Analytics code placed at bottom of web page HTML code.
Why is this an issue?  Because a person who clicks on your DoubleClick ad and bounces quickly will in all likelihood never trigger the Google Analytics or Adobe SiteCatalyst tracking code lying at the bottom of your page code.

And if you think about it, people will bounce more and more quickly on ads versus search results.  Right?



So, what is the solution?

One simple solution is to place your Google Analytics tracking code near the top of the page to minimize the risk.  Simple fix.

And, in fact, when you create a Google Analytics account today, Google tells you to place the code at the top of the page right before closing our the header section.  That is not what they used to tell us.   Hence that is why when viewing the source code for most websites you will see the tracking code at the bottom.

So, if you want to minimize the risk of your ad server impressions not matching your Google Analytics or Adobe SiteCatalyst figures, just move the code to the top of the page.  It will not remove the risk completely but will come close.

Good Luck!
Perry

Sunday, August 5, 2012

Measuring Campaign Attribution...Where Are We?

Today our methodology of assigning campaign attribution is quite flawed. We have typically focused on last touch. According to two new studies by Forrester, agencies are getting more sophisticated and offering more services in this regard. Google and Adobe are also making it easier for us to apply proper campaign attribution. 

What is Attribution Modeling?  
It is realizing that is is not the simple diagram as shown below in Figure 1 which depicts "last touch" attribution modeling.
Figure 1:  Last Touch Attribution
But rather something more complex as that shown in Figure 2 which tries to understand the true customer journey and gives credit where credit is due.

Figure 2:  Linear Attribution Modeling
To do it right, we must give credit where credit is due.  All prior touches must get some percent of the credit.  How much depends on the attribution model you wish to employ.  

There are many types:
  • Last Touch
  • First Touch
  • Linear
  • Decay
  • Customized

Assuming we selected the correct attribution model above, there are still many other issues that exist making attribution modeling a challenge and imperfect.  These include:
  • Bringing off line and on line marketing efforts together into a single view of the customer journey
  • Understanding online behaviors across multiple devices
  • Understanding the true impact that online browsing behaviors have on off line purchasing
  • Cookie deletion by customers
  • Customers surfing in stealth mode 

We must realize that these obstacles will make it impossible to accurately conduct campaign attribution for the foreseeable future.  But that does not mean we can't do better.  We can.  We just need to be realistic as to what we can do and know that it will never be 100%.  Not even 90%.
Download a presentation I recently made at a large agency event this August on this very subject.


In this presentation I discuss what Google and Adobe are doing to help us better understand attribution.   I also highlight two totally free white papers from Forrester on their assessment of various agencies doing it right and in comparison to Google and Adobe.  You will not want to miss those. 

Highlights of the presentation includes:
  • What campaign attribution is
  • Types of attribution models used today and when to use each
  • Interactive vs cross channel attribution
  • Attribution challenges
  • The Google Analytics "Attribution Modeling Tool" and "Playbook"
  • How Adobe is helping us better understand attribution with the help of Genesis and Insight including several articles
  • "Social Assisted Conversions" in Google Analytics
  • Offline complexities
  • Case example for a movie club
  • Forrester attribution report links and highlights

Many great things happening right now.  Take a look and stay informed and involved.

I hope you enjoy the presentation.
Perry

Thursday, July 12, 2012

Shopping Cart Abandonment Rates Continue to Rise

Wow, I can't believe all the money retailers are leaving on the table. As abandoned shopping cart rates continue to rise, e-commerce sites are not stepping up to the plate as aggressively as they could to recapture these lost sales.


I realize that in part the reason the rates are rising is that consumers are becoming more sophisticated with respect to shopping and comparing prices online.  But that is no excuse for an e-commerce sites not to reach back.  Just unacceptable.

Lets look at the facts.

First of all according to SeeWhy, abandonment rates have been on the rise since 2009 and are expected to continue to rise into 2012.  See the chart below.


According to Forrester, the main reasons consumers abandon their shopping carts are (multiple answers allowed):
  1. High shipping costs = 44%
  2. Not ready to purchase = 41%
  3. Price checking = 27%
  4. Price to high = 25%
  5. Wanted to save for later = 24%
So given these reasons, what is a retailer to do?  Just sit back, wait, and hope the consumer returns?  Well I would hope not.  But unfortunately that appears to be the case based on industry studies.

Based on a study by Listrak (a major email service provider), only 13% (yes only 13%) of e-commerce sites actually follow up on abandoned shopping carts.  Ouch.  That is pretty darn low.  But at least it is up slightly from 11% in 2009.  Think of all the money left on the table for a given e-commerce site...even if they just recapture 10% of that lost revenue.

Lets run a few numbers.

Consider a modest retailer with sales of $500,000 annually and a 70% abandonment rate.  What are they leaving on the table?  Using the formula below they are leaving $1,166,666 on the table.

[$500K / (1 - .70) ] - $500K 

Assuming they convert just 10% of those that abandoned with triggered emails (a realistic number), that yields $116,666 in saved revenue.  This would certainly cover the cost of the automation of a trigger based email program.  So, there is no excuse.

And, what is more surprising, for those retailers who are reaching out to customers, they are not doing the best job possible.  Believe it or not, based on that same study by Listrak, only 25% of the trigger based emails linked back to the abandoned shopping carts and only 34% of the emails were personalized.

So what is an e-commerce site to do?  My answer...test, test, test.  The cost here is minimal, yet the rewards could be huge.

  • Test the approach (soft or hard sell)
  • Test the offer (discounts, free shipping)
  • Test the timing (wait one or two or three days)

No excuses guys.  You will find your optimum!  And, if you need help setting up your test and evaluating the results, go to my website www.confidenttest.com for some free tools.  I am there to help.

As Nike says, just do it!

For a great presentation on best practice techniques on recovering abandoned shopping carts, check out the report by Silverpop and SeeWhy.  Enjoy!


Perry
VTMG9B37HFN8

Saturday, June 23, 2012

How Many Impressions Do I Need in Order to Read My Banner Ad Test Results?


"I am getting ready to run a new marketing test against my control and was wondering how many names I should be testing.  Can you tell me?"

I get asked this question all the time from clients and students alike.  My response is always the same.

"Not enough information."


In order to properly answer this question, and it is a very important question, you must be able to tell me two things:
  1. What is your current banner ad, direct mail format, or email currently yielding in terms of a response rate or click through rate?  In other words, where are you right now, baseline?
  2. And, most importantly, what would the new test need to yield in terms of a response rate, minimum, to be considered a success? 
Notice on point 2, I am not asking you what you want the test response rate to come in at.  I am sure you would want it to be triple where you are now, right?  ;-)  What I am asking you is, at a minimum,what would you need that test response rate to be in order to roll out with the new test.  That is what I need from you.  And, most likely that number will be the break even response rate for the test.

So, in order to answer this question you will need to conduct a break-even analysis.

Lets pretend you are going to be testing a new and more expensive direct mail format where the following assumptions exist:
  • The current control format costs you $0.75 per piece fully loaded. 
  • The profit per order not taking into consideration the cost of the promotion is $20.
  • The current format yields a known 1% response rate.  

Lets also assume the new format you are considering testing will run $0.82 per piece.  Using the "The Plan-alyzer" break-even calculator that I created and provide for free on my website www.confidenttest.com we end up with a break-even response rate of 1.35% (see Figure 1 below).

 Figure 1:  Break-even calculation using www.confidenttest.com

With this data, I now have everything I need in order to help you determine testing quantities.  We will determine the test quantities so that if you get a 1% response rate for the control and you get a 1.35% response rate for the test (what you hope worst case) you will be able to read it with statistical significance.  I know you want to do better than a 1.35%, but I am simply helping you set up the test so you can read the test at break even. 

I simply want to ensure you have tested enough names so that if you just break even on the test you will be able to say yes it is a winner.  If you do better than that, great.  You are covered there also.

So with that said lets go back to "The Plan-alyzer" located at www.confidenttest.com and use the sample size calculator to determine how many names we need to test in order to read the results with significance (see  Figure 2 below).

Figure 2:  Determining the sample sizes for a test and control when concerned
with accurately measuring the difference between response rate using www.confidenttest.com

Based on this calculator, we need to test 7,281 names for the test and 7,281 names for the control.  Doing so will ensure that if you get a 1% response rate for the control and a 1.35% response rate for the test (knowing you want to do better) you will be able to read that difference with statistical significance.

Let talk through a couple of things here.

First of all you will notice that I chose 95% as my level of confidence for this analysis.  That is what I strongly suggest you use as the stake in the ground.  To go below 90% is just too risky.  For more on this see my YouTube video embedded below regarding the steps to selecting your confidence level.  Very important.



Secondly, what if you come back and say you cannot afford to test a total of 14,562 names and can only afford to test about 5,000 per panel.  What do you do?

Well, if that is the case, I then suggest you go back to the sample size calculator and play with the difference you can detect until you resulting get sample sizes of 5,000 each (I call this a "what if" analysis).  Doing so, for this example, I have honed in on needing a a response rate difference of .43% or .0043 before you can detect significance (see Figure 3 below).

Figure 3: What-if analysis using the sample size calculator using www.confidenttest.com

So now the question I pose to you is: "Can you tolerate this much error?"  Because if you cannot, then there is no need to conduct the test.

This is important, so let me rephrase.

If you can only test 5,000 per panel and you get a 1% for the control and a 1.35% for the test (which is break even) you will not be able to tell you boss that the test won.  You will need to see a response rate of 1.43% for the test before you could declare the test a winner.  So the question is, do you think the test has the capability of yielding this high of a response rate.
  • If you answer no then I will suggest you pass on the test, because why test something if you will not be able to test enough names to read the results with significance.
  • If you answer yes, then I will ask you why you did not tell me that before.  Because testing the 7,281 names per panel would have been testing more than you really needed to read this test.

Test design and analysis is my passion.  If you find yourself in need of help establishing or evaluating your marketing campaigns or tests, do not hesitate to call on Drake Direct.

But first, check out all my other testing videos on my Test Design and Analysis YouTube Video Channel


Enjoy!
Perry