Blog / AB Testing
Table of Contents:
- What is A/B Testing In Marketing?
- Why Use A/B Testing?
- When Should You Use A/B Testing?
Where Can A/B Testing Get Applied In Marketing?
- Sales Copy
- Sales Video
- Facebook Ads
- Who Can Perform A/B Testing?
- Top Ten A/B Testing Software Tools
- How Does A/B Testing Work?
- A/B Testing Mistakes To Avoid
- How Long Should You Run An A/B Test?
- When A/B Testing Works - Understanding Statistical Significance
- Free A/B Testing Significance Calculators
- A/B Testing Checklist
What is A/B Testing In Marketing?
A/B Testing, also called split testing, is a technique used in digital marketing to make data-driven decisions. This powerful strategy can help avoid mistakes and ensure that you make only the best decisions for your marketing.
Essentially, A/B Testing means comparing two versions of something and seeing how they perform. For example, we can create two versions of a web page to observe how they perform against each other. After a significant number of visitors, we can select the version that performed best.
For example, if you sell a product from a landing page, you will probably have a sales script designed to convince people to buy from you. The script probably talks about the product's virtues and then tells the reader to take specific actions.
If your next step asks the reader to buy your product or service, you might get 1% of visitors to purchase from you on one page with a specific font and 0.5% from the other page with a different font. Half a percent is a significant profit difference.
You might have other ideas for perfecting your landing page to get more sales, such as:
- What heading?
- What color background?
- What size font?
- What style of writing?
- Should I have a video?
- What color call to action button?
The good news is you can avoid making the decision based on guesses or hunches. Just send traffic to two different landing pages, and over time, you'll be able to determine the page which performs better.
Why Use A/B Testing?
It dawned on me you might be thinking, is it necessary to create two different versions of the same pages? Couldn't I introduce the changes to my site and then see how it impacts my conversion rate?
Here is the problem with introducing the changes to the site or a single page.
- Time: This will take a long time. Suppose you go through all the trouble of introducing changes to your pages, only for the changes to have no beneficial effect. You're going to have wasted a considerable amount of time that you could have spent doing something more profitable.
- High-Risk: The wrong change can significantly hurt your profits and reputation. This is why Marketers avoid sending 100% of traffic to an unproven change.
- Confounding Variable: This is anything that will affect your results that you haven't accounted. For example, if you include a new search engine marketing strategy in a blog post and see a significant increase in conversions, you assume the change works. But wait, unknown to you, your high conversion rate is related to a trending topic. In this case, you could once again go to the effort of changing every single post on your site, only to find that the strategy doesn't work and you just seriously damaged your website ranking and your reputation. Bad news!
Finally, why conduct these experiments at all? Why not just use the knowledge provided by the search engine marketing community or other people who ran those experiments?
The simple answer is, your ideal target audience may respond differently.
When Should You Use A/B Testing?
It's simple, any time you are requesting someone to take action. You want to ask at least two different ways to determine which method gets more activity.
For example, if you have an advertisement and you want people to call you for more information, you should perform an A/B Test to determine what to say to get more people to request information from you.
Business owners should test every step and content of their marketing funnel. Leave no stone unturned.
Even before you publish your content on the Internet, you should have an A/B test plan. It's never too early to start.
Where Can A/B Testing Get Applied In Marketing?
Simple everywhere in your marketing, advertising, and sales funnel.
At 24/7 Online Marketing, we call the combination of your advertising, marketing material, and sales funnel a marketing funnel.
For the mathematicians, here is a simple equation.
Marketing Funnels = Advertising + Marketing + Sales Funnels
So here is a list of top elements you can test everywhere in your marketing funnels.
- Font Size
- Font Type
- Call to Action Placement
- Your offer
- Your price
- Button Colors
- Video Style
- Email Subject lines
- Frequency of Call To Actions In Emails
- Order Form Layout
- Sales Funnel Type
- Search Engine Optimization
Now let's deep dive into the five critical phases of a marketing funnel (Sales Copy, Sales Video, Webinar, Emails, and Facebook Ads) where you should apply A/B Testing.
Click the image to access our headline swipe file.
A/B Testing Sales Copy
Sales copy is the text used to influence and persuade your readers to take specific actions like clicking a link, downloading a file, subscribing to a list, or purchasing a product.
There are nine components to a sales copy covering different types and subjects. The following tests will assist you in deciding on the highest converting sales copy for your business.
Component #1 - Headlines: The first element your subscribers see when viewing your sales copy. This vital position means that your headline must be as exciting and spot-on as possible. The best headlines are ones that give the purest, most direct representation of your product. Take, for example, these two headlines:
- Businesses Grow Faster Online!
- Create A Webpage For Your Business
Based on the A/B Test results, the headline "Create a webpage for your business" returned the highest conversion rate at 40%, which gives a 90% improvement over the original. This headline provides precisely what the customers wanted while also highlighting the product's main feature.
Proven Converting Headlines: Sometimes, we may get tempted to get creative with our headlines, using wordplays or puns. But it is proven that regular fill–in–the–blank headlines outperform these more "fancy" headlines, to the tune of up to 44%. Here are examples:
- How To_______
- _______ In 3 Simple Steps!
- Who Else Wants _______?
Marketers don't have to go far to get good conversions. Using these proven headline strategies is often enough to help boost your open rates:
- Question Marks: "Can You Believe This Proof?"
- Percentages (%): "89% Of Marketers Can't Be Wrong"
- Add [Video]: "300+ High Quality Stock Footages [Video]"
- Include a (free report): "7 Ways To Increase Conversions (free report)"
- "RE": "Re: Watch Me Build Your Membership"
- Use Personal Pronouns: "You Need to See This Right Now"
Headline Length: Using shorter headlines may net you higher conversions in some instances. And when I say short, I mean around 35 characters, like the example given. ...The 'Photoshop' In The Clouds
- The headline above powerfully illustrates the software's main feature as a cloud version of a very complex and popular graphics editing tool. However, it is worth mentioning that short headlines don't always convert well because you may risk the headline being too vague if it is too short. Some a/b tests report that longer headlines have almost 12% higher conversion rates than shorter ones.
- The Design: The graphical element of your sales page is a critical but overlooked part by many. As a basic rule, the graphics on your sales page should relate to the content, and your background color should contrast your headline color to make it very clear. This theory is proven with a whopping 230% boost in conversions! Headline B massively outstripped Headline A because the background image contrasts the headline, making it very clear, legible, and attractive to read.
Component #2 Sub-Headlines: The next part of a sales copy is a sub-headline. As the name suggests, it further supports a headline by providing extra information or further highlights the main benefits of a product. Although you may not look at sub-headlines, this element is still vital to many sales letters.
- Speed Factor: As with headlines, sub-headlines benefit from numbers. These further support the headlines in terms of speed ("Setup Your Site In Only 60 Seconds!") or how minimal the steps are (Grab Your Own Graphics Set In Just 3 Steps). In general, when you highlight the speed of your product, you can get a 28% increase in your conversions.
- Being Specific: Sometimes, your sales copy can see a greater conversion rate if you include a personal touch on the sub-headline. In this case, the subheadline mentioned a specific point of a blog post that grew to almost 10,000 subscribers. When a marketer tested this, the results came back with a 21% increase in conversion.
Component #3 Introduction: When you introduce your product on your sales page, you should establish credibility by telling a story right away.
- Identifying the Problem: The next part after you have established an introduction is to identify your readers' problems. The effective way to do it is by determining whether your customers are motivated by pain or pleasure points. By identifying these kinds of topics, you can answer the question: are they qualified for my product? These points have different energies that motivate people to take action differently, so you need to understand what makes your customers buy.
Pain vs. Pleasure: When you reach out to the readers of your sales page, touching the main points attracts a prospect's attention and focuses on their problems, while pleasure points paint a complete picture of how your readers can achieve success when they buy your product. The more compelling (and converting) point you can employ is a pain because it agitates your readers' needs and opens up opportunities to position your product as the solution, much like medicine to ailments. Examples of pain points are:
- Lack of Time
- Lack of Expertise
- Lack of Budget
Component #4 - Solution to Problems: When you have elaborated on all your readers' problems or have sold the dream to them, they are ready to seek a solution to their problem or goals. And this usually boils down to one obvious answer: a product. When you give your product as a solution, you are also putting your product in the limelight.
- Product Image: When introducing your products, use images consistent with what you're offering to your customers. Professionally done photos will see a boost in sales conversion as much as 145%.
My Product is The Solution: When talking about your product, you want readers to agree that only your product will solve their problems. This is achieved by telling them what your product can offer them and how it can be helpful to them. If you can convince them, you will most likely get conversions. Examples are:
- With your permission, I would like to introduce ______
- Announcing ______
- Without further ado, I present you ______
Component #5 - Features vs. Benefits: Readers will always have this question in their heads: "What's in it for me?" or WIIFM. If your sales copy answers this golden question thoroughly, you have a much higher chance of converting your readers to buyers or loyal followers in the future. It is highly recommended you focus on your product's benefits to your readers because it answers the WIIFM question effectively, clearly, and efficiently. Big companies often list out the benefits of their products to their consumers, so it's high time you do too.
Adding A Story Element: when writing a sales copy, you are also doing another equally important thing: telling a story. So why do I mention stories when you want to create a document that sells? Here' the reason: when people are not ready to buy, they are easily turned off by straight-up promotional copies which only try to sell to them. Story-driven sale copy engages and hooks your readers from the beginning to the end. They have these things in common:
- Benefit focused
- Curiosity driven
- Emotion orientated
- Here are the statistics to prove it: Digitalmarketer shows that emotion-orientated copy increases conversions by 17.9%. This copy uses humor to provoke an emotional response. It's clever, and it made people laugh. Readers were able to relate to the problem as presented in the advertisement. The copyleft an impression on readers and made the product desirable.
Component #6 Testimony: Your testimonial page is a platform to tell your readers that this product has real benefits and has impacted them positively. Testimonials are a vital tool for establishing trust and encouraging potential buyers to take action.
- Social Proof: Another component you can consider adding to your sales copy is social proof. Why? In building a story-driven copy, social evidence shows that this product has impacted real people in similar situations with your readers. It serves as tangible results that show readers that they can achieve your product's promised results—adding success stories into your order form boost conversions by up to 33%.
- Placement of Testimonials: Simply slapping testimonials anywhere you want on your page will hurt the flow of the story and will probably make your copy incoherent. So, placement is essential. Sometimes, breaking up testimonials into sections, as seen below, can increase sales by 64.53%. Readers can read the testimonials when they are placed above the call to action. However, when grouped, readers wouldn't have been able to see them if they didn't scroll down the page.
Component #7 Handling Objection: In sales copy, you do this by addressing your readers' concerns to create technical, organizational, and personal buy-in. It's also worth remembering that you are still selling at this stage.
Guarantee: So why do many product vendors give out guarantees to the buyers? Guarantees are one of the best and impactful marketing statements you can make. Guarantees help build customer loyalty and also lead to outstanding feedback. Guarantees like "100% satisfaction guaranteed or it's your money back" helps to build trust and build a relationship with your potential customers. Another high-converting tactic you can employ is to back your offer up with a full 30-day risk-free trial. This implies to your readers that you are a trusted company.
- Trust Symbol: This is also a great way to increase conversion with guarantees. You can also replace the guarantee with a trust symbol that will boost conversions by 107%. This is one of the reputation and rapport-building elements you can consider putting into your sales copy. The reason is customers want to know that your products are of premium quality. Trust symbols also prioritize authenticity over low prices, so it's worth showing them.
- Bonuses: If you only consider bonuses an afterthought to your marketing campaigns, you are leaving money on the table. For example, when selling a course on transforming anyone into a highly paid copywriter, why not include a sales closing script that customers can use while talking to business prospects? This tactic is not without basis: Customers are willing to pay for an offer that hits home with what they're already looking for, rather than a freebie that isn't.
A/B Testing Guarantee Examples
Component #8 Call to Action (CTA): After all the storytelling, proof presentation, and handling objections, what do you do next? This part is where it gets direct: You straightaway tell your readers to buy your product now. This is achieved by an element called the 'call-to-action or CTA. In this section, I'll share with you several tips on how to convert higher with CTAs.
- Button Size& Color: CTA buttons need to be as clear and strategically placed as possible. Avoid having your readers looking for the buy button, and it is tucked away in an obscure location. So here are some tips: Using a more prominent buy button is a proven method to boost conversion rate. This is because a large button makes it more visible. However, I advise against making the button very large as it puts off your readers and makes your copy seem desperate for sales. Color-wise, there's no better than orange. This is because, in color psychology, orange signifies attention and automatically draws your readers' eyes to it. The color also makes your button pop and differentiates it from other elements on your page.
- Belcher Button: I can guess that you have looked at many sales pages. You probably have noticed that almost all of them use a particular type of button. This button is called the "Belcher button." So why is this button so prevalent? The creator of the button, Perry Belcher, tested the design element of the button in over 10,000 transactions. The results were terrific: conversion rates rose by 35% to 320%. So if you are building a sales page, a Belcher button is a go-to.
- Choice of Words: In CTAs, you have limited space for words to convince your readers to click on them. It's best to keep your CTAs concise and persuasive. Here are some tips I'd like to share: Here's a surprise: Changing "your" to "my" can make a huge difference, which resulted in a 90% increase in signups. This is because the word 'my' refers back to the marketer. This signifies that the word represents something personal and connects well with the reader.
- Being Clear and Direct: Your buy button serves one purpose: the button readers click to buy. So it needs to be crystal clear. Amazon AWS showed that text-based design CTA buttons increased sales by 49%. CTA buttons that featured a shopping cart with a "+" sign did not achieve well. The "Add To Cart" order button converted the highest number of sales than the rest. This is probably because unclear buy buttons will only confuse your customers. They wouldn't know what the button's for and will be less likely to click on them. Make sure that your buttons are clear so that customers know what to do.
- Scarcity: One powerful thing about scarcity is it compels customers to perceive limited quantities as more valuable. To create urgency that if they don't get your solution now, they are postponing an opportunity to turning their life around? This tactic works well to reduce the amount of deliberation with your reads and makes them decide faster.
- Number of Call-to-Actions: One CTA is not enough. It's as simple as that. CTAs also strengthen the overall offer of your copy. With this in mind, keep the offer at the top of their mind by positioning CTAs around the copy. Create more opportunities to convert by breaking up your copy with multiple CTAs. However, too many CTAs can hurt conversions. The optimum number of CTAs is 3.
- Price: After the CTA, there's the element of the cost of your product. The question is always the same: what price to put on my CTA? Here are some pointers: Prices that end with 5 or 9 (example: $79.99) after the decimal point convert more sales than the rest. This is because the figure seems more natural than a round number ($80).
Component #9 Post-Script (P/S): Did you know that the post-script, or P/S, is the second most important thing after the headline. A P/S is very effective because before your readers buy your product, they usually have a final line of mental defense that prevents them from purchasing. And a powerful P/S will give them that final push to move forward and buy from you.
Here are some excellent examples:
[Source: Marketing Sidekick]
- I have no idea when I'll pull this from the market, but I know it won't last forever. If you want it, please don't wait. Click the "Add To Cart" button now.
- If you aren't 100% sure this powerful system can work for you, grab this 100% risk-free test drive to get you started…
- It's impossible to lose here… unless you walk away empty-handed.
- You're just one click away, and a few minutes away, from discovering how to make more money than you ever believed possible. And what's more, this isn't going to cost you hundreds to find out if this is for you or not! I am assuming ALL of the risks…
A/B Testing Homepages with Sales Video Example
These days, virtually every sales page will have a sales video headlining it. Sales videos are fantastic: they have better appeal and attention-grabbing, serve several types and purposes, are cost-effective and easier to create, can be the highlight of your promotion, give a wholesome effect to your viewers and effectively summarize a large piece of information.
Home Page Video: believe it or not, your homepage can use the treatment of an excellent video. Dropbox put a video on their home page, and conversions went up by 100%, a vast number to consider. This is due to a simple reason: visitors to your sales page or homepage prefer watching over reading.
Video Title: Among the first thing your viewers see on your videos is the title. This varies between the demography of your viewers. However, keeping the video title short and neat is a good rule of thumb. And above all, use words that are easy to understand and attention-grabbing. You may be tempted to get fancy with your titles, but more often than not, clarity wins. Examples are:
- The secret of creating your own sales funnel
- Two quick and easy ways to start selling online
- Edmund Loh's business model made easy
Thumbnail Image: Whatever else that may be said, people DO judge a book by its cover. Before viewing your video, the thumbnail is the main thing that attracts your viewers' eyes at this point. A good video thumbnail:
- Carries the subject of the video content well
- Compels the viewer to click-through to the video
- Differentiates a brand from others
- It is optimized for any device
There is no one size fits all video thumbnail which is the perfect one for everything. But a well-made thumbnail will always get a nod from your viewers. Source: 6 Killer Tips to Make Your Custom Thumbnails Pop http://tubularinsights.com/custom-thumbnails-best-practice/#ixzz4ZHwS31vF
Video Headline: How many times have you read the headline to a video and closed it just because it didn't grab your interest? If you did, you should know that the headline plays a vital role in deciding if the video will be watched or not. Here are some excellent examples:
- "Businesses grow faster online!"
- "Online advertising that works!"
- "Get found faster!"
- "Create a webpage for your business"
Headlines have the potential to increase conversions of your viewers by up to 90%. So it will be very beneficial for you if you give this attention.
Agitate the Problem. You have stated your problems and captured your viewers' looking for more, so what do you do next? The next best thing to do here is to position yourself in their shoes and explain that you understand the problem. Doing this has increased viewership and conversions because it helps your viewers relate to your video and watch on, expecting a solution. At this stage, they may not be making a beeline to the buy button, but they will sooner or later.
The Solution: The process of selling the dream is part and parcel of your video promotion. This is because you have agitated your viewers' problems. If you do it right, at this point, they are simply begging to see a solution to their troubles. The next step for you to do is to introduce your product. This way, you can increase conversions because you tell your viewers that they don't need to look for when the solution is right in front of their eyes.
Handling Objections. This critical element is called tending to your viewers' objections. Tending to objections works well to convert viewers because this implies that you already know that your viewers will have doubts, and you are here to clear them. Keep in mind that these questions will always be your viewers' minds:
- Will this product work for me?
- Will it need a lot of money and energy?
- Will I need to learn something new?
- Will it do what it guarantees to do?
- Is it worth the price?
- Who has it helped up until now?
Other tools you can use to handle objections further are testimonials and a money-back guarantee. Testimonials help your handling because they prove that other users with similar situations have benefited from your product. A money-back guarantee also establishes good product quality and gives your customers peace of mind that they can return the product if they are not satisfied.
Scarcity: If you're launching a new product or course and the number of seats or available copies is limited, let your viewers know. If the price will go up or the bonuses are for a limited time, tell them. Scarcity works well in conversions because viewers see things in limited numbers as more valuable and will decide sooner. However, I advise that you be honest in this part. Don't say the Firesale will be only for three days and still have it on day four. Hurting your reputation is the last thing you want here.
Call to Action: One of the last (and most important) steps to include in any video is the call to action (CTA). This is where you close the deal and prompt your viewer to do something. It's the virtual equivalent to a closing handshake. That said, if you end the video with a suitable and powerful call-to-action, your conversion rates will get a considerable boost.
So how do you do this? Vidyard has done some research and has put together these high-converting tactics:
- Ask a question –invite viewers to join in with your brand's social channels.
- Enter to win – A simple giveaway on your website or social channel is easy to generate responses and interaction with your brand.
- Votes – Ask viewers to vote on something they care about with a simple poll
- Free trial – Include a free trial of your product to allow your subscribers to take it for a test drive.
- Sign up for a webinar – CTA to include at the end of your video if you frequently give webinars to highlight key features of your product or service
- Fill out a short form – The most direct way of a CTA. Restrict the number of form fields to only those that matter so people won't neglect the form for fear of a lengthy process.
CTAs are not limited to the ones on your video. You can also direct the readers of your sales copy to your video for positive results. Visual Website Optimizer found that if they changed their direct "Get Instant Access" call-to-action button to "Watch the Video" button, conversions rose to 15.3% compared to the direct one, 11.9%.
Landing Page Videos: Landing pages should not be boring pages that simply ask your subscribers to sign in or get a free download. A video on your landing pages is an excellent addition because it allows visitors to experience your message with minimal effort. Conversion rates can also go up to 80% by adding a video there.
Video Position: Your videos may be of excellent quality and has great storytelling and content. But Marketers will waste all that effort into making your videos watchable if it is slapped on your page without careful consideration of the positioning. Your videos should have a prominent place on your pages to capture as many viewers as possible. Your videos should be the most noticeable element of your page. Therefore, the best place to put your videos is above the fold. Again, your readers prefer to watch rather than read.
Auto-Play: Videos have a certain amount of control over your viewers. That said, a click–to–play video decreases the power you have over your message. If your visitor has to click on a video to play it, chances are the viewer will never see it. Setting the video on an auto-play without audio, as Facebook and YouTube do, is enough to attract your viewers to see the video and ultimately increase conversions. This is a rule of thumb, making it work on general occasions. However, if you notice that your viewers go away from your page due to the video auto-playing, turn it off.
Video Controls: With sales videos, it's essential to control the message. It would be best if you didn't give control to your viewer to skip forward the video or move the video playback slider around as it disrupts the flow of the message and ruins the buildup to your CTA.
Furthermore, if you disable the control, it will not appear in the video, making it more clean-looking. Your viewer will also appreciate this omission by having more focus on the video itself.
Short and Entertaining Videos: Your viewers lose their focus on a video after some time. Wistia found that only 60% of viewers stay until the end for a 4-5 minute video, while 75% stay until the end of a 1-2 minute video.
Voiceovers: Voiceovers are one of the essential parts of a good video. However, you will never get the viewership and conversion you dream of by simply speaking into a microphone without any structure or even a good voiceover talent. For US viewers, voiceovers with a British accent have a higher advantage to capture an audience. This is possible because the British accent sounds more formal and trusted to the US citizens; this is not the case with an eastern audience as they see the American accent as a higher class.
Informational vs. Sales Videos: Some of your buyers react best to videos that explicitly tell them what to do, like clicking the buy button or acting now, without feeling like they're being "sold" to. For Internet Marketing videos, this direct tone works best to gain conversions because of the concise nature of the market. Other markets, like Personal Development, have a strikingly different approach in the video's tone - they need to be informed instead of telling them what to do.
Fonts Matter: I will make this statement simple: fonts DO matter in your videos. If possible, always stick to sans-serif fonts because they increase your message's readability and increase conversions. While it may seem insignificant, we've seen up to a 30% higher conversion rate by just changing the font to sans serif.
Serif is little "feet" at the base and ends of the letters for those not familiar with this term. In some instances, like videos, these minor additions impede the clarity of the words and interfere with your viewers' comfort.
A/B Testing Webinars
Why webinars? They have many essential and powerful functions in any marketing effort. A relatively recent survey from quick sprout found that just over 60% of content marketers are incorporating webinars into their marketing strategies. In essence, a webinar is excellent because it is convenient and establishes credibility. Webinars have many applications in training and education, create brand awareness, and have impressive consumer reach.
Here are 11 A/B Tests to perform with your Webinar to get better results.
Attraction: This component serves to plan and attract people to your Webinar and helps increase conversions. When using the attraction factor, you should first research your audience, which suits your topic and value delivery, by identifying which industry, market, or type of viewer will benefit from this topic before delivering your content. Then create an avatar because it narrows down your audience to the ideal prospect who will benefit from your webinar content.
Social Media Posts: The second element you can do is reach out to people with Social Media Posts, which means connecting with your audience via Facebook, Linked In, Twitter or Instagram.
Being social with your prospective viewers is the best thing to increase viewership and engagement depending on the number of social media sites you utilize. The one with the most engagements should be your primary focus.
Webinar Ad Placement: The next thing to consider is the placement of webinar ads on your website. Should you put it right on your homepage or as a sidebar on your blog page, within the blog, or above the blog comments?
The answer is within the blog; above the fold converts better. Ads placed above the fold generate more clicks than placing ads on the comment option or within the blog content.
Webinar Email Reminders: Timing is also essential. The question is, when do you send invitations and reminders to your webinars? Do I make people register and attend the Webinar within the next hour, or do I send out invitations a few days before and send out regular reminders?
I found out that scheduling a Webinar within four days is the best approach compared to scheduling a Webinar within the next hour, as it will provide ample time for your audience to prepare.
As for reminders, sending them out to the attendees will increase your webinar attendance by up to 70%, as there will be people who did not recall having registered.
Webinar Titles: What type of titles convert better?
So, how do you ensure your webinar titles look, feel and sound good? Is there a formula available that can achieve this, or do you gun it and type something, hoping it'll work?
Fortunately, there is a formula to do this. I have tested this formula for a title, and here are some examples.
- How [your target] can [enjoy this benefit] by [method of achieving the benefit] Even if [common objection]
Using this formula allows you to use the best keywords for a more compelling title compared to just coming up with one. The title answers;
The "what's in it for me" question gives one method to deliver the benefit and addresses the obstacles the Webinar will help overcome.
- How Millennials can Still Succeed in Online Marketing despite A Slow Economy
- How You Can Increase Your Business Exposure With SEO And Edge Out The Competition
Visual Clues: In your Webinar, sometimes you will need visual cues to direct your viewers to act on something like clicking on a button for a new lesson or simply clicking the 'like' button. So should you do it?
If you do, kudos! You've just added an element that increases your conversion. This is because a visual cue will help people understand or assist them in doing something, enriching their overall value in your Webinar.
Interactions: Another hack you can include in your Webinar is adding open-ended questions that build discussions among your viewers. However, this depends on the number of viewers.
If your Webinar has a small viewership, then a viable option is to ask a question every 5-10 minutes. However, this can be time-consuming, especially for larger audiences.
A dedicated Q&A session is suitable for Webinar sessions with a medium to a large audience to control the questions and limit the questions you want to answer.
Sometimes, you can also drop a question between segments. Helping viewers stay on track and keeps participation levels high.
Webinar Tone: Webinars work similar to conversations. The last thing you want to express is a robot voice just droning out nuggets upon nuggets of lessons to almost no end.
So it's a wise move to build an intellectual and emotional connection with your audience in the Webinar session.
One excellent and proven way is using emotion. A sparing usage of emotion works because your viewers can relate more to the lessons you share. Emotions also help build authority and confidence among your viewers.
WARNING, too much emotion in your webinars will interrupt the lessons and make you look unprofessional.
Call To Action Buttons: An ideal way to end your Webinar is by making a space for your viewers to decide. This is achieved with a Call to Action or CTA. Your goal at this point is to entice your audience to take action, usually to buy or subscribe to your product.
The recommended method is to include a CTA. You have sold the dream and fulfilled the "what's in it for me" question, much like a sales letter.
Now is the perfect moment to compel your audience to take action.
CTAs work because they give more opportunities for your funnel and future promotions instead of just leaving them on the table.
Great! But how do we get attendees to stay until the end to see the CTA?
One highly effective tactic you can use here is to offer a bonus right after the Webinar session ends to get people to stay with you till the end of your Webinar session. It can be a free download or extra training.
A study found that bonuses increase viewership by up to 60-70% because more viewers will take action at the CTA and creates a valuable opportunity for future business deals. Bonuses can be your way of saying thanks to them for staying until the end. This increases the sense of value your viewers get and builds your reputation of trust.
Slide Formats: Your viewers like variety in your Webinar. What you can do is develop a variety of slides formats to maintain your viewers' interest. Having various graphs, quotes, or even a simple picture is a great way to keep your Webinar's visual and mental direction interesting. The inclusion of graphical cues also keeps your viewers engaged longer.
Follow-up: Follow-up, Nurture and Ascend. These three words have great importance for the future of your business, and it applies to Webinars too. What should you do after it has ended? Do you say goodbye or take this opportunity to engage your audience further?
After your Webinar has ended, you should include ways to further the relationship. A survey right after the Webinar is a great tool to gauge what your audience wants.
In surveys, I have found that open-ended questions work best because they give your viewers a place to share their opinions or feedback. This opens new opportunities for you to explore the possibilities and improve your future webinar session.
However, you can also include close-ended questions like ratings or yes/no questions to generate measurable data from your audience.
Another excellent method to build trust and boost conversion is to send a series of follow-up emails with valuable information. Marketers call it an indoctrination sequence that demonstrates they can help them by actually helping them.
A/B Testing Emails
Emails are the bread and butter for many, if not all, marketers and business owners. One of the easiest and most efficient ways to send emails is using autoresponder services like Kartra, Mailchimp, or ActiveCampaign.
Email and newsletters campaigns are an excellent way to get continuous business and new ones growing in your list. Another plus point is you will be mostly sending them to your qualified list or people who want to receive information from you while also looking out for new ones.
With that said, it is essential that you A/B tests on your emails. But as split testing is far from cheap and financially achievable for many, I'm here to share the hacks for better emails in your marketing.
Before you start, there are best practices that you can take a look at:
- Always test simultaneously to avoid noise factors that can skew results.
- Test a large enough sample size to achieve 95% confidence in the results.
- Make decisions based on data, not your emotion.
- Use online statistical tools available for evaluating and reporting results quicker and easier.
- Test early and frequently for the best results.
- Only test one variable at a time for best results. If you are an advanced Marketer or Statician, you can try more than one variable at a time, look into multivariate Testing instead of A/B Testing.
Email Titles: The subject line is the first thing your list will see in their inbox when you send your email. So it pays that your title is clear, compact, and personalized.
The first element of your email subject line is its clarity. In short, you should write titles that are clear and understandable, not vague or misleading. Sending out emails with clear titles creates rapport, signifies that you're an honest marketer, and ultimately increases open rates and conversions.
Email Title Length: The second element that your email subject lines should have is short or long. This must be a significant consideration as there will be limited space for the titles of the emails to appear on your lists' inboxes. According to the Campaign monitor, business owners should optimize the subject lines to between 41 to 70 characters. This is because the number of characters that appear on the screens across standard devices ranges from 40 to 73.
Personalization: Along with clarity and compactness of your titles, another important converting element is personalization. This is where your email subject lines become personal with your readers.
I will show you one of the simplest ways you can personalize your email subject lines: by using simple words like;
Experian has found that by simply doing this, your open rates can be increased by 29% and 41 % higher click rates than emails without any personalization.
Types of Subject Lines: I can talk all day about what you should write for your subject lines, but which types of subject lines should you write? So here I have the top five subject line types that highly-converts.
Question: A question is an excellent subject line because it gets the reader to stop and pique interest into opening the email. The best questions resonate with the readers' experiences while creating a sense of curiosity. Here are some examples:
- Do you need $1,000 by the month's end?
- Do you wake up stressed or excited?
Questions work because your readers will want to know whether their experiences are similar to others, creating curiosity.
"How To": This is one of the most compelling subject lines as it makes the content very clear. You simply can't go wrong when you use this type. This type effectively increases open rates and conversions because readers will know exactly what they learn upon opening the email. Another add-on advantage is this type plays on the curiosity of your list. Examples are:
- How to make your first $1000
- How to dominate YouTube in 3 simple steps
The key to success with this subject line formula is focusing on the benefit. This is worth trying. When they subscribe to your list, your list will always have a "what's in it for me?" mindset.
Scarcity: Scarcity is, in essence, a powerful driver of human behavior. We are compelled to act when we receive information that something is now in short supply. This angle is excellent for you to use in your email subject lines. For Example:
- Only 48 hours left to get Avatar Genie Pro!
- [Only 3 Seats Remaining] Your chance to see Edmund Loh in person!
TIP: Let your list know of your product's importance to them so that they are compelled to make a decision. If not, they will simply ignore your email.
Numbers: This is my favorite type of email subject line as marketers can use it to set people's expectations about what I'm about to tell them and provide a tease to my email content. And it works; Campaign Monitor has tested subject lines with numbers, and they outperformed ones without them by up to 57% in email opens. So if possible, always include numbers in your subject lines:
- Grow your fan base in just three steps
- Establish a full-scale business in just seven days!
- Your $1660 bonuses for my new software
Remember that if the number relates to the commitment your list needs to give (steps, how long), keep the number low (3 steps, seven days, 60 seconds flat). IF it is related to value, give generous ones like $1000 in 30 days.
Personalized Subject Lines: As I mentioned before, it's always good to add a personal touch to your marketing because it builds a connection to your readers, establishing sincerity. Using the recipients' first name in your subject lines increased the email open rate by 14.68%! Marketers can combine this with other angles like:
- John, do you wake up stressed or excited?
- Ken, here's how to make your first $1000.
- Dominic, you have only 48 hours left to get Avatar Genie Pro!
This works because your readers feel like you're speaking to them. They will feel appreciated by this gesture and will reward you with open rates and conversions.
Email Content: After the email subject line, the next part you should look at is the email content. In essence, the content of your email is where you will tell the story to your readers. This is because the best emails tell stories that enable the readers to relate them to their life.
Opening Lines: Many times, what we see on opening lines are greetings like 'Hi' or 'hey, it's Edmund here.' This greeting establishes friendliness and signifies an actual person is writing the email. I will show you three more engaging ways of opening sales emails than the standard name-and-company introduction.
- 'Did you know...': The first is 'did you know that...'. This is effective because starting your email with a thought-provoking statistic related to your readers' issues encourages them to read more and possibly turn it into a buy.
- Question: asking a question at the start of your email is a great way to get your readers curious. They will wonder if there is an answer to it further in the email. If the question hits home, the chances of them converting will be higher. One example is: Is getting $1000 income in 7 days possible?
- Scarcity: A scarcity opening line usually follows a scarcity-angled subject line. The primary function of this is to elaborate more on the limited availability of whatever you're promoting at the time and details things like 'only three seats left' or 'limited time discount.'
- Storytelling: This is our basic foundation of communication. So what makes an email with a story so attractive to your readers? Storytelling stirs emotions, is personal, simple, and shareable. This is why marketers use storytelling; it invokes a sense of relation to their list as it is relatable to the product. Putting a story into your marketing emails can help with conversions. Companies like Jack Daniels, Uber, and American Express use it, so why aren't you?
Problem Statements: The main goal of marketing a product is solving a problem. The most effective style of stating issues is to give the problems your readers have not noticed before but is crucial to be solved. For example:
- Are you still struggling day-to-day with your financials, even though you feel it's enough?
- Are you still feeling that your products don't perform well, even though you regularly create and market them?
By stating these problems in your email, you can get your readers to say, 'why didn't I think of that?' It gives them an "aha" moment, which can be the gateway for them to convert.
- Introducing a Product: After you have stated the problems your readers may have if they don't have your product, you have now primed them to seek a solution for their previously-unseen problems. If this isn't exploited, you will miss a huge opportunity to introduce your product. By introducing the product in your email, you have significantly reduced the amount of effort your readers need to find a solution. In other words, it is right there in the email; no need to seek elsewhere. This realization can increase conversions and increase the sales of your product.
Feature & Benefit: As the saying goes: 'Features tell, benefits sell.' This is very true to emails, too, because they, like sales letters and videos, act as a promotion tool from a different light. According to case studies conducted by Wishpond, if you include benefits (and features too) inside your email, the conversion rates can go up to 65% because benefits give experiences to your readers, an edge that features don't have. Examples of benefits are:
- Creates impressive graphics in just 20 seconds
- Sets up a fully working sales funnel in just four clicks!
- Speeds up your content creation time by up to 75%
- Email Tone: The tone of voice also exists in emails, not just spoken words. Written words get trickier as there's no element of speech. What I'm about to share here is the negative and positive tone. Usually, emails are written in a very positive tone to give the product a good light. But that's not always the case; a positive tone can indeed convert well, emails with a negative tone can shock and surprise people. This deviation from the norm is called 'pattern interrupt,' a technique established marketers use to great success.
Email Angles: You don't simply send a single email and hope it will do all the work for you and bring you cash overnight. You need some finesse in this department. The best tactic to do here is to play with angles. There are many, many angles you can use for your emails. But I'll show you the three best ones:
- Pre-Launch Angle: This email is sent a few days to 24 hours before the actual product launch. It is usually short and straightforward, with only a few sentences announcing that you will launch a product soon. This type of email is highly converting because it creates excitement for your readers and makes them ready to buy or pre-order.
- Hard and Soft Sell Angle: A hard sell email is a type of email that uses direct selling language. A hard sell is designed to get your readers to buy right now. This is proven to work well with readers who are ready to purchase and aren't deliberating anymore. A soft sell email builds a relationship, context, and story first before selling. These emails find passive ways to show them that you have the solutions they need. According to a study released by New Century Media, consumers were 30% more willing to buy a product through non-direct advertising rather than media advertising. This is because the readers feel that the email is more personal and the tome is more acceptable than a hard sell.
- Scarcity Angle: This angle is most famous as the last email sent in a marketing campaign. The specific content usually has this: there's only limited time left to get product X. This type of email works well and converts highly because it motivates readers to take action now. The reason behind this is scarcity increases the value of something when it has only limited quantities.
When To Send?: Timing is essential when sending emails. For example, you'll never get the open and click-through rates you hope for if you promote Christmas-related products in March. To help you get started today, follow these two tips below regarding the timing.
- Time of the Day: The best time to send email in the day is around 8–10 am. MailChimp had found that at this time, 7% of CTR was achieved when marketers sent emails around this time. This is most probably because many people start their day around this time and take time checking their emails.
- Time of the Week: The most open rate is achieved on a Tuesday, according to GetResponse. The percentage was recorded at 19.9%. They also recorded that the highest click-through rate was 4.9% on Friday. The click-through rate achieved that level probably because the emails get engagement level on the weekends as well.
Call to Action: Every marketing email will have a call to action (CTA) inside. A CTA is a request to your reader to take action. In plain text emails, a CTA can be a line of text with a hyperlink, while others use an image of a button. A CTA is a crucial part of an email since it tells your readers it's time to act. So here are some proven converting CTAs.
- URL Vs. Line of Text with Hyperlink: Usually, with plain text emails, the CTA is a simple line of text. Marketers recommend using a hyperlink instead of a URL because few people understand them. When they see a hyperlinked text, they will know that this is the place to click to find out more about the product.
- Number of CTAs: I have found that the best number of CTAs in an email is only two, with three CTAs the highest number. The first CTA should be seen right away (or above the fold), while the second serves as a reminder to take action now. Longer emails may require three CTAs so that readers have the button nearby when they decide to click through. Any more than 3 CTAs is highly not recommended because it will make the email seem desperate for clicks and drive your readers away.
Right now, with 1.8 billion users, Facebook is simply too big to ignore. And the way to reach them with your promotions is through Facebook Ads. With such a vast platform, Facebook Ads is the way to go right now because it gives you a highly scalable content promotion, can target your prospects, and rake in more conversions with its powerful remarketing feature.
I will share with you hacks that increase your marketing power on FB Ads.
URL or Text With Hyperlink: First up, when redirecting your prospects to your promo pages or other sites, do you put in a website URL or turn a line of text into a hyperlink?
Results show that hyperlinks convert better than web addresses. The reason is Marketers can redirect your prospects to the website with just a click rather than having to type out the full web address to go to the website.
Ad Position: The second part focuses on ad positioning. This test determines whether your ads convert better when they appear in the right column versus the News Feed.
The result is: your ads convert better when they appear on the News Feed rather than the right column. The News Feed is better because most people focus more on their News Feed than the right column ads on their desktop. Plus, the right column does not show on mobile screens.
Call To Action Buttons: In your FB Ads, do you put in a call-to-action or CTA? Is it needed for a simple ad on a person's profile? At this point, I think the answer is obvious: Ads with a CTA convert much higher than without a CTA. This is because, as with other promotion efforts, a CTA gives a space for your prospects to take further actions if they like what you offer in your ad, like signing up or adding your product to their cart.
Speaking of CTAs, which CTA button works the best in your ads? Should it be direct or worded more tactfully? So here, I'll share with you one example of a high-converting CTA. The test explored the conversion rate between a "Buy Now" versus the "Learn More."
The result is, "Learn More" button converts better than the "Buy Now." This is possible because of consumer behavior. The "Learn More" is less risky. The consumers' first step is non-commital.
Now we move on to the buttons themselves, more specifically, the color of the button. Which color converts? Orange or green?
Test results have established that orange converts best. This agrees with color psychology which suggests orange appears to capture attention better than other colors. Furthermore, orange is a friendlier and warmer color.
Compelling Words: Power words are words that compel your prospects to click through. Some examples are "free", "new", "now", and "instantly". Test results for headlines or sub-headlines with these words bring in higher conversions than those which don't have them.
So why is this possible? These words represent the edge your products have over others and effectively describe it with a single word. And as for the power word "free," the answer's simple: everyone loves free stuff!
Scarcity: The next element you should look at is the element of scarcity, which in essence, tells your prospects that something won't be available easily or is in short supply. Scarcity works for other platforms because if a product is perceived as scarce, it will have a sense of exclusivity. Sales will also increase with the scarcity effect.
It turns out scarcity also has a significant effect on FB Ads. Ads with the scarcity angle convert more than a typical ad. There are two reasons for this: Your prospects will perceive limited items to have more value and the fear of missing out if they don't click through soon.
Headline Length: The choice is yours to write a long title or a short title for your promos. Both types are proven to be highly converting, depending on what kind of promotion you're doing. FB Ads is a unique platform with distinct headline writing techniques. Tests have pointed out that longer descriptive titles convert better than short ones.
This comes down to being transparent. Descriptive titles allow your prospects to read through and clearly understand what the ad is about right away. This will generate interest and clicks.
Ad Headline Type: What kind of titles do your prospects react best to in your ad pieces? Is it a regular title or a title that snatches their attention? Should you ask a question in your title outright? Being descriptive in your headlines is one thing, but asking questions does not harm you.
Headlines that ask questions do get higher conversions than regular titles. They succeed in attracting your prospects' attention because questions trigger their curiosity. It also leaves an open loop for your prospects to fill in rather than a closing statement.
Social Proof: In social media, the most significant driving force is the social element. No post is ever considered introverted. It always tries to reach out. With FB Ads, ones with broad reach have better conversions, and it can be shown in the form of social proof. So I suggest your include social proof like testimonials in your ads as they convert well. Success stories are great; they give others confidence that it might work for me if the product works for others.
Visual or Text: In your ads, there is a chance that you have included an image to accompany your text somewhere on the ad piece. So the question is, does including only pictures in your ads gain conversions?
The answer? A big Yes.
Images have a massive appeal to them, rather than plain text. Ads with images feel authentic to your prospects. This feeling sends a message that your ad is honest and, therefore, worth checking out.
Images increase click-through by 94% and only take a fantastic 13 milliseconds to understand. Plus, photos, especially for your products, are 67% more important to consumers than detailed information, descriptions, and customer ratings.
Generic or Specific Images: So, do your ads feature a generic or specific image to accompany your promotional text? The result of this test shows that targeted images bring in better outcomes than generic or unrelated images. A particular image strengthens the ad's clarity, allowing your prospects to have a better idea of what you're advertising.
Single or Multiple Images: So what about the number of images? Does a single image work better than multiple ones?
The answer is Images that are arranged in a slider form convert higher than single image ads. Multiple related images make the visual elements of the ad strong and spot on. Imagine a very interesting galley. That's where your prospects will flock too.
Images With or Without People: In your ads, do you portray only your product or show it in use by people?
The result of this test is, ads portraying your product being used convert better. This is due to a higher human feel to the ad. Your prospects will perceive these kinds of ads to be user-friendly.
However, if you can't show your product in use, you can use pictures of your satisfied customers. It works wonders as the images can cut bounce rates by 27% and increase completions by 36% when tested on landing pages.
Video Ads: How about videos? Are they effective in your FB Ads compared to images? Tests found that video ads convert better than standard ads because videos can compress more information in a shorter period, and the result is more interesting.
Consider this, 85% of Facebook videos are viewed because dynamic visual elements attract the interest of your prospects more than images.
Who Can Perform A/B Testing?
Before today, your best bet for getting the best results from conducting an A/B Test is to hire a Statistician or Conversion Rate Optimizer. But because of software tools, entrepreneurs and small business owners with no statistical experience can perform A/B Testing.
If you're looking to hire a Conversion Rate Optimization Specialist as an employee to your company, the average national annual salary is $76,676 or $36 per hour.
If you prefer to avoid the hassles of hiring an employee, you can hire a conversion rate optimization agency. Depending on the agency you employ, the service fee can range from $2,000 to $30,000 per month.
A low-tier agency with one to two staff employees will charge between $2,000 to $5,000 per month. A mid-tier agency with more than three years of experience with a proven track record will charge between $6,000 to $8,000. And a high-tier agency with seven-plus years and an employee size of more than ten employees will charge between $10,000 to $30,000 per month.
But as I stated before, with the software tools available today, you could perform A/B Testing yourself.
Top Ten A/B Testing Software
If you decide to perform A/B Testing yourself, here is a list of the top ten recommended software used by Conversion Rate Optimization Specialists.
- Convert Experiences
- AB Tasty
- Google Optimize
- Adobe Target
Now you might also have A/B Testing features in with your CRM software, or sales funnel software.
These features are good to start but can become limited as you progress in experience and begin to have a stronger desire for more data and control in selecting winners.
For a complete review of all the A/B Testing software listed on the Internet, read the A/B Testing software reviews by G2 Business Software Reviews.
So what are key features to look for when selecting the right A/B testing software?
Here are our recommended must-have features when selecting the right a/b testing software for small business owners.
- Run A/B Experiments
- Detailed Results and Statistics
- Templated A/B Tests
- No Need To Monitor Experiments
- Test Individual Elements, Not Just an Entire Page
How Does A/B Testing Work?
A/B Testing, when performed correctly, will yield good results. But when done incorrectly, you can make a wrong decision, which will affect your business. This section will make things clear for you.
So, if you want to run an A/B test accurately and get long term results, you need to do three things first:
- Have A Clear Goal: Is your goal to drive traffic? Is it clicks? Or even opt-ins? A great result is accomplished by setting a clear goal at the start. An email subject line that wins at an open rate may differ from the winner for click-through rates. Define your plan first.
- Only One Variable Tested: Only one variable gets tested for the A/B test to deliver a conclusive result. For example, you published two bonus pages. Bonus Page A at 10:00 am on Monday with a green-colored buy button, and Bonus Page B, with an orange button and posted on Monday at 1 pm Bonus Page B gets the higher click-through rate. In this case, because there are two variables, you won't know what made Bonus Page A lose: the button color or timing?
- Random Audience: If your leads list sees Video Thumbnail A and only your buyers' list sees Video Thumbnail B, can you say that Thumbnail A is better if it wins an A/B test with confidence? You can, but it won't be effective since different groups react differently. The recommended method to perform an A/B Test, apart from analyzing the effect of changing just one attribute, is to test your variable with an utterly random audience.
Once you design your A/B test based on the three things explained above, it's time to set up the test in the A/B Testing Software you use. When enough data is collected, your software will declare a winner based on your selected confidence interval.
A/B Testing Mistakes To Avoid
A/B Testing is a simple and powerful concept in marketing and something that every marketer should understand. Unfortunately, this is not always the case for DIY small business owners who implement marketing themselves.
By using A/B Testing repeatedly, you can make a series of data-driven decisions that will guarantee to help your site climb the ranks of Google.
But while this can be hugely effective, it is essential to make sure you don't make any common mistakes. A/B Testing done well is game-changing for all the right reasons. A/B Testing done poorly can result in incorrect data at best and ultimately destroy your profits!
Read on to learn what some of that area, and how you can avoid them.
Generalizing Your Findings
A common mistake Small Business Owners make when performing A/B Testing is to generalize their results to all websites. Experienced A/B testers know that a technique that works on one website won't necessarily work on another.
We must keep testing new strategies to ensure we do the best thing for that specific site to become successful.
Skipping a Canonical Tag
When testing two versions of a webpage, often we use a redirect to send 50% of traffic to one page and the other 50% to the other page. If the redirect isn't done correctly, we can confuse Google search bots, leading to both pages losing their search engine ranking.
To help Google understand we are testing pages, we have to include the canonical attribute on one version of the webpage. This earmarks that page as being the main version, which in turn will ensure that Google doesn't think we're trying to pull the wool over its eyes or spam the SERPs.
Not Running the Test Long Enough
Another common mistake is not running a test long enough to collect enough data to draw a conclusive difference between the two versions tested.
We run a test long enough to collect enough data to declare statistical significance in the statistical world. Meaning we have enough data to conclude with high confidence that version a or b is either different or not different.
Forgetting To Archive
Finally, you should ALWAYS make sure to archive your web pages to ensure you can revert to the original. Small business owners often forget to archive their changes and cannot quickly get back to their original page.
How Long Should You Run an A/B Test?
In SEO and internet marketing, A/B Testing is one of the most powerful techniques available to marketers. An A/B test means taking two slightly different web page or website versions and seeing which one performs best, allowing marketers to make profitable data-driven decisions.
Unfortunately, small business owners don't understand how statistical significance works and end up collecting the wrong sample size, leading them to make a decision from bad results.
So how big of a sample size is required?
Why Length Matters for A/B Testing?
The ideal answer to how long you should run an A/B test is, indefinitely, because it will produce more accurate results. But no one has the budget or time for an indefinite test.
If we were to run an indefinite test, we would avoid what are called confounding variables. A confounding variable is an unknown variable that isn't accounted for in our testing, directly impacting our results.
For example, you have two versions of a website. On version A, you use "Headling A," and on version B, you use another Headline. You hope to see if one of the headlines improves the number of times people stay on your website.
50% of the visitors are sent to version A, and half are sent to version B.
You run the test for 48 hours.
By coincidence, many people who go to version A of the website are from California. And unknown to you, the people from California are having internet issues due to weather.
This unknown weather pattern in California can skew your results.
If you run the test for five weeks, then the likelihood of this being a sheer coincidence will shrink remarkably. You can never be 100% sure, but you can be more confident the more prolonged the test runs.
Don't lose hope; there is good news. Using a chi-squared test, we can determine how long to run a test by a method called testing for significance. This way, you get a result called P-value. The P-value represents the possibility of your experiment being invalid. As long as the P-Value is lower than 0.05, representing a 95% Confidence Interval, you are good to act on the information.
When A/B Testing Works - Understanding Statistical Significance
Are you asking why you should go through all the trouble of creating an A/B test?
Is A/B Testing a fancy way of describing a kind of public beta test?
As mentioned before, A/B testing is a proven scientific strategy that allows researchers, engineers, scientists, and marketers accurate data to make decisions concerning factors that influence their decision-making. Without using this technique, you are simply guessing, and could your guess could be wrong, which could cost you significant profits in the short term.
Let's look at an example.
When a doctor wants to know if a particular drug will help cure an illness, they will conduct a scientific study with a control group and an experimental group to avoid the impact of unknown confounding variables. The control group is the one that gets the drug, and the experimental group is the one that doesn't.
This allows the doctor to compare the results from the two groups accurately within a certain confidence level.
As an example, one person using the medicine has added a vitamin program. How will you know if the vitamins are the cure rather than the treatment? The introduction of the vitamins influences the results. Therefore we cannot conclude with high confidence the drug cured the illness.
When A/B Testing Goes Wrong
So an A/B test is impervious to error, right? Wrong.
Anyone who has done a science or math degree will already know the problem with this strategy is; the sample size. The sample size is the number of data points in the study.
To illustrate again, using the drug example, if you have one person simultaneously taking vitamins in addition to the drug in the control group. You can't be certain that the person taking medicine didn't also happen to get over a cold that same day because of the vitamins. It looks as though they saw a boost in performance, but in fact, it was just a coincidence.
The same thing goes for running A/B tests in marketing. If you have two versions of a website and one enjoys a more significant boost in SEO than the other, this could result from one page having lower bounce rates than the other. That means that people didn't spend long on one version of the page, which impacted the page being demoted in the SERPs.
But what if the one page had higher bounce rates because people who went there happened to be in a hurry.
If you only ran the experiment for one day, and you had ten people go to each site, then it's statistically unlikely that all the people on one of the pages just so happened to be in a hurry. But it's not impossible.
And this is where the idea of 'statistical significance' comes in. To rule out freak coincidence and chance, you need a large enough sample size to be able to state with 95% or 99% certainty that the results of your test are accurate.
Suppose you flip a coin and get heads ten times in a row. That's an improbable outcome, but it's possible. But if you flip a coin and get heads 100 times in a row, that is so astronomically unlikely to be a coincidence that you can probably assume the coin is biased.
The same goes for your A/B test. Run the test long enough, and the results become increasingly likely to be accurate. This is what we refer to as 'statistical significance.
You can run tests to find out if your results are significant, which will give you a 'P' value. This P-value tells you the percentage likelihood that the outcome was a fluke with anything less than 5% (P < 0.05) considered 'significant.'
For now, the main 'take home' is that you should make sure you run your tests long enough, don't draw premature conclusions, and never assume that the outcome is 100% guaranteed to be accurate.
IMPORTANT: To be highly scientific about this, you can always run the chi-squared test. This is a test of significance, which is a calculation that takes numbers, crunches them, and provides a P-value. You can use the number of impressions on each page or the page ranking over a given amount of time.
Nothing is Certain
Warning: Keep in mind is that search engine strategies change. A great SEO strategy can change on a dime once Google decides to change its mind. If you're comparing two different pages for their SEO positioning, keep in mind that what works now might not work in the future. That's why it's vital that you keep on testing and experimenting and always be cautious when adopting new strategies wholesale.
Free A/B Testing Significance Calculators
Here is a list of free tools you can use to determine if your test has significance quickly.
- VWO A/B Test Significance Calculator
- Survey Monkey A/B Test Significance Calculator
- Splitly A/B Test Significance Calculator
- Dynamic Yield A/B Test Significance Calculator
A/B Testing Checklist
Now let me summarize everything for you in a friendly step-by-step checklist so you can get started today.
Decide on what marketing variable you want to test.
- Headline / Subject Line for Emails
- Sub-headline / First Line In Body of Email
- Set up your experiment through your preferred software tool
- Run a test until you reach at least 95% statistical significance. Use your preferred software tool or one of the free A/B test significance calculators. You can also determine the sample size required to achieve statistical significance with the following free tools below.
- Pick the winner and update your material. If there is no winner, then go to step 5.
- Start a new A/B test.