Why startups should, and how to run a landing page testing

Lately, when I discuss a new plan with a potential business partner, I mention that we should run a landing page test before settling on the idea. She wasn’t too convinced at first, and I had to explain why it was important. This makes me think that not everyone, even those in the startup space, is completely aware of the importance of testing.

So while I’m putting together my thoughts to make the plan more comprehensive, I’d like to share it publically and exchange insights with some other peers.

Why is landing page testing important?

Before we talk about landing page testing, it's important to align on one thing: a successful startup is established because it often found the right, burning problem and solved it with a creative new solution.

It is very rare for a startup to find the perfect problem or solution in one attempt. Or sometimes even if they find the right problem and solution, they could be selling the solution to the wrong customers. The product-market fit issue often occurs in one of three ways: “solving the wrong problem,” “providing the wrong solution,” or simply “selling to the wrong customer group.”

Imagine you have a brilliant idea for a problem you accidentally discovered, and you're sure people are going to buy it. You invest $10K to build the product and try your best to sell it, then you find out that one of your early assumptions was wrong: the problem wasn’t that prevalent or the solution wasn’t needed. It would be costly, time-consuming, and financially unwise to iterate on the idea at this point, because you just spent all your money and time building the wrong one!

This is why most startups run several tests before building the product, such as hosting a focus group or user testing with a prototype. Landing page testing is one of the most popular testing mechanisms across different startup fields. It is sometimes called the “non-functional MVP test”. Because in testing like this, you demonstrate everything you have for your business, set the marketing tone, and explain your solutions just like a real business, except the company is not operating.

Numerous successful startups had great landing page testing stories. Personally, my favorite was a car rental company Zipcar (it’s my favorite because I signed up in their pre-launch). In the testing phase, you can go to their website and read through the product descriptions and instructions on how to use them. When you try to book it, it asks you to specify which city you are in and then shares a notification that they are not available in {city name}, but they would like to notify you when they are ready. That’s how they determine which city will have the most demand, so they don’t just blindly choose a city to launch, but pick the high-demand ones.

So now, how do you plan a landing page test?

Step 1. Identify what exactly needs to be tested.

As mentioned above, the most common product-market fit issues occur at one of three points: the wrong problem, the wrong solution, or the wrong target audience. Identifying the right problem is often the first step to a successful business. Therefore, in most cases, startups at the ideation stage should focus on testing the problem or how to position it. If the problem is less of an issue (e.x. it’s prevalent), then focus on the solution options: would people pay for this particular solution to solve their problem?

Step 2. Identify what the testing options are

Once you identify a focus, the next step is to list the possibilities.

Testing a problem

If you are testing the problem, what assumptions and learnings have you already identified? For example, my startup Ferne Health has a very obvious problem statement — women have more health issues than men. But what are the health issues? Based on our research and past operational experience, we think women at {age group} in {market} commonly have issue A, issue B, issue C, and possibly issue D. Then our landing page should highlight the four different options and pass a clear message that we are here to help with the problems. Then the metrics to compare should be the click rate, registration rate, or content viewing rate for each option.

Testing a solution

Similarly, suppose you are confident in your ability to solve the problem but are offering solutions for the first time. In this case, you could list out all solutions to the issues on your landing page to compare consumers’ interests.

For example, if your problem statement is: women in the {age group} in {market} have trouble choosing skincare products. The solutions you have identified include: A. building a new skincare brand that fits this group’s lifestyle; B. creating a subscription program to offer different selections each month; C. developing a review platform for customers to leave candid reviews. (P.S. All three options here had legit business cases). You can either set up three different pages, run ads, compare the responses, or create a page that offers three different programs (which might not be the best testing case in this specific scenario) and compare the click rate.

Step 3. Identify the metrics and mechanism.

One common question people ask is how to stimulate the actual customer behaviors when the services are not available to use.

There are things to keep in mind: 1). it’s an experimental test, so there will always be a bias caused by how the test is set up, just like every other research method, but smartly designing the landing page can minimize deviations; 2) the landing page should only be used to test for one or two things at a time so that it wouldn’t cover every aspect of the business. But you could always run multiple tests or combine the landing page tests with other research methods.

For example, if you test for solutions but are also curious about pricing. You can use a landing page to test which solution receives the most sign-ups, and then run a survey or user interview with the signed-up users to understand their expectations of prices. This way, you could also get more accurate pricing insights by narrowing down your customer demographic to those who are already interested.

The testing mechanism here refers to how to simulate a real business experience and customer acquisition funnel. For example, suppose your business plans to offer a booking service. In that case, the landing page should be tailored in a way that’s close to your ideal booking flow, and the final CTA (call to action) should be “Book Now” or “Pre-book” for people to take action to book. If your business is a SAAS tool, the final CTA might be “Start Trial” or “Request Demo.” And more commonly, many platforms use “Sign up,” “Subscribe,” or “Get Notified” to capture an email list that can be used to send launch news later.

Common metrics for landing page testing include the click rate, number of email sign-ups, number of people who attempted to place an order, and number of pre-orders (if available). Choosing the right metrics is crucial to drawing the correct conclusion. Sometimes multiple metrics are needed to evaluate the test. For example, issue A can have the highest click rate (more people are interested in this issue). Still, issue B can have the highest pre-booking (more people are willing to pay). In this scenario, a further evaluation might be taken to assess whether issue A is not monetizable or the solution is not a good fit.

Step 4. Identify the content on your landing page.

Once you've selected the testing options, testing mechanism, and key metrics, the next step is to set up a page, visualize your solution, and post the right marketing messages. If you or your team has the resource to build everything on your own, it’s a pretty straightforward process that requires some design and dev work. Website builders like Wix, Squarespace, Webflow, or even Notion can all be used to set up a webpage quickly in a day. If you don’t have the right resources to build such a page yourself, there are tons of freelancers on Upwork you could hire.

Step 5. Launch, promote and track

Once the landing page is ready, the next step is to promote it. Paid ads — Facebook Ads or Google Ads are the most common and fast way to capture new users and, of course, in this scenario, to stimulate the real acquisition experience as well.

However, if your business is in a different field or has already established other acquisition approaches, such as communities or email lists, sending the page out via those channels can help you get a much cheaper and quicker result! But be mindful that if your community or channel is leading to a bias, you may not want to serve only this group.

There are multiple tools to track the performance of the testing. Google Analytics is the most commonly used one. If you run Facebook Ads, you can use the Facebook Pixel and Event Manager to track the performance. If your page is built on a website builder like Squarespace or Wix, there are default tracking tools available. I use Fullstory to track the performance because it also captures the actions on the page. Still, the limitation is that it doesn’t capture all customers. Of course, if your landing page has a CTA to let people register, the number of sign-ups, bookings, or pre-orders are often the most important ones.

Step 6. Iterate and continue

You should set up a testing period before launching the page: such as running it for two weeks or a month. Then a deep analysis of the testing performance should be measured. Testing is not a one-stop thing, but a continuous process. So, it's common for people to iterate on a few components on the page after testing to continue the test until all needed insights are collected.