Should You Beta Test Your SaaS or Go Straight to Market?

if you've ever launched software or if you've ever launched software or you're thinking about launching a you're thinking about launching a you're thinking about launching a software product a big question on your software product a big question on your software product a big question on your mind is should we beta test it or go mind is should we beta test it or go mind is should we beta test it or go straight to Market there's some pros and straight to Market there's some pros and straight to Market there's some pros and cons to this I'm Rob Walling I've cons to this I'm Rob Walling I've cons to this I'm Rob Walling I've started six companies five of them started six companies five of them started six companies five of them bootstrapped I've written four books on bootstrapped I've written four books on bootstrapped I've written four books on entrepreneurship and I've invested in entrepreneurship and I've invested in entrepreneurship and I've invested in more than 150 startups so if you're more than 150 startups so if you're more than 150 startups so if you're watching this video you're probably watching this video you're probably watching this video you're probably wondering how should I roll out the wondering how should I roll out the wondering how should I roll out the first version of my product and a lot of first version of my product and a lot of first version of my product and a lot of people think of a beta test or let's go people think of a beta test or let's go people think of a beta test or let's go into beta so we can get it in users into beta so we can get it in users into beta so we can get it in users hands and they can find bugs for us and hands and they can find bugs for us and hands and they can find bugs for us and that's what Google did back in what 2005 that's what Google did back in what 2005 that's what Google did back in what 2005 when they launched Gmail and there's a when they launched Gmail and there's a when they launched Gmail and there's a famous joke that Gmail was in beta for famous joke that Gmail was in beta for famous joke that Gmail was in beta for years and years and Google kind of used years and years and Google kind of used years and years and Google kind of used it as an excuse to say oh that's broken it as an excuse to say oh that's broken it as an excuse to say oh that's broken that's because it's a beta having a beta that's because it's a beta having a beta that's because it's a beta having a beta test or going into beta with beta users test or going into beta with beta users test or going into beta with beta users it really is kind of a dated concept it really is kind of a dated concept it really is kind of a dated concept these days I would say don't beta test these days I would say don't beta test these days I would say don't beta test test internally meaning your beta test internally meaning your beta test internally meaning your beta testers should be you and your testers should be you and your testers should be you and your developers and your team and you find as developers and your team and you find as developers and your team and you find as many bugs as humanly possible and then many bugs as humanly possible and then many bugs as humanly possible and then you start your Early Access with your you start your Early Access with your you start your Early Access with your first potential customers Early Access first potential customers Early Access first potential customers Early Access is a concept I first heard about from is a concept I first heard about from is a concept I first heard about from peldy gilzoni the founder of balsamic peldy gilzoni the founder of balsamic peldy gilzoni the founder of balsamic which is a very successful software which is a very successful software which is a very successful software company and pelity is a very thoughtful company and pelity is a very thoughtful company and pelity is a very thoughtful founder and the idea behind Early Access founder and the idea behind Early Access founder and the idea behind Early Access is that you charge for it you get your is that you charge for it you get your is that you charge for it you get your first 5 10 20 potential customers into first 5 10 20 potential customers into first 5 10 20 potential customers into your product and you're not looking for your product and you're not looking for your product and you're not looking for them to discover bugs testing your them to discover bugs testing your them to discover bugs testing your software is your job it's the job of software is your job it's the job of software is your job it's the job of your team whether you use a ton of unit your team whether you use a ton of unit your team whether you use a ton of unit tests integration tests whether you hire

tests integration tests whether you hire tests integration tests whether you hire QA people which I'm going to be honest QA people which I'm going to be honest QA people which I'm going to be honest I've had millions in SAS ARR with zero I've had millions in SAS ARR with zero I've had millions in SAS ARR with zero QA testers because myself and the rest QA testers because myself and the rest QA testers because myself and the rest of my team handled the testing and we of my team handled the testing and we of my team handled the testing and we had such extensive unit test coverage had such extensive unit test coverage had such extensive unit test coverage and we had such high code quality that I and we had such high code quality that I and we had such high code quality that I think bringing Q8 testers on early can think bringing Q8 testers on early can think bringing Q8 testers on early can be a crutch for developers to not test be a crutch for developers to not test be a crutch for developers to not test the code it can be a crutch for product the code it can be a crutch for product the code it can be a crutch for product people to not test the features and so people to not test the features and so people to not test the features and so it bugs me when people say well I'm it bugs me when people say well I'm it bugs me when people say well I'm going to do a beta test what that means going to do a beta test what that means going to do a beta test what that means is I'm going to let my customers my is I'm going to let my customers my is I'm going to let my customers my potential customers find bugs that's a potential customers find bugs that's a potential customers find bugs that's a terrible idea the software we're terrible idea the software we're terrible idea the software we're building these days the SAS product building these days the SAS product building these days the SAS product you're going to build is just not that you're going to build is just not that you're going to build is just not that complicated you're not building Gmail in complicated you're not building Gmail in complicated you're not building Gmail in 2005 when it was pretty hard to do this 2005 when it was pretty hard to do this 2005 when it was pretty hard to do this remember Google Maps when it first came remember Google Maps when it first came remember Google Maps when it first came out how complex it was all the Ajax and out how complex it was all the Ajax and out how complex it was all the Ajax and the JavaScript that ran in the browser the JavaScript that ran in the browser the JavaScript that ran in the browser that we had never seen that's the kind that we had never seen that's the kind that we had never seen that's the kind of thing where you need thousands tens of thing where you need thousands tens of thing where you need thousands tens of thousands or millions of people to of thousands or millions of people to of thousands or millions of people to test a piece of code across tons of test a piece of code across tons of test a piece of code across tons of browsers and devices in order to find browsers and devices in order to find browsers and devices in order to find these weird Edge case bugs that's not these weird Edge case bugs that's not these weird Edge case bugs that's not what you're going to be doing I would what you're going to be doing I would what you're going to be doing I would never call an early release to my never call an early release to my never call an early release to my customers or potential customers a beta customers or potential customers a beta customers or potential customers a beta test I always call it an early access or test I always call it an early access or test I always call it an early access or something of that nature to imply that something of that nature to imply that something of that nature to imply that you are getting early access to a you are getting early access to a you are getting early access to a product that is mostly fully baked and product that is mostly fully baked and product that is mostly fully baked and you are paying for the privilege having you are paying for the privilege having you are paying for the privilege having a beta of your software is a bit of a a beta of your software is a bit of a a beta of your software is a bit of a crutch in my opinion and I think a lot crutch in my opinion and I think a lot crutch in my opinion and I think a lot of folks used to give beta users free of folks used to give beta users free of folks used to give beta users free lifetime access to their SAS and that's lifetime access to their SAS and that's lifetime access to their SAS and that's a big mistake imagine if you've worked a big mistake imagine if you've worked a big mistake imagine if you've worked hard for months and months to find 10

hard for months and months to find 10 hard for months and months to find 10 interested parties who want to purchase interested parties who want to purchase interested parties who want to purchase your software and then you give them all your software and then you give them all your software and then you give them all free lifetime access doesn't make any free lifetime access doesn't make any free lifetime access doesn't make any sense at most I would consider giving sense at most I would consider giving sense at most I would consider giving them a temporary discount maybe a them a temporary discount maybe a them a temporary discount maybe a discount off the first year maybe a discount off the first year maybe a discount off the first year maybe a discount off the first six months as discount off the first six months as discount off the first six months as thanks for giving you feedback but the thanks for giving you feedback but the thanks for giving you feedback but the bottom line is if folks have the ability bottom line is if folks have the ability bottom line is if folks have the ability to gain early access to your software to gain early access to your software to gain early access to your software and input and interaction directly with and input and interaction directly with and input and interaction directly with you the founder and they can impact the you the founder and they can impact the you the founder and they can impact the future of the product itself that alone future of the product itself that alone future of the product itself that alone is a nice privilege and so any discount is a nice privilege and so any discount is a nice privilege and so any discount you would add on top of that is a bonus you would add on top of that is a bonus you would add on top of that is a bonus another question you might be asking another question you might be asking another question you might be asking yourself is how do I know when my yourself is how do I know when my yourself is how do I know when my product is ready for Early Access and product is ready for Early Access and product is ready for Early Access and you know when it's ready when code is in you know when it's ready when code is in you know when it's ready when code is in production and you have run through all production and you have run through all production and you have run through all the scenarios that you can think of to the scenarios that you can think of to the scenarios that you can think of to try to break it if you've never tested try to break it if you've never tested try to break it if you've never tested software if you're non-technical that's software if you're non-technical that's software if you're non-technical that's fine go learn how to do this this is one fine go learn how to do this this is one fine go learn how to do this this is one skill that as a SAS founder you need to skill that as a SAS founder you need to skill that as a SAS founder you need to know this is not something you're going know this is not something you're going know this is not something you're going to Outsource from day one maybe a year to Outsource from day one maybe a year to Outsource from day one maybe a year two three years down the line you wind two three years down the line you wind two three years down the line you wind up hiring a QA tester someone to help up hiring a QA tester someone to help up hiring a QA tester someone to help out with it maybe you have a developer out with it maybe you have a developer out with it maybe you have a developer who's a good tester you also want to who's a good tester you also want to who's a good tester you also want to build small features and ship them build small features and ship them build small features and ship them frequently such that you don't get 10 frequently such that you don't get 10 frequently such that you don't get 10 000 lines of code shipping to production 000 lines of code shipping to production 000 lines of code shipping to production because that's when it's easy to get because that's when it's easy to get because that's when it's easy to get bugs into production but frankly my bugs into production but frankly my bugs into production but frankly my motto was always ship early ship often motto was always ship early ship often motto was always ship early ship often get things into production and test the get things into production and test the get things into production and test the crap out of them before they make it crap out of them before they make it crap out of them before they make it there software is complex building good there software is complex building good there software is complex building good software requires discipline and it's software requires discipline and it's software requires discipline and it's the discipline to sit down and do what the discipline to sit down and do what the discipline to sit down and do what might be a bit of a grind or might be a might be a bit of a grind or might be a might be a bit of a grind or might be a bit boring but it's building the

bit boring but it's building the bit boring but it's building the software and testing it and making sure software and testing it and making sure software and testing it and making sure it does what you think it's going to do it does what you think it's going to do it does what you think it's going to do before you hand it to your Early Access before you hand it to your Early Access before you hand it to your Early Access users another question you might be users another question you might be users another question you might be asking yourself is are there any people asking yourself is are there any people asking yourself is are there any people that you don't want in your Early Access that you don't want in your Early Access that you don't want in your Early Access meaning should you cater only to people meaning should you cater only to people meaning should you cater only to people who are maybe a precise fit your ideal who are maybe a precise fit your ideal who are maybe a precise fit your ideal customer profile or should you go wider customer profile or should you go wider customer profile or should you go wider and have a lot of different kinds of and have a lot of different kinds of and have a lot of different kinds of customers trying out your Early Access customers trying out your Early Access customers trying out your Early Access the answer is it depends I think that the answer is it depends I think that the answer is it depends I think that most startups and most SAS companies most startups and most SAS companies most startups and most SAS companies from the start don't know who their ICP from the start don't know who their ICP from the start don't know who their ICP is unless you're in a really is unless you're in a really is unless you're in a really well-defined space and you know exactly well-defined space and you know exactly well-defined space and you know exactly who you're competing with meaning if I'm who you're competing with meaning if I'm who you're competing with meaning if I'm competing with MailChimp and aweber and competing with MailChimp and aweber and competing with MailChimp and aweber and Infusionsoft then maybe I know there's Infusionsoft then maybe I know there's Infusionsoft then maybe I know there's two or three icps that are the best fit two or three icps that are the best fit two or three icps that are the best fit for me but a lot of folks are building for me but a lot of folks are building for me but a lot of folks are building new products or products that are at the new products or products that are at the new products or products that are at the intersection of two or three different intersection of two or three different intersection of two or three different product types and so sometimes it can be product types and so sometimes it can be product types and so sometimes it can be hard to know who your ICP is when you're hard to know who your ICP is when you're hard to know who your ICP is when you're starting out now if you're niching down starting out now if you're niching down starting out now if you're niching down and let's say you're catering only to and let's say you're catering only to and let's say you're catering only to hairdressers or only to web designers or hairdressers or only to web designers or hairdressers or only to web designers or only two photographers you have an only two photographers you have an only two photographers you have an advantage because then you know that advantage because then you know that advantage because then you know that your ICP is exactly that person exactly your ICP is exactly that person exactly your ICP is exactly that person exactly that Persona and in that case I would that Persona and in that case I would that Persona and in that case I would absolutely limit my early access to that absolutely limit my early access to that absolutely limit my early access to that type of person because because one big type of person because because one big type of person because because one big mistake that you can make and a mistake mistake that you can make and a mistake mistake that you can make and a mistake I actually made early on with my last I actually made early on with my last I actually made early on with my last SAS app drip is I got 20 people into SAS app drip is I got 20 people into SAS app drip is I got 20 people into Early Access a bunch of them were Early Access a bunch of them were Early Access a bunch of them were Founders they were developers they were Founders they were developers they were Founders they were developers they were power users and then I got a few folks power users and then I got a few folks power users and then I got a few folks who were let's say more non-technical

who were let's say more non-technical who were let's say more non-technical and there was one person who was like a and there was one person who was like a and there was one person who was like a photographer and another person who I photographer and another person who I photographer and another person who I don't remember what they did but they don't remember what they did but they don't remember what they did but they were non-technical and they gave a were non-technical and they gave a were non-technical and they gave a dramatically different feedback than dramatically different feedback than dramatically different feedback than everyone else and if I'd had more of a everyone else and if I'd had more of a everyone else and if I'd had more of a 50 50 split I think it would have been 50 50 split I think it would have been 50 50 split I think it would have been really confusing and hard to know who to really confusing and hard to know who to really confusing and hard to know who to listen to as I built the product out but listen to as I built the product out but listen to as I built the product out but in the end I realized that the more in the end I realized that the more in the end I realized that the more technical folks and the more marketing technical folks and the more marketing technical folks and the more marketing Savvy startup Founders and developer Savvy startup Founders and developer Savvy startup Founders and developer types were probably going to be my early types were probably going to be my early types were probably going to be my early ideal customer profile and so I was able ideal customer profile and so I was able ideal customer profile and so I was able to listen more to those folks than the to listen more to those folks than the to listen more to those folks than the others one of the hardest things to do others one of the hardest things to do others one of the hardest things to do in this early stage is dealing with in this early stage is dealing with in this early stage is dealing with feedback during this phase because you feedback during this phase because you feedback during this phase because you have to strike this balance of not have to strike this balance of not have to strike this balance of not building everything everyone requests building everything everyone requests building everything everyone requests but building some of the things that but building some of the things that but building some of the things that some people request and this is where it some people request and this is where it some people request and this is where it comes down to going with some of your comes down to going with some of your comes down to going with some of your founder gut and filtering that through founder gut and filtering that through founder gut and filtering that through your vision of where you think the your vision of where you think the your vision of where you think the product should go while holding on to product should go while holding on to product should go while holding on to that Vision relatively Loosely if you that Vision relatively Loosely if you that Vision relatively Loosely if you don't have an opinion about what you don't have an opinion about what you don't have an opinion about what you want your product to be or who you want want your product to be or who you want want your product to be or who you want to compete with your product is going to to compete with your product is going to to compete with your product is going to wander all over the place because if you wander all over the place because if you wander all over the place because if you get 10 50 100 customers into an early get 10 50 100 customers into an early get 10 50 100 customers into an early access or even paying you full price and access or even paying you full price and access or even paying you full price and using the product you're gonna get a ton using the product you're gonna get a ton using the product you're gonna get a ton of different feedback and if you build of different feedback and if you build of different feedback and if you build all of that you're going to have a all of that you're going to have a all of that you're going to have a Frankenstein's monster of a product it's Frankenstein's monster of a product it's Frankenstein's monster of a product it's going to have settings that do all types going to have settings that do all types going to have settings that do all types of things it's going to be too of things it's going to be too of things it's going to be too complicated without focus and a vision complicated without focus and a vision complicated without focus and a vision in the early days it's easy to build in the early days it's easy to build in the early days it's easy to build really crappy software and you can see really crappy software and you can see really crappy software and you can see this with inexperienced Founders if

this with inexperienced Founders if this with inexperienced Founders if you've used their software a lot of Open you've used their software a lot of Open you've used their software a lot of Open Source unfortunately winds up looking Source unfortunately winds up looking Source unfortunately winds up looking like this because there's there's no like this because there's there's no like this because there's there's no Central person saying no let's say no to Central person saying no let's say no to Central person saying no let's say no to these things to keep the software these things to keep the software these things to keep the software elegant and easy to use and focused and elegant and easy to use and focused and elegant and easy to use and focused and so my biggest input when dealing with a so my biggest input when dealing with a so my biggest input when dealing with a myriad of feedback in the early days is myriad of feedback in the early days is myriad of feedback in the early days is to have some type of a vision that to have some type of a vision that to have some type of a vision that you're holding on to Loosely and that you're holding on to Loosely and that you're holding on to Loosely and that you think is the direction you want to you think is the direction you want to you think is the direction you want to go and then filter all feedback through go and then filter all feedback through go and then filter all feedback through that vision and you may need to adapt to that vision and you may need to adapt to that vision and you may need to adapt to that Vision over time like when I launch that Vision over time like when I launch that Vision over time like when I launch drip it was just an email capture widget drip it was just an email capture widget drip it was just an email capture widget and some email sequences it was very and some email sequences it was very and some email sequences it was very simple and I realized that Vision was simple and I realized that Vision was simple and I realized that Vision was too small and at a certain point had to too small and at a certain point had to too small and at a certain point had to make the adjustment to say well my make the adjustment to say well my make the adjustment to say well my vision is now building an email service vision is now building an email service vision is now building an email service provider and or a marketing automation provider and or a marketing automation provider and or a marketing automation provider so now that was the new vision provider so now that was the new vision provider so now that was the new vision and I got to filter all the incoming and I got to filter all the incoming and I got to filter all the incoming requests through that mindset another requests through that mindset another requests through that mindset another question you might be thinking about is question you might be thinking about is question you might be thinking about is how should you roll out access meaning how should you roll out access meaning how should you roll out access meaning do you go to your whole list your whole do you go to your whole list your whole do you go to your whole list your whole email list or your whole list of Early email list or your whole list of Early email list or your whole list of Early Access people or do you phase it out in Access people or do you phase it out in Access people or do you phase it out in what I call a phased launched or a what I call a phased launched or a what I call a phased launched or a staged launch the answer is it depends staged launch the answer is it depends staged launch the answer is it depends on how big your Early Access list is and on how big your Early Access list is and on how big your Early Access list is and it depends on how big your I'll say your it depends on how big your I'll say your it depends on how big your I'll say your launch list is I'm differentiating these launch list is I'm differentiating these launch list is I'm differentiating these two things with my last SAS our Early two things with my last SAS our Early two things with my last SAS our Early Access list was 16 17 people these were Access list was 16 17 people these were Access list was 16 17 people these were people that I knew personally or people that I knew personally or people that I knew personally or professionally and that expressed a dire professionally and that expressed a dire professionally and that expressed a dire interest in using the software but the interest in using the software but the interest in using the software but the email launch list was 3 400 people so email launch list was 3 400 people so email launch list was 3 400 people so these are two different things so when these are two different things so when these are two different things so when we went to Early Access I was letting in we went to Early Access I was letting in we went to Early Access I was letting in one two three people at a time from The

one two three people at a time from The one two three people at a time from The Early Access list and hand onboarding Early Access list and hand onboarding Early Access list and hand onboarding them hand billing them we had no billing them hand billing them we had no billing them hand billing them we had no billing code written we had no onboarding of any code written we had no onboarding of any code written we had no onboarding of any type except for me manually walking them type except for me manually walking them type except for me manually walking them through what today would be a loom video through what today would be a loom video through what today would be a loom video but it was basically a screencast at the but it was basically a screencast at the but it was basically a screencast at the time and that's how you have to think time and that's how you have to think time and that's how you have to think about it is how many people can I manage about it is how many people can I manage about it is how many people can I manage manually to bring into my software at manually to bring into my software at manually to bring into my software at this point if I had emailed 3 400 people this point if I had emailed 3 400 people this point if I had emailed 3 400 people on that launch list it would have been a on that launch list it would have been a on that launch list it would have been a catastrophe we did not have the ability catastrophe we did not have the ability catastrophe we did not have the ability to scale to that until many months down to scale to that until many months down to scale to that until many months down the line and so to give you an idea this the line and so to give you an idea this the line and so to give you an idea this was 2013 and we got our first customer was 2013 and we got our first customer was 2013 and we got our first customer onboarded in June and we launched to the onboarded in June and we launched to the onboarded in June and we launched to the final four or 500 people on the launch final four or 500 people on the launch final four or 500 people on the launch list in November so it took us five-ish list in November so it took us five-ish list in November so it took us five-ish months to phase this launch to Stage it months to phase this launch to Stage it months to phase this launch to Stage it out and once the Early Access folks were out and once the Early Access folks were out and once the Early Access folks were in we were building some features to in we were building some features to in we were building some features to help them be more successful and then we help them be more successful and then we help them be more successful and then we started rolling out to the email list started rolling out to the email list started rolling out to the email list about 300 400 people at a time and the about 300 400 people at a time and the about 300 400 people at a time and the last topic I want to cover is you might last topic I want to cover is you might last topic I want to cover is you might be thinking about when do I stop calling be thinking about when do I stop calling be thinking about when do I stop calling it an early access and just ask people it an early access and just ask people it an early access and just ask people to sign up and usually that happens once to sign up and usually that happens once to sign up and usually that happens once you've gone through your email launch you've gone through your email launch you've gone through your email launch list or your launch interest list you list or your launch interest list you list or your launch interest list you still call it early access look you're still call it early access look you're still call it early access look you're getting Early Access you get an early getting Early Access you get an early getting Early Access you get an early access discount or you get some type of access discount or you get some type of access discount or you get some type of bonus or you get something special for bonus or you get something special for bonus or you get something special for being there early once that's done flip being there early once that's done flip being there early once that's done flip that page to self sign up or maybe book that page to self sign up or maybe book that page to self sign up or maybe book a demo depending on your sales model and a demo depending on your sales model and a demo depending on your sales model and you let it rip and at that point you let it rip and at that point you let it rip and at that point hopefully you've already started on some hopefully you've already started on some hopefully you've already started on some marketing approaches that are driving marketing approaches that are driving marketing approaches that are driving traffic to your site so you can take off traffic to your site so you can take off traffic to your site so you can take off with a bang if you enjoyed this video with a bang if you enjoyed this video with a bang if you enjoyed this video you're going to enjoy my new book The

you're going to enjoy my new book The you're going to enjoy my new book The SAS Playbook how to build a SAS Playbook how to build a SAS Playbook how to build a multi-million dollar startup without multi-million dollar startup without multi-million dollar startup without Venture Capital almost everything I know Venture Capital almost everything I know Venture Capital almost everything I know about building SAS products without about building SAS products without about building SAS products without having to raise venture capital is having to raise venture capital is having to raise venture capital is contained in that book and you can head contained in that book and you can head contained in that book and you can head to sasplaybook.com to order your copy I to sasplaybook.com to order your copy I to sasplaybook.com to order your copy I have physical copies PDF copies and of have physical copies PDF copies and of have physical copies PDF copies and of course an audiobook if you enjoyed this course an audiobook if you enjoyed this course an audiobook if you enjoyed this video I'd really appreciate a like and video I'd really appreciate a like and video I'd really appreciate a like and of course subscribe to the channel and of course subscribe to the channel and of course subscribe to the channel and if you want to see a video that shows if you want to see a video that shows if you want to see a video that shows the next steps after the video that I'm the next steps after the video that I'm the next steps after the video that I'm talking about today which is after your talking about today which is after your talking about today which is after your Early Access how do you find your first Early Access how do you find your first Early Access how do you find your first hundred customers click on the thumbnail hundred customers click on the thumbnail hundred customers click on the thumbnail here you can watch that video I'll see here you can watch that video I'll see here you can watch that video I'll see you next week