blogarticle
The importance of User Acceptance Testing
User Acceptance Testing (UAT) is essential for the successful implementation of any bespoke system and relies on the end user " often the client " investing sufficient time into the process. So what should you " the client " be expected to do, or is this simply an excuse that developers use to get you to debug their code?
Having written the introduction above, it sound like some sort of ?A Level? question that should have the word ?discuss? appended to it. However, this is actually a serious topic and can often determine the success of a project. Completed successfully it will remove any unintended errors before you launch the system; the alternative is, at worst, massive disruption to your users and at best, tantamount to washing your dirty laundry in public.
So what is User Acceptance Testing (UAT)?
User Acceptance Testing (UAT) is the procedure of testing the functionality of a bespoke system once the development phase has been completed. In an ideal world the test platform will recreate a real life replica " including data " that allows you to test the user interface and functionality of the system.
A good example of a UAT would be for an e-commerce website. Although this is by no means a complex system, it does explain both the need and the process of a properly managed UAT. Before being able to test the site you?ll need to data load it with customers, products, etc., The development studio should provide you with something that essentially looks like the completed site and you would need to do the following: "
- Access the site as a new visitor, browse products and then add them to the basket. Complete the checkout process, ensuring you track any options so that you can test all the different variables.
- Access the site as an existing customer browse products and then add them to the basket. Complete the checkout process, again ensuring you track any options so that you can test all the different variables.
- Access the site as an employee, checking each of the individual procedures contained within the site, e.g. processing an order, packing lists, delivery notes, etc.,
This is obviously an oversimplified example, but even with an e-commerce site you can end up with a staggering number of variables. We completed a site earlier this year and calculated there were over one hundred different variables that could be selected during the check out process and each one needed testing.
Who should do the testing?
In an ideal world the budget should allow for a professional tester to carry out all testing against a specification, checking all functionality works as intended. When I first started working for software companies in the nineties every project went through a robust testing procedure, but theses days budgets and agile software development seems to have reduced this practice. With constant pressure on software developers to keep budgets low, testing is an easy target for cutting corners and cost. In reality both the developers and clients should be involved in testing, albeit at different stages and with different objectives.
System testing
When a developer is programming they test the code, but they know what should happen and subconsciously test with that knowledge in mind. Once the developer has completed his / her work, a second developer or preferably a professional tester should then test the application. Ideally a tester should be used, as they will approach the task differently, professionally and methodically. I used to work for a software studio that employed a technical support team and they were responsible for system testing.
User Acceptance Testing
The UAT is usually performed by the client and / or end-users and is not usually focused on identifying simple problems such as spelling errors and cosmetic problems as these should have been picked up during the system testing. We can sum up the difference with the analogy of a machine process:
In your system you expect
" To input A and B and receive C
Upon testing you actually find
" You get D (result not as expected)
" You actually needed to input A, B and C (information required but not given)
" A and B are different from your understanding.
These are all abstract examples of the way software works. In each, the software does not necessarily have a ?bug? (an error with the code) but it is programmed in a way that doesn?t meet your understanding. This is effectively the definition and purpose of UAT.
The client should thoroughly test the system using actors that will use the system in a particular way; in the example given above we would use three actors, a new customer, an existing customer and an employee. The client needs to test various scenarios based on the agreed functionality to ensure it has been correctly implemented. Each scenario can have multiple acceptance tests, to ensure the functionality works as expected. Clients are responsible for verifying the correctness of the acceptance tests and reviewing test results to decide what needs addressing.
Testing should be completed on any subsequent upgrades to ensure tests are completed successfully as new issues can be introduced inadvertently.
What should you be expected to provide in terms of feedback?
As developers, we need to know how an error occurred. This means that we need to know what you did to create the problem and to resolve it quickly and efficiently. When testing, you " the client " need to provide a replicable error. Although this isn?t always easy, in most cases with careful observation, you should be able to record this.
The development studio will not appreciate you providing ambiguous information as this is tantamount to looking for a needle in a haystack. Furthermore, they are unlikely to locate the error and you run the risk of the system being launched without identifying potential issues.
Conclusion
This is a massive topic and I have only scratched the surface, but hopefully it gives you some idea of what to expect. It is essential that you allow sufficient resources to complete the User Acceptance Testing to avoid issues later on. We have heard numerous excuses for not testing, everything from ?you should have done it? to ?I haven?t got time?. Neither of these are really acceptable and clients need to take responsibility for ensuring the system has been adequately tested and be prepared to sign off the project.
When embarking on a bespoke system make sure you understand what and when you are expected to test and ensure as many users are given the opportunity to participate. A bespoke solution relies on thorough testing by both the development studio and the client.
Resources
Here are some additional resources you might find interesting:
Wikipedia article (Acceptance testing)
Five Reasons Not To Do UAT
What is User Acceptance Testing?
User Acceptance Testing a business analyst?s perspective
Date: 20/06/2014
The importance of User Acceptance Testing
User Acceptance Testing (UAT) is essential for the successful implementation of any bespoke system and relies on the end user " often the client " investing sufficient time into the process. So what should you " the client " be expected to do, or is this simply an excuse that developers use to get you to debug their code?
Having written the introduction above, it sound like some sort of ?A Level? question that should have the word ?discuss? appended to it. However, this is actually a serious topic and can often determine the success of a project. Completed successfully it will remove any unintended errors before you launch the system; the alternative is, at worst, massive disruption to your users and at best, tantamount to washing your dirty laundry in public.
So what is User Acceptance Testing (UAT)?
User Acceptance Testing (UAT) is the procedure of testing the functionality of a bespoke system once the development phase has been completed. In an ideal world the test platform will recreate a real life replica " including data " that allows you to test the user interface and functionality of the system.
A good example of a UAT would be for an e-commerce website. Although this is by no means a complex system, it does explain both the need and the process of a properly managed UAT. Before being able to test the site you?ll need to data load it with customers, products, etc., The development studio should provide you with something that essentially looks like the completed site and you would need to do the following: "
- Access the site as a new visitor, browse products and then add them to the basket. Complete the checkout process, ensuring you track any options so that you can test all the different variables.
- Access the site as an existing customer browse products and then add them to the basket. Complete the checkout process, again ensuring you track any options so that you can test all the different variables.
- Access the site as an employee, checking each of the individual procedures contained within the site, e.g. processing an order, packing lists, delivery notes, etc.,
This is obviously an oversimplified example, but even with an e-commerce site you can end up with a staggering number of variables. We completed a site earlier this year and calculated there were over one hundred different variables that could be selected during the check out process and each one needed testing.
Who should do the testing?
In an ideal world the budget should allow for a professional tester to carry out all testing against a specification, checking all functionality works as intended. When I first started working for software companies in the nineties every project went through a robust testing procedure, but theses days budgets and agile software development seems to have reduced this practice. With constant pressure on software developers to keep budgets low, testing is an easy target for cutting corners and cost. In reality both the developers and clients should be involved in testing, albeit at different stages and with different objectives.
System testing
When a developer is programming they test the code, but they know what should happen and subconsciously test with that knowledge in mind. Once the developer has completed his / her work, a second developer or preferably a professional tester should then test the application. Ideally a tester should be used, as they will approach the task differently, professionally and methodically. I used to work for a software studio that employed a technical support team and they were responsible for system testing.
User Acceptance Testing
The UAT is usually performed by the client and / or end-users and is not usually focused on identifying simple problems such as spelling errors and cosmetic problems as these should have been picked up during the system testing. We can sum up the difference with the analogy of a machine process:
In your system you expect
" To input A and B and receive C
Upon testing you actually find
" You get D (result not as expected)
" You actually needed to input A, B and C (information required but not given)
" A and B are different from your understanding.
These are all abstract examples of the way software works. In each, the software does not necessarily have a ?bug? (an error with the code) but it is programmed in a way that doesn?t meet your understanding. This is effectively the definition and purpose of UAT.
The client should thoroughly test the system using actors that will use the system in a particular way; in the example given above we would use three actors, a new customer, an existing customer and an employee. The client needs to test various scenarios based on the agreed functionality to ensure it has been correctly implemented. Each scenario can have multiple acceptance tests, to ensure the functionality works as expected. Clients are responsible for verifying the correctness of the acceptance tests and reviewing test results to decide what needs addressing.
Testing should be completed on any subsequent upgrades to ensure tests are completed successfully as new issues can be introduced inadvertently.
What should you be expected to provide in terms of feedback?
As developers, we need to know how an error occurred. This means that we need to know what you did to create the problem and to resolve it quickly and efficiently. When testing, you " the client " need to provide a replicable error. Although this isn?t always easy, in most cases with careful observation, you should be able to record this.
The development studio will not appreciate you providing ambiguous information as this is tantamount to looking for a needle in a haystack. Furthermore, they are unlikely to locate the error and you run the risk of the system being launched without identifying potential issues.
Conclusion
This is a massive topic and I have only scratched the surface, but hopefully it gives you some idea of what to expect. It is essential that you allow sufficient resources to complete the User Acceptance Testing to avoid issues later on. We have heard numerous excuses for not testing, everything from ?you should have done it? to ?I haven?t got time?. Neither of these are really acceptable and clients need to take responsibility for ensuring the system has been adequately tested and be prepared to sign off the project.
When embarking on a bespoke system make sure you understand what and when you are expected to test and ensure as many users are given the opportunity to participate. A bespoke solution relies on thorough testing by both the development studio and the client.
Resources
Here are some additional resources you might find interesting:
Wikipedia article (Acceptance testing)
Five Reasons Not To Do UAT
What is User Acceptance Testing?
User Acceptance Testing a business analyst?s perspective
Date: 20/06/2014
The importance of User Acceptance Testing
User Acceptance Testing (UAT) is essential for the successful implementation of any bespoke system and relies on the end user " often the client " investing sufficient time into the process. So what should you " the client " be expected to do, or is this simply an excuse that developers use to get you to debug their code?
Having written the introduction above, it sound like some sort of ?A Level? question that should have the word ?discuss? appended to it. However, this is actually a serious topic and can often determine the success of a project. Completed successfully it will remove any unintended errors before you launch the system; the alternative is, at worst, massive disruption to your users and at best, tantamount to washing your dirty laundry in public.
So what is User Acceptance Testing (UAT)?
User Acceptance Testing (UAT) is the procedure of testing the functionality of a bespoke system once the development phase has been completed. In an ideal world the test platform will recreate a real life replica " including data " that allows you to test the user interface and functionality of the system.
A good example of a UAT would be for an e-commerce website. Although this is by no means a complex system, it does explain both the need and the process of a properly managed UAT. Before being able to test the site you?ll need to data load it with customers, products, etc., The development studio should provide you with something that essentially looks like the completed site and you would need to do the following: "
- Access the site as a new visitor, browse products and then add them to the basket. Complete the checkout process, ensuring you track any options so that you can test all the different variables.
- Access the site as an existing customer browse products and then add them to the basket. Complete the checkout process, again ensuring you track any options so that you can test all the different variables.
- Access the site as an employee, checking each of the individual procedures contained within the site, e.g. processing an order, packing lists, delivery notes, etc.,
This is obviously an oversimplified example, but even with an e-commerce site you can end up with a staggering number of variables. We completed a site earlier this year and calculated there were over one hundred different variables that could be selected during the check out process and each one needed testing.
Who should do the testing?
In an ideal world the budget should allow for a professional tester to carry out all testing against a specification, checking all functionality works as intended. When I first started working for software companies in the nineties every project went through a robust testing procedure, but theses days budgets and agile software development seems to have reduced this practice. With constant pressure on software developers to keep budgets low, testing is an easy target for cutting corners and cost. In reality both the developers and clients should be involved in testing, albeit at different stages and with different objectives.
System testing
When a developer is programming they test the code, but they know what should happen and subconsciously test with that knowledge in mind. Once the developer has completed his / her work, a second developer or preferably a professional tester should then test the application. Ideally a tester should be used, as they will approach the task differently, professionally and methodically. I used to work for a software studio that employed a technical support team and they were responsible for system testing.
User Acceptance Testing
The UAT is usually performed by the client and / or end-users and is not usually focused on identifying simple problems such as spelling errors and cosmetic problems as these should have been picked up during the system testing. We can sum up the difference with the analogy of a machine process:
In your system you expect
" To input A and B and receive C
Upon testing you actually find
" You get D (result not as expected)
" You actually needed to input A, B and C (information required but not given)
" A and B are different from your understanding.
These are all abstract examples of the way software works. In each, the software does not necessarily have a ?bug? (an error with the code) but it is programmed in a way that doesn?t meet your understanding. This is effectively the definition and purpose of UAT.
The client should thoroughly test the system using actors that will use the system in a particular way; in the example given above we would use three actors, a new customer, an existing customer and an employee. The client needs to test various scenarios based on the agreed functionality to ensure it has been correctly implemented. Each scenario can have multiple acceptance tests, to ensure the functionality works as expected. Clients are responsible for verifying the correctness of the acceptance tests and reviewing test results to decide what needs addressing.
Testing should be completed on any subsequent upgrades to ensure tests are completed successfully as new issues can be introduced inadvertently.
What should you be expected to provide in terms of feedback?
As developers, we need to know how an error occurred. This means that we need to know what you did to create the problem and to resolve it quickly and efficiently. When testing, you " the client " need to provide a replicable error. Although this isn?t always easy, in most cases with careful observation, you should be able to record this.
The development studio will not appreciate you providing ambiguous information as this is tantamount to looking for a needle in a haystack. Furthermore, they are unlikely to locate the error and you run the risk of the system being launched without identifying potential issues.
Conclusion
This is a massive topic and I have only scratched the surface, but hopefully it gives you some idea of what to expect. It is essential that you allow sufficient resources to complete the User Acceptance Testing to avoid issues later on. We have heard numerous excuses for not testing, everything from ?you should have done it? to ?I haven?t got time?. Neither of these are really acceptable and clients need to take responsibility for ensuring the system has been adequately tested and be prepared to sign off the project.
When embarking on a bespoke system make sure you understand what and when you are expected to test and ensure as many users are given the opportunity to participate. A bespoke solution relies on thorough testing by both the development studio and the client.
Resources
Here are some additional resources you might find interesting:
Wikipedia article (Acceptance testing)
Five Reasons Not To Do UAT
What is User Acceptance Testing?
User Acceptance Testing a business analyst?s perspective
Date: 20/06/2014
