AB Testing has become a huge part of my role as UX Design Specialist at AS Watson. In the last 18 months I have built my own AB Tests, created Test plans and schedule and built up a community of AB testers to share knowledge with. I have also trained 20 people within my company on AB testing and best practise.
Process
AB Testing can be a lengthy and time consuming task, especially without the right processes in place. Taking on AB Testing within the UX Team, I worked alongside a Lead Designer to create a new process: designing new Briefing and Reporting forms, a work flow and a spreadsheet to record all the data. Most importantly we realised this knowledge wasn't being shared internally, and so we aimed to communicate more within the business and share test and results on a Sharepoint site.
Organisation
To organise AB testing over 12 business units. we created a spreadsheet which automatically added AB testing job numbers, and then we could update the details to reflect where each test was at. The tests are all labelled from Briefed In, to In Progress or Live / Complete with an assigned department for task that is next to complete. We checked all stakeholders had signed off each task, relevant tracking added and estimated Go Live and Completion dates. In here we also kept links to every test brief and results document so we can easily look up previous tests. It also gives a clear indication to workflow, how many tests we have live and how many CRs we have raised with tech.
AB Testing Plans
Without a proper plan in place, AB testing was not efficient and conflicting tests were overlapping each other, voiding our results. To avoid this I set up a plan for each business units, outlining different areas of the website from Homepage, PLPs to Checkout and Registration. Only one test could run on each areas of the site at any one time to avoid overlaps. It also ensured we were maxing out our 5 AB test slots each month.
AB Testing Community
I created a 2020 plan to kick of testing for the business units that we assist in the AB Testing Community. We work in sprint to ensure that everyone is working to the same deadline, so we can assist with any difficulties or issues. We all then share our results internally to see what worked well, what didn't, and what findings we can take back to create our own tests. With the same focus for each sprint (e.g checkout), the business units are a lot more engaged and keen to help each other out.