Estimating the Impact of Website Changes on Conversion Rates
This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyze...
Main Author: | |
---|---|
Format: | Other/Unknown Material |
Language: | English |
Published: |
Lunds universitet/Nationalekonomiska institutionen
2023
|
Subjects: | |
Online Access: | http://lup.lub.lu.se/student-papers/record/9134717 |
id |
ftulundlupsp:oai:lup-student-papers.lub.lu.se:9134717 |
---|---|
record_format |
openpolar |
spelling |
ftulundlupsp:oai:lup-student-papers.lub.lu.se:9134717 2023-12-17T10:29:26+01:00 Estimating the Impact of Website Changes on Conversion Rates Jarco, Jan 2023 application/pdf http://lup.lub.lu.se/student-papers/record/9134717 eng eng Lunds universitet/Nationalekonomiska institutionen Lunds universitet/Statistiska institutionen http://lup.lub.lu.se/student-papers/record/9134717 website design causal inference observational study average treatment effects double machine learning Business and Economics H1 2023 ftulundlupsp 2023-11-22T23:29:17Z This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyzed. External data sources were also included, with the dataset covering 12 changes to the website's layout and payment process. The changes' effectiveness was assessed using three methods: comparing conversion rates before and after the changes, a modified linear regression model, and the Double Machine Learning (DML) method with Random Forests as the base learners. The analysis revealed that the only modification with a statistically significant positive impact on conversion rates was related bug fixing. Most changes did not significantly affect conversion rates, and some even demonstrated a non-significant negative impact. The DML method proved a useful tool in this context, outperforming simpler comparison methods with better control for confounding variables and reducing potential bias in Average Treatment Effect (ATE) estimation. However, estimates from the DML model were sensitive to the analysis time window. This study suggests future website design should focus on user-friendly and intuitive design, clear and detailed information provision, and careful evaluation of changes' potential impact on user experience. Other/Unknown Material DML Lund University Publications Student Papers (LUP-SP) |
institution |
Open Polar |
collection |
Lund University Publications Student Papers (LUP-SP) |
op_collection_id |
ftulundlupsp |
language |
English |
topic |
website design causal inference observational study average treatment effects double machine learning Business and Economics |
spellingShingle |
website design causal inference observational study average treatment effects double machine learning Business and Economics Jarco, Jan Estimating the Impact of Website Changes on Conversion Rates |
topic_facet |
website design causal inference observational study average treatment effects double machine learning Business and Economics |
description |
This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyzed. External data sources were also included, with the dataset covering 12 changes to the website's layout and payment process. The changes' effectiveness was assessed using three methods: comparing conversion rates before and after the changes, a modified linear regression model, and the Double Machine Learning (DML) method with Random Forests as the base learners. The analysis revealed that the only modification with a statistically significant positive impact on conversion rates was related bug fixing. Most changes did not significantly affect conversion rates, and some even demonstrated a non-significant negative impact. The DML method proved a useful tool in this context, outperforming simpler comparison methods with better control for confounding variables and reducing potential bias in Average Treatment Effect (ATE) estimation. However, estimates from the DML model were sensitive to the analysis time window. This study suggests future website design should focus on user-friendly and intuitive design, clear and detailed information provision, and careful evaluation of changes' potential impact on user experience. |
format |
Other/Unknown Material |
author |
Jarco, Jan |
author_facet |
Jarco, Jan |
author_sort |
Jarco, Jan |
title |
Estimating the Impact of Website Changes on Conversion Rates |
title_short |
Estimating the Impact of Website Changes on Conversion Rates |
title_full |
Estimating the Impact of Website Changes on Conversion Rates |
title_fullStr |
Estimating the Impact of Website Changes on Conversion Rates |
title_full_unstemmed |
Estimating the Impact of Website Changes on Conversion Rates |
title_sort |
estimating the impact of website changes on conversion rates |
publisher |
Lunds universitet/Nationalekonomiska institutionen |
publishDate |
2023 |
url |
http://lup.lub.lu.se/student-papers/record/9134717 |
genre |
DML |
genre_facet |
DML |
op_relation |
http://lup.lub.lu.se/student-papers/record/9134717 |
_version_ |
1785581811713179648 |