This is part of a university research experiment run by Professor Dr. Nigel Harvey, University College London (UK) and Professor Dr. Shari De Baets, Open University of the Netherlands.

All results are anonymous. Should you have any further questions, feel free to contact shari.debaets@ou.nl


You will see a series of graphs with sales numbers and two forecasts: one made by a competent forecaster, and one derived from a statistical algorithm. Your task consists of studying the graphs and deciding which forecast is the judgmental one.

You will receive feedback on the accuracy of your choice via an error score. We will give a bonus payment in the form of an Amazon voucher of 5€ (or equal value in your currency) to the people with the 5 lowest error scores across all 18 trials! More detailed instructions are available on the next page.



Do not refresh the page or use the back button - this will reload the experiment from graph 1! Press "Start" to go the instructions page.

You will see a graph of 24 past sales values for each of 18 different products. In each case we will show you two forecasts, one derived from a statistical algorithm and the other produced by a competent forecaster using their judgment. All you have to do is decide whether the orange forecast is the judgmental one and the algorithmic forecast is the blue one - or vice versa.


You can do so by using the slider underneath the graph.


The more confident you are that the orange forecast is the judgmental one (and the blue forecast is the algorithmic one), the closer your rating should be to the right-hand end of the scale.





The more confident you are that the blue forecast is the judgmental one (and the orange forecast is the algorithmic one), the closer your rating should be to the lef-hand end of the scale.



Thus, if you are completely confident that the orange forecast is the judgmental one, the scale value (shown next to the scale) should be 100.



If you are completely confident that the blue forecast is the judgmental one, the scale value should be 0.



If you think that it is equally likely that the judgmental forecast is blue or orange, the scale value should be 50.


After you have indicated your choice on the sliders, a feedback button will appear. Click on this to proceed. Note that you cannot change your slider anymore after receiving feedback!


The feedback consists of three sources of information: first, the sales line (in grey) will be extended to show what the actual sales value was. Second, you will see whether the judgmental forecast was orange and the statistical forecast blue, or vice versa. Third, you will see your error score. A next button now appears to move on to the next graph.


The five lowest overall error scores will receive a bonus payment (Amazon 5€ or equivalent voucher for the five people with the lowest error scores). How is this error score calculated? If you can do this task, the ideal scale value should be 0 when the judgmental forecast is in orange and the statistical one is blue, and 100 when the judgmental forecast is in blue and the statistical one is in orange. To get an error score for each trial, we take the absolute difference between the scale value that you give and the ideal scale value and then square it.

For example:
If you put in a rating of 50 when the ideal scale value is 0 or 100, your error will be 2,500 (i.e., 50 x 50).
If you put in 100 when the ideal scale value is 100, your error will be 0. If you put in 100 when the ideal scale value is 0, your error will be 10,000 (i.e., 100 x 100).
If you put in a rating of 89 when the ideal scale value is 100, your error will be 121 (i.e., 100 - 89 = 11, and 11 x 11 equals 121).

Notice that, if you are totally uncertain as to which forecast is judgmental and which is statistical, you could proceed in two ways.

First, you could enter a rating value of 50. This will mean that your error score will certainly be 2,500.

Alternatively, you could randomly choose between a rating of 0 and a rating of 100. There would be a 50% chance of you being correct (i.e., an error score of 0) and a 50% chance of you being incorrect (i.e., an error score of 10,000). On average this strategy would give you an error score of 5,000.

So, on average, this second ‘gambling’ strategy would yield a higher error score than the first ‘non-gambling’ strategy. However, even if you have a small suspicion as to which forecast is which, your rating should deviate from 50. Only by acting on such small suspicions will you have a chance of obtaining one of the bonus payments.


You will be asked to do this for 18 consecutive graphs. At the end you will be asked for your gender and age.


Don't forget to press submit to receive your Prolific payment!

Press the button to start the experiment.

As a last step, please provide us with your age and gender. If you wish to participate in the competition for the lowest error score, please provide your e-mail. Note that this is stored separately and temporarily, and cannot be led back to your Prolific profile.

Background information:
Gender:
Male
Female
Age (in numbers; e.g., "47", not "forty-seven"):


(Type 'NA' if you do not wish to do so) E-mail for participating in the lowest error score contest

Statistical forecast

Judgmental forecast

Judgmental forecast

Statistial forecast

Statistical forecast

Judgmental forecast

Judgmental forecast

Statistical forecast