Hello @madscientist

So, can you please imagine that you are working at a research center for oncology.

This is where I work, the DKFZ - Deutsches Krebsforschungszentrum.

@rumtscho I actually have some people getting funded by the DKFZ in the office next to me ;-)

You are a Ph.D. student, or maybe a postdoc. I advertised the survey in a lecture and also on an internal mailing list. You are completing it because you want to help me, not because your boss told you so.

The first page of the survey contains about everything which I told my real participants, so you'll know it in a minute.

And an important thing: This is my real, live survey, with the real data inside it too. I thought of posting a link to a copy, but if there is a technical problem, I'd like to have a chance of catching it too.

So, at the second page, you'll be asked to enter a "participant code". Please don't follow the instructions there, but enter "madscientisttest" or similar, so I can remove your answers from the real data.

www-survey.ifi.uni-heidelberg.de/limesurvey/index.php/225365/…

This is the link. 40 people just stopped entering anything after the "participant code", starting with the third page.

If you see anything possibly wrong with the survey - something which may make you revise your decision to fill out the survey - please tell.

And also any technical problems, of course.

@rumtscho The first question looks a bit intimidating, not necessarily a problem but this might be a factor why people stop answering.

The "what problems has the previous question" is pretty unusual, I was a bit confused at first.

@MadScientist I tried to explain why it is in there, did it come across? Maybe I did not phrase it well enough.

@rumtscho I've never seen something like this before in a survey, and it also makes the whole thing look longer and more complicated than it actually is.

Were you confused because you didn't know why it is there, or what you are supposed to answer, or because the question phrasing was unclear?

@MadScientist I know it is unusual. It is because what I'm doing is a kind of meta-survey, I need to measure the quality of the questions in the survey and not the quality of the software.

I've submitted it now, so you can check for technical problems. I didn't have any obvious ones on my end

@MadScientist OK, thank you.

There are no technical problems, I can see your answers until the end.

If the meta information is not that important, I'd hide it behind a button "this question is problematic" or give the option to not answer which will trigger the display of the meta questions.

If those meta questions are more important, explaining them a bit more might help. But that might also be too much text again

The meta information is actually quite important. Else I would have removed it altogether.

@MadScientist What were you missing in the explanation? Do you mean more justification why the meta information is needed, or making it more clear what each option means?

@rumtscho It's just unusual for a survey, but I'm not sure how one could improve it. It is simply different than what I'm used to when filling out a survey, and it confused me for a moment

@MadScientist OK, I understand. But it wasn't confusing to the point where you had the feeling of "this is so weird, I'd rather not do the survey further"?

@rumtscho For me not, but I had a completely different motivation for filling out the survey than your target group. I don't know how quickly they would abandon a survey

If the meta questions are the important ones, making the example questions not interactive might be one way to simplify it. So the only questions the participants fill in are the meta questions

@MadScientist OK. Then just one more thing: can you please tell me what browser and OS you used? Because I'm still investigating the "technical problem" hypothesis too. (And if it was IE, was it in compatibility mode)?

@rumtscho Chrome on Win 7

@MadScientist The example questions are also important. I want to evaluate the results of both. I have the hypothesis that the answers from the non-meta questions now will correlate with the answers of the same non-meta questions later, after the people have used the software. And beside this, I also want to say for each question, "people found it intrusive, so it shouldn't be used in future surveys", or "people found no problems, so do include it in surveys" or similar.

I'm gone for half an hour now

@MadScientist OK. Thank you very much. This detective work of "what's wrong" is hard, and every bit of information helps.