Limesurvey randomizing

It is kind of an odd problem.
For the following pretty straight forward question: How do I randomise questions within a group in Limesurvey? It seems to be really hard to find an answer.

With the help of Jonas I figured out that there is a randomisation option hidden in the ‘Advanced Settings’ section of a question. What you have to do is provide the same number (this is important) for each question in a group that you want to randomise. Limesurvey will then take care of the rest. It did not seem to work if I named the variable with a string, eg, ‘group1’ but only numeric counters work fine.

Thanks Jonas! (I would not have finished at all) …

Screen Shot 2013-06-03 at 9.37.52 PM

Psychology as a reproducible Science

Is Psychology ready for reproducible research?

Today the typical research process in psychology looks generally like this: we collect data; analyze them in many ways; write a draft article based on some of the results; submit the draft to a journal; maybe produce a revision following the suggestions of the reviewers and editors; and hopefully live long enough to actually see it published. All of these steps are closed to the public except for the last one – the publication of the (often substantially) revised version of the paper. Journal editors and reviewers evaluate the written work submitted to them, they trust that the analyses described in the submission are done in a principled and correct way. Editors and reviewers are the only external part of this process who will have an active influence on what analyses are done. After the publication of an article the public has the opportunity to write comments or ask the authors for the actual datasets for re-analysis. Often however, getting access to data from published papers is hard, if not often impossible (Savage & Vickers, 2009; Wicherts, Borsboom, Kats, & Molenaar, 2006).  Unfortunately only the gist of the analyses are described in the paper and neither exact verification nor innovative additional analyses are possible.

What could be a solution for this problem? An example from computer science provides a concept called “literate programming” which was advocated by one of the field’s grandmasters, Donald Knuth, in 1984. Knuth suggested that documentation (comments in the code) should be just as important as the actual code itself. This idea was reflected nearly 20 years later when Schwab et al. (2000) formulated a concept that “replication by other scientists” is a central aim and guardian for intellectual quality; they coined the term “reproducible research” for such a process.

Let’s move the research process to a more open, reproducible structure, in which scientific peers have the ability to evaluate not only the final publication but also the data and the analyses.
Ideally, research papers would have code for analyses which are commented in detail and are submitted in tandem with drafts as well as the original datasets. Anybody, not only a restricted group of select reviewers and editors, could reproduce all the steps of the analysis and follow the logic of arguments on not only the conceptual level but at an analytic level as well. This openness facilities easy reanalysis of data also. Meta-analysis could be done more frequently and with greater resolution as the actual data are available. Moreover, this configuration would allow us collectively to estimate effects in the population and not restrict our attention to independent small samples (see Henrich, Heine, & Norenzayan, 2010 for a discussion of this topic).

What do we need to achieve this? From a policy perspective, journals would have to add the requirement for data and code submission together with the draft of each empirical paper. Some journals already provide the option to do that (e.g., Behavior Research Methods) in the supplemental material section on a voluntary base, some require the submission of all necessary material to replicate the reported results (e.g., Econometrica), however most do not offer such a possibility (it is of course possible to provide such materials through private or university web sites, but this is a haphazard and decentralized arrangement).

Tools are a second important part of facilitating this openness. Three open source (free of cost) components could provide the bases for reproducible research:

  • R (R Development Core Team, 2010) is widely recognized (cite) as the “language of statistics” and builds on writing code instead of a “click and forget” type of analysis that other software packages encourage. R is open source, comes with a large number of extensions for advanced statistical analysis and can be run on any computer platform, including as a Web based application (
  • LaTeX was invented to provide a tool for anybody to produce high quality publications independent of the computer system used (i.e. one could expect the same results everywhere,
  • Sweave (Leisch, 2002) connects R and LaTeX providing the opportunity to write a research paper and do the data analysis in parallel, in a well documented and reproducible way (

The power of these different tools comes from the combination of their being open source, their widespread adoption (across a wide range of fields in sciences), and the fully transparent means by which data analysis is conducted and reported.  It levels the playing field and means that anybody with an Internet connection and a computer can take part in evolving scientific progress.

John Godfrey Saxe famously said that: “Laws, like sausages, cease to inspire respect in proportion as we know how they are made.” We should strive that this is not true for psychology as a science.


two good things come together
LaTeX and Google Docs together in one nice (free)application – this is a brilliant idea, which gives you the power of LaTeX combined with the excellent collaboration possiblities of Google Docs – if I would have a button it would say “I like” …

Here is a screenshot from the project page – with LaTeX code on one side and the final output on the other …

R goes cloud

Jeroen Ooms did for R what Google did for editing documents online. He created several software packages that help running R with a nice frontend over the Internet.
I first learned about Jeroen’s website through his implementation of ggplot2 – this page is useful to generate graphs with the powerful ggplot2 package without R knowledge, however it is even more helpful to learn ggplot2 code with the View-code panel function which displays the underlying R code. If you are into random effect models another package connected to lme4 will guide you step by step through model building.
I think this is a great step forward for R and cloud computing!

LaTeX tips

Two things are often bothering when one starts to work with LaTeX: in text referencing of literature and Umlaute (for our German speaking friends)

Referencing: Here is a list of 5 types of in text referencing. To get them to work in a LaTeX file you need to define a referencing style in your document (in the example I use APA – American Psychological Association), you can of course use any style you want!

In the header:


In the actual text part the command on the left results in the reference on the right:

cite{Bortz1993} – (Bortz, 1993)
citeA{Bortz1993} – Bortz (1993)
citeNP{Campbell1959} – Campbell, 1959
citeauthor{Spiel2001} – Spiel citeyearNP{Bortz1993} – 1993

At the end of the document enter a reference to your .bib file, in the example below my file is called biblio.bib: bibliography{biblio}

Umlaute: In German there are several Umlaute that are not included in the standard english setup of a LaTeX installation. There are two ways to get them right: 1) given a standard LaTeX document use (again command on the left, result on the right): “a – ä “u – ĂĽ “o – ö) given that you actually want to write in German do the following:


Two useful links:

1) Wikipedia book on LaTeX

2) A page at the University of Salzburg’s Psychology department (German!)

Gernot Kleiter was the person who introduced me to LaTeX, decision making, good experimenting … actually, there should be a whole post about him …

XAMPP activating mysql

Here is the problem: I want to have a server (Apache, MySQL) on my local machine (for the current purpose it was a Windows machine) to do some developing, testing or a demo. Getting XAMPP at Sourceforge is straight forward, installing it works like a charm.


A problem coming back from time to time is the following: the MySQL daemon does not start.

Here are some steps that helped me:

1) make sure to deactivate any services related to MySQL and Apache
2) shutdown XAMPP
3) to be sure I restart the machine
4) do not start XAMPP with the .bat file but use the MySQL and Apache batch scripts – these open a command window which you should not close as long as you use XAMPP

Through these steps everything worked fine …