Growing up to be old

Some papers have somewhat weird starting points – this one had an awesome starting point – Lake Louise (Canada):

In a little suite we (Joe Johnson, Ulf Böckenholt, Dan Goldstein, Jay Russo, Nikki Sullivan, Martijn Willemsen) sat down during a conference called the ‘Choice Symposium‘ and started working on an overview paper about the history and current status of different process tracing methods. One central result (why can’t all papers be like that) is the figure below where we try to locate many process tracing methods on the two dimensions: temporal resolution and distortion risk (i.e., how fast can a method measure a process and how destructive is this measurement).

Schulte-Mecklenbeck, M., Johnson, J.G., Böckenholt, U., Goldstein, D., Russo, J., Sullivan, N., &  Willemsen, M. (in press). Process tracing methods in decision making: On growing up in the 70ties. Current Directions in Psychological Science.

Ah – everybody was trying to find a path all the time:




Something about reverse inference

Often, when we run process tracing studies (e.g., eye-tracking, mouse-tracking, thinking-aloud) we talk about cognitive processes (things we can’t observe) in a way that they are actually and directly observable. This is pretty weird – which becomes obvious when looking at the data from the paper below. In this paper we simply instruct participants to follow a strategy when making choices between risky gamble problems. Taking the example of fixation duration we see that there is surprisingly litte difference between calculating an expected value, using a heuristic (priority heuristic) and just making decisions without instructions (no instruction) … maybe we should rethink our mapping of observation to cognitive processes a bit?

Here is the paper:

Schulte-Mecklenbeck, M., Kühberger, A., Gagl, S., & Hutzler, F. (in press). Inducing thought processes: Bringing process measures and cognitive processes closer together. Journal of Behavioral Decision Making. [ PDF ]


The challenge in inferring cognitive processes from observational data is to correctly align overt behavior with its covert cognitive process. To improve our understanding of the overt–covert mapping in the domain of decision making, we collected eye-movement data during decisions between gamble-problems. Participants were either free to choose or instructed to use a specific choice strategy (maximizing expected value or a choice heuristic). We found large differences in looking patterns between free and instructed choices. Looking patterns provided no support for the common assumption that attention is equally distributed between outcomes and probabilities, even when participants were instructed to maximize expected value. Eye-movement data are to some extent ambiguous with respect to underlying cognitive processes.

Everything is fucked …

This syllabus of an (obviously) awesome class has a ton of good reads:

Everything is fucked: The syllabus

by Sanjay Srivastava

I would have two additions:

  1. A multi lab replication project on ego-depletion (Hagger & Chatzisarantis, 2016)
  2. And the response from Roy Baumeister and Kathleen D. Vohs

It’s a really good statement of how f… up things are (in addition to all the other good examples above) …

“A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.” – Max Planck


New Paper on pychodiagnosis and eye-tracking

Cilia Witteman and Nanon Spaanjaars (my dutch connection) worked together on a piece on whether psychodiagnosticians improve over time (they don’t) in their ability to classify symptoms to DSM categories. This turned out to be a pretty cool paper combining eye-tracking data with a practical, and hopefully, relevant question.

Schulte-Mecklenbeck, M., Spaanjaars, N.L., & Witteman, C.L.M. (in press). The (in)visibility of psychodiagnosticians’ expertise. Journal of Behavioral Decision Making.


This study investigates decision making in mental health care. Specifically, it compares the diagnostic decision outcomes (i.e., the quality of diagnoses) and the diagnostic decision process (i.e., pre-decisional information acquisition patterns) of novice and experienced clinical psychologists. Participants’ eye movements were recorded while they completed diagnostic tasks, classifying mental disorders. In line with previous research, our findings indicate that diagnosticians’ performance is not related to their clinical experience. Eye-tracking data pro- vide corroborative evidence for this result from the process perspective: experience does not predict changes in cue inspection patterns. For future research into expertise in this domain, it is advisable to track individual differences between clinicians rather than study differences on the group level.

about illusions

Andrew Gelman talked about a really old paper I did together with Anton Kühberger ages ago. It was actually the first paper / ‘real’ scientific project I was involved in.

It generated quite the buzz over its 20 year lifespan and was cited a whopping 13 times (stats look good without y-axis) …





Going back to it, I was happy to see that we already talked about replication (and were very reluctant to push the button harder – as we would have not been able to get through the reviews, I guess) … Things have changed.

When something old …

Schulte-Mecklenbeck, M., & Kühberger, A. (2014). Out of sight – out of mind? Information acquisition patterns in risky choice framing. Polish Psychological Bulletin, 45, 21–28.

I teamed up with Anton Kühberger to write about one of our old, favorite topics: framing and process tracing …

Here is the abstract:
We investigate whether risky choice framing, i.e., the preference of a sure over an equivalent risky option when choosing among gains, and the reverse when choosing among losses, depends on redundancy and density of information available in a task. Redundancy, the saliency of missing information, and density, the description of options in one or multiple chunks, was manipulated in a matrix setup presented in MouselabWeb. On the choice level we found a framing effect only in setups with non-redundant information. On the process level outcomes attracted more acquisitions than probabilities, irrespective of redundancy. A dissociation between acquisition behavior and choice calls for a critical discussion of the limits of process-tracing measures for understanding and predicting choices in decision making tasks.