The replication crisis has many interesting effects on how people (and scientists) think about Psychology (and, of course, other fields) … Here is a nice summary of effects that are hard to replicate. Among them ‘classics’ like the power pose or the big brother eyes.
A lot is happening because of these new insights in terms of research (e.g., replication studies) and communication (e.g., Fritz Strack on Facebook).
And then this: Susan Fiske in an upcoming piece in the APS Observer … I am really struggling with this rhetoric – Daniel Lakens to the rescue 🙂
Ah – and of course Gelman.
This syllabus of an (obviously) awesome class has a ton of good reads:
Everything is fucked: The syllabus
by Sanjay Srivastava
I would have two additions:
- A multi lab replication project on ego-depletion (Hagger & Chatzisarantis, 2016)
- And the response from Roy Baumeister and Kathleen D. Vohs
It’s a really good statement of how f… up things are (in addition to all the other good examples above) …
“A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.” – Max Planck
Before there was R, there was S. R was modeled on a language developed at AT&T Bell Labs starting in 1976 by Rick Becker and John Chambers (and, later, Alan Wilks) along with Doug Dunn, Jean McRae, and Judy Schilling.
Here is a talk by Rick Becker telling the story of R. Good Stuff!
I gave the R package exams a shot for my decision making lecture. Here is what it does:
“Automatic generation of exams based on exercises in Sweave (R/LaTeX) or R/Markdown format, including multiple-choice questions and arithmetic problems. Exams can be produced in various formats, including PDF, HTML, Moodle XML, QTI 1.2 (for OLAT/OpenOLAT), QTI 2.1, ARSnova, and TCExam. In addition to fully customizable PDF exams, a standardized PDF format is provided that can be printed, scanned, and automatically evaluated.”
After some fiddling and help from one of the authors (the incredible nice Achim Zeileis, Uni Innsbruck) I got the following setup going:
- pool of ~ 100 questions in .Rmd format (all multiple choice, 3-6 answer options) grouped into lectures
- sampling out of the pool (e.g., 5 questions out of each lecture)
- random order of questions in each version of the exam (while keeping the lecture order, which I think is useful to give student more structure to work from)
- random order of the answers for each question
- exam with the correct answers
There are three parts:
- questions defining the answers to a question
- solutions defining the correct answers
- in LaTeX the actual question
All of this information goes into an .Rmd file.
Once this is done one has to define the questions to be included (the pool) and set the details for the selection process:
sol <- exams2pdf(myexam,
n = 2,
nsamp = 5,
dir = odir,
template = c("my_exam", "solution"),
encoding = 'UTF-8',
header = list(Date = "10.06.2016")
This code would give me 2 exams with a sample of 5 questions out of each block of questions.
Pretty awesome (after some setup work).
Thanks Achim et al. !!
They say about Sublime: “The text editor you will fall in love with”
hmm – kind of did that.
One reason being that when coding LaTeX in Sublime there is an awesome autocompletion feature:
Type ‘enum’ and you get:
Shift + Enter within an itemize adds a new \item
AFK to cry … tears of joy 🙂
Everything stolen from here
I spend a lot of time writing and answering email. Email is, according to timing, the third longest activity on my computer (although I am using three computers and can check this only on one of them – #timing please let us link computers for an overall analysis) … anyway – back to no email – as a holiday treat I decided to shut down all my email accounts 5 days before Dec. 24th and promised myself not to touch them until Jan. 11th. It turns out that I will come short one day of this plan. Nevertheless, I am quite happy with the result and the positive effects of this email absence. Needless to say that reading emails during vacation brings you back into a working mood (or never lets you out of it), not reading email had positive side effects before (I did this twice in the last 20 years of ‘doing’ emails). Many issues that come during such a break often solve themselves without intervention or can be solved quickly within a few hours after being back in the email world.
Well, I will turn on my email accounts now and see how much work has piled up … BRB.
So, 380 emails later – a paper submitted by a co-author, a rejection for a previous submission, a talk accepted, a chapter revised by a co-author – the best part of all this is that dealing with a ton of mails is a very quick thing, with a relatively low threshold for simply deleting out of date emails or replying quickly to urgent matters. What’s left are some longer replies I will do now …
Happy New Year!
Cilia Witteman and Nanon Spaanjaars (my dutch connection) worked together on a piece on whether psychodiagnosticians improve over time (they don’t) in their ability to classify symptoms to DSM categories. This turned out to be a pretty cool paper combining eye-tracking data with a practical, and hopefully, relevant question.
Schulte-Mecklenbeck, M., Spaanjaars, N.L., & Witteman, C.L.M. (in press). The (in)visibility of psychodiagnosticians’ expertise. Journal of Behavioral Decision Making. http://dx.doi.org/10.1002/bdm.1925
This study investigates decision making in mental health care. Specifically, it compares the diagnostic decision outcomes (i.e., the quality of diagnoses) and the diagnostic decision process (i.e., pre-decisional information acquisition patterns) of novice and experienced clinical psychologists. Participants’ eye movements were recorded while they completed diagnostic tasks, classifying mental disorders. In line with previous research, our findings indicate that diagnosticians’ performance is not related to their clinical experience. Eye-tracking data pro- vide corroborative evidence for this result from the process perspective: experience does not predict changes in cue inspection patterns. For future research into expertise in this domain, it is advisable to track individual differences between clinicians rather than study differences on the group level.
Andrew Gelman talked about a really old paper I did together with Anton Kühberger ages ago. It was actually the first paper / ‘real’ scientific project I was involved in.
It generated quite the buzz over its 20 year lifespan and was cited a whopping 13 times (stats look good without y-axis) …
Going back to it, I was happy to see that we already talked about replication (and were very reluctant to push the button harder – as we would have not been able to get through the reviews, I guess) … Things have changed.