This is the first a three-part series. In it, we will hear from Dr. Letisha R. Wyatt, who has the unique distinction of being both a scientist and an academic librarian. She therefore has a unique perspective on the relationship between librarians and the research communities they support. In these blogs, she will examine the problem of experimental reproducibility — and offer her views on how scientists and librarians both can combat it.
Before I Was A Librarian …
My elevator pitch now goes something like this: “I’m a neuroscientist-turned-librarian, and my day-to-day work involves helping basic and translational scientists come up with solutions to tricky issues in the lab — ones related to managing their complex data and collaborating across research groups.”
While the aforementioned “solutions to tricky issues” can be unique to each group or independent researcher, there are plenty of commonalities in their pain. Let me give you a brief snapshot of what it can be like for many graduate students in the basic sciences:
- First, the training start line is littered with other peoples’ paper lab notebooks, data and protocols contained in messy folders and files, contained across several old computers.
- And there are also piles of slide boxes, with poorly labeled slides or freezers full of antibodies that may no longer work.
- If you’ve been lucky enough to previously work in a lab that did a better job at organizing everything, then you might only need to spend a couple months getting everything in order.
The Devil Is In The Details: Grad Student Research
The messy start aside, the ambitious grad student jumps in, ready to make some solutions and run some experiments. The solution recipes live in electronic and paper form, and there are three (or more) conflicting documents! “Recipe X” is used by one person; “Recipe Y” by everyone else; and the other recipes were abandoned, but for no clear reason.
That’s OK, though, because most of the lab agrees: the most recent version is an improvement on recently published methods — and works 75 percent of the time. With solutions made, the grad student is ready to do the experiments and process the samples.
The student must search through all the lab notebooks to determine what methods to use — and find hints on what pieces of the procedure to modify — because “nothing ever works the same in someone else’s hands”.
After settling on a method that seems to work, the grad student digs through the slide boxes and imaging files to compare his or her results to previous findings — only to find the complete opposite! Which finding or result is true?
Navigating Old Experimental Data Morasses
When facing this dilemma, the best solution is for the grad student to repeat the experiment and validate his or her own findings but … the student (or PI for that matter) was never really trained on how to best document the work, organize the information, or preserve the data. Now, the details in the lab notebook are too vague to make sense of, and because the old computer being used runs obsolete user apps, the student can’t open the files there, either.
This is where “GAME OVER!” flashes on the screen, and the grad student must start over at Level One. I admit this is a somewhat egregious example, but if you talk to researchers, they will attest that such experiences are fairly common. Until the end of my postdoc, I never realized the problems that I personally had encountered all along fell under the umbrella of reproducibility.
In the next installment, Dr. Wyatt will explore how scientists (with the aid of librarians) can take steps to overcome the reproducibility crisis.
Blog