Several things got me thinking this week. First was this article that was reposted on social media. It’s an old NYT article, following a case of scientific fraud. I don’t really read novels, but this was written like a novel, a great article, very well done. It’s interesting to see how it unfolded in the age just before widespread computer use. It involves peer review, scientific publication, and all that comes with being a published academic. Read it.
The second thing that settled on me this week was the decision by the Tribeca Film Festival to screen (and then, not screen) a documentary by Andrew Wakefield about (I assume) the harms of vaccination. Mr Wakefield had published the notorious “evidence” that vaccines cause autism. Fraud was discovered, Mr Wakefield was discredited, but the impact that initial publication left is remarkable. I was somewhat disappointed to hear that the film would not be viewed at the Festival. I think it’s important that people see how “science” and “documentary” films are really the works of an author. It’s too bad that the result of Mr Wakefield’s work led to decreased vaccination rates, but there is a story in every documentary--the story the author wants to tell. How much fact-checking do documentaries undergo to determine if they are, in fact, non-fiction?
How do we know what is true? Finding good evidence for my own practice is important to me. I like to think that if I open a journal in my field, the content published there has merit. I see some boring studies, some of minimal effort that perhaps do not advance the knowledge of the profession too much. I take these with a grain of salt, but I expect that they are valid
I am not a researcher. The U.S. has many academic medical centers where research is being performed. However, there are many more community hospitals, critical access hospitals, specialty care centers, and nursing homes where ICPs like me work. We rely on the work that is done by others. I’ve written here about moving towards fewer patients on isolation precautions, mostly due to new research in the field. But there are some who disagree with that research--people whose opinion I value. Why do they disagree?
One reason I read recently is some people feel the studies that concluded no increased risk from not using gloves and gowns with certain MDROs were not adequately powered to show a difference. If you aren’t familiar with “power” in a study, it’s how many people or events you need to observe to conclude with some (predetermined) degree of certainty that there was or was not a difference in the groups being compared.
For example, say I’d like to know if eating eggs prevents the occurrence of a certain disease. I take 2 groups of 100 people each, feed one group 2 eggs a day for 2 years, while the other group eats none. After 2 years, nobody has the disease. I conclude the eggs group was the same as the no-eggs group--eggs didn’t help. What I didn’t consider was the likelihood of the event--if this disease occurs in one out of every 1 million people annually, then I didn’t have enough people-time to really see the effects. Of course nobody got it--it’s really rare. And it had nothing to do with eggs. My study was underpowered.
A number of years ago, the CONSORT agreement was developed, requiring authors performing randomized trials to provide more detail about their data. They needed to spell out who was included, who was excluded, how they were randomized, what happened to each participant, and how sample size was determined. It increased transparency in what data was being included for analysis and reporting conclusions. How the sample size was determined is similar to power--what assumptions did you make about the events or outcomes, what degree of difference between the groups was considered significant, and how many subjects were needed to show those outcomes. This needs to be clear, and if it’s not, someone needs to ask why, before it goes to print.
So I went back and looked at the power of one MDRO-glove-gown study. I see the authors have a section detailing how they determined their sample size. I don’t see the actual calculation there, but I suppose I could plug the numbers into a statistics calculator. What I don’t know is if their assumptions are correct. And this is where it’s hard to be the end-user of research. Do I really need to check if every study I read is adequately powered? Shouldn’t this be part of the peer review process? If there were doubts, how did it get published? Who is responsible for putting that paper in that journal? What are their standards, and what do they owe their readers?
There’s a “new” study this week, garnering a lot of press--about the effects of moderate alcohol consumption. It’s not really anything new, it’s a reanalysis of old stuff--with some studies left out, and some put in. Moving mosaic tiles around to make a new picture, really; or “selecting” data to tell a story. It’s not my area of interest, but similar things come up in infection control. It’s a lot to weed through for a regular person like me, and I’m feeling a little discouraged lately with this "science." Right now, I have the latest issues of two journals in my kitchen--both still in plastic. I usually dive right in, but now, I feel kind of….meh.
How do you assimilate new research into your work? Do you trust your sources? Do you wait for your professional society or a higher organization (like CDC or HICPAC or WHO) to collate it into new guidelines? Do your policies have current references? Any references? Do you update the references, or check to see if they are still applicable, when you update policies? Do you just do what the person before you did? Are your policies written by committee, with little or no scientific evidence? Just a vote on what’s most convenient?
