Rational Arguments

Go back through nearly a year of posts, and I bet you won’t find many that talk about my experiences with patients.  You might not even know I am a doctor.  You’d never know that I am actually a pediatrician, and that I do still see patients now and then.  I don’t talk about my clinical experience much with friends, or even family.

And even when I sit and think about being a doctor, I don’t think about the last 7 years of being a faculty member here at IU.  I rarely think of clinic when I was a fellow before that.  When I think about being a doctor, I almost always go back to residency.

Look, I know many physicians who love caring for patients.  I even know those who remember fondly their days as residents, being in the trenches and completely immersed in clinical care.  I was not one of those people.  I hated residency.  It really didn’t agree with me.  Ask my wife; for that matter, ask any of my friends.  It was obvious to anyone who spent any time with me.

I didn’t hate residency because of the hours, although they were terrible.  I didn’t hate the pay.  I didn’t hate being overworked or underappreciated.  I didn’t hate patients or the people I worked with.  I hated the system.  More specifically, I hated being a doctor in the system.

I just finished Atul Gawande’s latest masterpiece.  I am rarely so jealous of anyone as I am of him and his skill right now.  He brought it all back for me.  I can tell you many horror stories of those three years in Seattle.  But ask me to rank the top few and this one inevitably comes to the top:

I was on a rotation in the Neonatal Intensive Care Unit (NICU), where babies who are born prematurely or really sick are cared for.  A couple came in with a midwife after a way-too-long and rather botched attempt at a home delivery.  As soon as they arrived, we knew things were not going to go well.  The baby was born in extreme distress.  It appeared to be septic, or massively infected.  Initial vital signs looked really bad.  And then things got worse.

One by one, the baby’s systems seemed to shut down.  He couldn’t breathe on his own, so we put in a breathing tube.  Then his heart started to fail, so we put lines into his umbilical cord to pump in medications.  His lungs collapsed, so we put in tubes into his chest to help them reinflate.

While another doctor and I struggled to keep all this going, I listened as, right behind me, the doctors in charge sounded downright optimistic to the parents, who were, understandably, a mess.  They could not imagine how things had gone wrong so fast.  They wanted to hear good news.  No one seemed to be able to tell them the truth.  They were given messages of hope and told us to do everything.  That’s what we do in medicine.  That’s especially what we do in the NICU.

They left to go home and get clothes and supplies.  Everyone dispersed.

So I was alone with this baby.  It was small, and blueish, and had an ungodly number of devices and tubes coming out of it.  I was 25, depressed, and I started to cry.

The baby never moved.  His heart would slow down, and I’d up his meds.  His heart rate would come back up until it didn’t and started to drop again.  So I upped the meds some more.  I don’t know how long this went on.  I didn’t eat, I didn’t go to the bathroom, I didn’t talk to anyone.  I just stood and watched.

Eventually, the ventilator stopped getting the job done, so we had to put the baby on an oscillator.  Basically, instead of giving normal breaths, this machine shoves tiny amounts of air in and out really fast.  It sometimes works when other things fail.  It was loud, noisy, and made the baby shake.  I don’t think he noticed.

Things slowly got worse.  Nothing was working, and every vital sign was heading downwards.  As instructed, I just kept adding stuff to keep him alive.  But deep down inside, I started to think that what I was doing was wrong.  Not incorrect – wrong.  I wondered if I was hurting the baby.  I just wanted him to be at peace.  And, for a moment, I wanted the baby to die.

I don’t like to think about it.  I try not to.  Ever.  But it happened.

Not long after, nothing I was doing was working.  I called in the doctors in charge, and they agreed.  They asked where the parents were.  It suddenly dawned on me that they hadn’t yet returned.  We called them, and they were shocked to hear how bad things were.  After all, those same doctors had told them things were going to be ok.

They rushed back as fast as they could.  They didn’t make it in time.

I thought I would post a piece of Gawande’s article and talk about how we completely screw up end of life care.  I thought I would make a comment about how we spend too much money or waste resources.  I thought I would talk about tradeoffs and better choices.  But I can’t.  Partly because I can’t do his work justice, and partly because this is an issue where deep down inside I think we are doing a ton of harm.  Full stop.

I went home that night and bawled uncontrollably.  This kind of thing happened all too often.  I toyed with the idea of getting out.  I even had resumes prepared to send off to companies outside of medicine.

But, some time later, I found myself back in the NICU.  A similar situation was occurring.  This time, though, the doctor in charge handled everything differently.  She spoke to the patients honestly and in a completely different tone.  She asked the parents what they wanted out of the short time they might have with their baby.

They cried at first.  And then they stopped.  They cleaned the baby up and dressed him in clothes his grandparents had bought.  And they took him out.

They were gone for a few hours, and then they came back.  They allowed us to give the baby drugs to comfort him.  They held him, as a family, as he quietly passed.

I remember quite clearly his sister was in the room.  She was about six.  I asked her how they had spent the day.  She told me how they had taken the baby to the park, to see the water.  They had brought him to family members so everyone could hold him.  They showed him the sun and let him lay in the grass and let a dog lick his face.  Her mother was listening in at the end, and somehow smiling.

Some months later, I ran into the mother in a different part of the hospital.  She remembered me, and thanked me for all I had done.  I remarked that I hadn’t done much; they had cared for the baby.

“No,” she replied.  “Without all of you, he never would have known what chocolate ice cream tastes like.”

I spent four years in medical school learning how the body works, how it can break down, and how to repair it.  I spent three more learning how to give the right drugs and do the right procedures to fight illness.  And in all the time I’ve been a doctor, I honestly don’t know if I’ve ever done any more good than helping to stop the system so that baby, and that family, could share some ice cream.

I have a hard time explaining what I do to people.  I’m not trying to discover a drug or cure a disease.  I want to fix the system.  That’s how I found peace.  That’s how I practice medicine.

That’s why I’m a health services researcher.

Per Avik Roy, the study we’ve been discussing is up online.  Unfortunately, it’s behind a paywall, so most of you won’t be able to see it yet.  Moreover, it’s complicated, and I don’t think it’s likely permissible for me to repost huge sections of it here.  So I’m going to have to summarize, and you will have to trust, but verify.

As has been said over and over, this study used a large inpatient database to examine the association between primary payer status and outcomes.  And, as has been said, it found that being uninsured or having Medicaid led to worse outcomes, including a higher risk of death.

I’ve read the whole thing now.  If you came looking for me to tell you the methodology is fatally flawed, I will have to disappoint you.  The authors were thorough, careful, and skilled.  Their analysis is complex, well controlled, and uses established methodology.  I do not doubt that their results are robust, nor do I think there is anything major that I would change.

Where Avik and I will likely agree is on the strict recitation of the findings:

Unadjusted mortality for Medicare (4.4%; odds ratio [OR], 3.51), Medicaid (3.7%; OR, 2.86), and Uninsured (3.2%; OR, 2.51) patient groups were higher compared to Private Insurance groups (1.3%, P < 0.001). Mortality was lowest for Private Insurance patients independent of operation. After controlling for age, gender, income, geographic region, operation, and 30 comorbid conditions, Medicaid payer status was associated with the longest length of stay and highest total costs (P < 0.001). Medicaid (P < 0.001) and Uninsured (P < 0.001) payer status independently conferred the highest adjusted risks of mortality.

Patients with private insurance do best.  After controlling for important covariates, Medicaid had the longest length of stay and highest total costs.  I could come up with explanations as to why this is, but – to be honest – it’s irrelevant to the larger point I want to make, so let’s leave that for the moment.  The other key finding were that Medicaid and Uninsured patients were independently found to have higher mortality after adjusting for other factors.  So far, I bet Avik and I won’t quibble.

But one key point, and again it is not huge, is that all the comparisons showing significance were done with private insurance as the reference.  So it is totally correct to say that both Medicaid and Uninsured were significantly worse that private insurance.  But there was no test comparing Medicaid with Uninsured.  So this study did not show that Medicaid patients had a higher mortality than the uninsured.  Yes, the odds ratio was higher for Medicaid than for the Uninsured, but the confidence intervals overlap with respect to mortality, so we can’t make clear distinctions between those two groups.

But, again, I don’t want to spend too much time on that point.  I agree with the larger point that Medicaid patients and the Uninsured both fared much worse than those with private insurance.  I accept the results.

Now on to likely disagreements.

First off let me address this:

Aaron also notes that Medicaid is voluntary: but this is weak support for the implication that Medicaid, in its current form, is the best we can do. In the Vietnam days, some conservatives used to tell liberals to “love [America] or leave it.” I don’t remember liberals being too happy about that. Nor does Medicaid’s voluntary nature mean, ipso facto, that it must be doing some good. Does the voluntary nature of Medicare overutilization mean that Medicare overutilization is a good thing? Most liberal health policy types that I know believe otherwise.

It’s not that I think Medicaid is voluntary.  It’s that I don’t think Medicaid keeps you from seeing doctors you could otherwise see if you didn’t have it.  As I said before, Medicaid patients are able to see physicians that accept Medicaid plus those that they can pay for out of pocket.  I don’t believe this equals “doing some good”.  I do think this equals increasing the available pool of physicians over what they would have without Medicaid.  I think I know what a response might be to this argument, but I hate when people presuppose what the opposition must think, so I’m going to let others make their own arguments as to why this is wrong.

But the larger issue is to what this all means.   I agree that Medicaid is associated with bad outcomes.  I agree that Medicaid is flawed.  I agree that Medicaid could be improved.  But I don’t necessarily agree that Medicaid is the cause of the problem.

I wish I could repost huge tracts of the manuscript, because I think the authors do an excellent job of describing many of the reasons this association might exist.  Almost none of them are causal.  I will summarize as best I can:

• Elective operations were more common with Private Insurance and nonelective were more common in Medicaid and Uninsured patients, pointing to the fact that those populations don’t have the same choices up front.  Elective surgeries usually are planned for and have better outcomes.  Yes, this is controlled for, but still significant (and noted by the authors).
• Patients with private insurance may have better access to higher quality physicians or facilities.  There is a good amount of evidence that surgeons’ experience matters.  I don’t disagree with this.  I do disagree that Medicaid worsens this over being Uninsured because of the argument I made earlier.
• Other differences may exist between these populations that are unaccounted for.  The authors note:

Both Medicaid and Uninsured payer groups had the highest incidence of drug and alcohol abuse. In addition, Medicaid patients had the highest incidence of acquired immunodeficiency syndrome, depression, liver disease, neurologic disorders, and psychoses. Furthermore, Medicaid patients had the highest incidence of metastatic cancer, which likely reflects the combined influence of deficits in access to care, poor health maintenance, and delayed diagnosis resulting in the presentation of advanced disease stage within this population.

• Patients with Medicaid and the Uninsured use the system differently than those with private insurance. More of them get their primary care in the emergency department.  They are more likely to have language barriers or health literacy issues.  They are more likely to be malnourished and have other issues.

Again, though, we can’t know for sure.

So what should we do with this?  If I read Avik’s argument correctly, he thinks Medicaid is fatally flawed and should be replaced with, perhaps, subsidies to buy private insurance.

Now – in theory – that’s not the worst idea in the world.  In fact, it’s what the insurance exchanges in PPACA represent.  I am going to take Avik at his word and go with this a little further.

In a previous post, Avik seemed to endorse giving the poor a lump sum payment and letting them try and get insurance.  I don’t think that’s a good idea.  With individual ratings, insurance for much of the Medicaid population would still be out of reach even with the subsidy he cites.  But in his most recent post, Avik endorses the Swiss system.  Now, that is much more interesting.

The Swiss system has a powerful mandate.  They require everyone to buy private insurance.  They also give out subsidies so that insurance is not more than 8%-10% of anyone’s income.  All insurance companies are mandated to have a “basic” plan with regulated benefits.  This basic plan also must be non-profit.  All insurance is community rated.  The Swiss system also takes a pretty heavy hand with practitioners.  As Uwe Reinhardt notes:

On the surface, the Swiss health system may give the impression of a price-competitive, consumer-directed health care model. However, the heavy government regulation that pervades the entire system—including the health insurance sector—makes it a far cry from the vigorous, price-competitive health care market envisaged by the advocates of consumer-directed health plans in the United States. Some gestures to competition aside, the Swiss system so far has remained mainly a de facto cartel of insurers and health care practitioners who transact with one another in a tight web of government regulations.

I agree that the Swiss system costs less than ours, achieves comparable and sometimes better outcomes, and covers nearly 100% of their population.  If this is the direction Avik thinks is a good idea, then we may agree more than I think.

I remember this one moment back in the midst of health care reform when I was sitting in a radio studio, feeling pretty glum about the whole ordeal, and I mused aloud, “I wonder if anyone is enjoying this at all.”

For some of us, improving the health care system is more than a passing fancy.  I’m a health services researcher.  It’s my job.  Those people on the TV pay attention to it when it gets ratings, and those people in Washington get into it every other decade or so, but it’s what I do every day.

I really with I could make you believe that I couldn’t care less about the politics of it all.  Seriously.  Yes, like everyone else I care about who is in the White House of Congress, and I do lean different ways on different issues, but – on this I swear – I just want a better health care system.  That’s all.  I don’t care if it’s labelled conservative or liberal.  I don’t care what names you call it.  I just want it to be better.

Specifically, I want it to have phenomenal outcomes, I want everyone to have access to it, and I want it to be cost-effective.

I hope you notice I didn’t say cheap.  I think good things DO cost money.  I just wouldn’t like to spend any more than we have to.

People may think I’m closed minded, but I swear I’m not.  I am persuaded by good research.  That’s all.  Evidence.  Data.  Science.  I really don’t have a stake in who “wins” this.  It’s only the final product that matters.

Which brings me back to the title of this post.  I’m heartened by some recent activity I’ve seen.  I watched a debate occur between Austin Frakt and Avik Roy without it degenerating into partisan hackery.  I somehow managed to say a few words on the topic without resorting to sarcasm.  And I believe (maybe I’m delusional) that someone following the whole thread may have seen that it is possible to discuss good, unbiased research in a civil manner.  I stand by this:

I don’t think there is necessarily a right or wrong here.  Research is, more often than not, baby steps.  Each study adds a small amount to our understanding; each study is only so generalizable.

And I’d add that by talking about the body of work that exists, and discussing what from it we can agree on, we could find a starting point for moving forward.  Wouldn’t it be nice if we could devise pilot programs in health care reform in different ways, and then study them?  Agree to the methodology beforehand and then see what happens?  How great would it be if we all agreed to stand by the results, no matter what they might be, no matter what our ideology?

For instance, Avik cites the Medicaid program in Indiana which set up health savings accounts for Medicaid recipients and then coupled them with high deductible health care plans.  Why oh why did they not agree to allow an independent third body to set up a study to see how it worked?  Maybe good research would have shown that reform to be a spectacular success, vindicating the consumer-driven approach.  Maybe good research would have shown that reform to be a spectacular failure, suggesting that such an approach cannot succeed in a high-risk population.

It sure would have been nice to find out.  Instead, too many are driven by a desire to see their pre-chosen method of reform succeed, and so prevent any chance of it being seen in a bad light.  These types of people deny any flaws in a single-payer approach, no matter if such flaws exist.  They refuse to examine the downside of increasing out-of-pocket costs because it might hurt their “side”. They are more interested in the ideology than the goal of improving the health care system.

No good can come of that.

Anyone who reads this blog knows I think highly of Austin.  And, I opened by email this weekend to receive a very nice note from Avik concerning my recent posts.  Those personal contacts really do matter.  They remind me that some of us do have a shared goal – a better health care system.  We may disagree on how best to get there, but we can at least discuss our differences, and – perhaps – look for similarities. This kind of post gives me hope.

I hope this type of debate continues.  For my part, I’m going to keep an eye out for the study that started this all.  When I can finally read the whole thing, I will, and I will make sure I tell you what I think.  I’ll tell you if I think it’s a good study (or not), and how I think it adds to our body of knowledge about Medicaid.  And maybe, if more people keep on doing that, we can shift the discussion of health care reform away from politics and back to what really matters.

You hear all this talk about whether or not people approve of the Affordable Care Act, and you start to believe that somehow the exact center of the country lies somewhere between those who feel the Act was the end of freedom as we know it and those who feel it was the answer to all our prayers.  Then a reader sends you this:

The gloves are off in the fight for the Democratic nomination for governor. Former State Senator Matt Dunne is calling out Senator Peter Shumlin for what he says is a troubling misstatement.

Peter Shumlin first said it two weeks ago in an online questionnaire, then again this week in a mailing to voters: “I am the only candidate who sponsored a single-payer health care bill.” Dunne says that is simply not true.

“The facts are that I also sponsored a single-payer health care bill,” Dunne told reporters outside the Statehouse Wednesday.

He did. During the 1993-1994 legislative session when Dunne was serving as a State Representative he co-sponsored a bill that would have created a single-payer system. He wants a public retraction of Shumlin’s claim and is requesting that Shumlin send out another mailer with the correction.

Wait…  what?  Are you kidding me?

One Democratic candidate for governor, Matt Dunne, is upset at another Democratic candidate, Peter Shumlin, because he Shumlin claimed that Dunne did NOT support single-payer health care.  Did you get that?  They are fighting to show which of them is the more fervent supporter of single-payer health care reform.

Watch the TV, and you’d think that voicing anything less than total hatred for such reform would be the kiss of death in modern politics.  Not so in Vermont.

Vermont is still a state, right?

Just when you thought it was over:

And you thought all talk of a public option health insurance plan was dead. But no, Rep. Pete Stark (D-Calif.) asked the Congressional Budget Office to crunch the numbers on a public plan, funded by premiums, not taxes, and they found the following:

“The Congressional Budget Office (CBO) estimates that the public plan’s premiums would be 5 percent to 7 percent lower, on average.”

Also:

“CBO and the staff of the Joint Committee on Taxation (JCT) estimate that the proposal would reduce federal budget deficits through 2019 by about $53 billion.”

The next year would probably save another $15 billion, for a total of $68 billion shaved off the deficit by 2020.

You can read the full CBO report here.

But look, what I said back then still applies:

It’s just not that important. It’s really not. Maybe there was a time it was, but no longer. Now it’s a withered appendage. No more people will be insured with a public option than without it. Nor will the quality of care differ. It may decrease the total cost of the bill by a few percent, but that’s it. It’s not a path to single payer. It’s not a secret government takeover of care. And I say that not caring whether you hope or fear if either of those things is true.

I’m all for saving another $53 billion a decade, but come on.  We’re spending $2.5 trillion a year on health care.  I applaud the idea of continuing to improve the system, but we need to make bigger changes than this.

The debate continues.

As I said before, I think debate is good.  And I don’t think there is necessarily a right or wrong here.  Research is, more often than not, baby steps.  Each study adds a small amount to our understanding; each study is only so generalizable.  Avik believes that the research speaks to the harm Medicaid is doing.  I think that the evidence is not so clear.  But I come from the pediatric world, and the research in pediatrics and Medicaid isn’t necessarily the same as the research in surgery, which Avik cites:

As to Carroll’s point about the fact that the uninsured aren’t getting mammograms and colonoscopies, but Medicaid patients are: that isn’t true, because Medicaid patients can’t get appointments to see doctors (see above). If Carroll was right, it would only render even more striking the fact that the uninsured have their cancers detected earlier than do Medicaid patients. With breast cancer specifically (since he brings up mammograms), Medicaid patients were 31% more likely to have late-stage breast cancer than the uninsured. With surgical resections for colon cancer (since he brings up colonoscopies), in a separate study, Medicaid patients had a 27% higher risk of mortality than the uninsured, and a 9% higher risk of surgical complications.

I freely admit I’m not as well versed in the surgical literature as I am in the pediatric literature.  There are only so many hours in the day, and I really do have a full time job.  But I think we are fundametally interpreting this literature differently.  I have trouble understanding the idea that Medicaid patients have more trouble seeing doctors than uninsured patients because of Medicaid.  The pool of physicians uninsured people can see is equal to the ones they can afford to pay out of pocket.  The pool of physicians a patient a on Medicaid can see is equal to the ones that accept Medicaid plus the ones they can afford to pay out of pocket. So the pool of physicians a Medicaid patient can see is actually a superset of those that the uninsured can see.

Of course, it’s entirely possible that those on Medicaid can’t afford to see the same physicians that the uninsured can, because the Medicaid population if often poorer than the uninsured.  That would be a key difference, and not the fault of Medicaid.

This should not be taken as any sign that the studies Avik cites don’t show Medicaid is flawed.  It absolutely is. Not even single payer advocated believe that we should have Medicaid-for-all.  YES, Medicaid is flawed.  It’s state based, so it can run out of money.  It reimburses too little.  It needs to be improved.

But because Medicaid is state based, it can differ in quality from state to state.  It ranges in its benefits from state to state.  The number and quality of docs who accept it differ from state to state.

Contrary to what some believe, I welcome high quality research no matter what the outcome.  The only way we can improve the system is to first find out what is wrong with it.  And so I hope Avik is not talking about me when he says:

Skeptics of the study are setting themselves up for disappointment if they hope that the detailed results will exonerate Medicaid.

I’m not skeptical of this study.  I’m skeptical of all studies until I can personally evaluate their methods.  And I won’t be disappointed no matter what the final outcome because I have no vested interest in Medicaid.  I have a vested interest only in a high quality, cost-effective health care system.

As I have said before:

You can’t ignore [research] because it doesn’t fit your ideology… [w]hen you do science, you agree to accept the results…

That’s how it goes in research.

I’ve been away the last few days and trying to relax.  Now that I’m catching up, I see Austin Frakt and Avik Roy are having a slight disagreement as to the association of Medicaid on outcomes.  Long story short, Avik points to the literature that shows that Medicaid is associated with worse surgical outcomes.  Austin asks the important follow-up question: if you believe that’s true, are you suggesting that we make those on Medicaid more like the uninsured?  I don’t want to wade into this too deep.  I will make a number of small points:

-Insurance doesn’t equal care.  Insurance can affect how likely you are to get care and how quickly you might get it.  But any study that looks at insurance has to adjust for many, many other variables in order to get the true effect of insurance.

-There is a large body of literature out there on insurance.  A lot of it shows that people with private insurance do better than those with public insurance or those without insurance; that should not be a surprise.  Most people (and most of your docs) would rather have private insurance than Medicaid.  But would you really rather have no insurance than Medicaid?  If so, that is everyone’s right.  Don’t get the Medicaid.  I wager few would make that choice.

-I find it interesting that most of the literature that Avik cites is about surgery.  Surgery is different than other types of care (like emergency care) in that it is harder to refuse.  So it may be that the uninsured are getting care on a compassionate basis.  Few would provide a screening mammogram or yearly colonoscopy to someone uninsured, however, and you would get that with Medicaid.

On the whole, I think the debate is healthy and good.  No one is claiming that Medicaid is perfect, or that we should all just get Medicaid.  There is always room for improvement.  I also don’t necessarily think that Avik is arguing that we should just dump all the Medicaid people on the street, which is what I think some (not Austin) are implying he is saying.

And I’m stopping there.  Were I on the radio, I would be happy to debate this.  But blogging is too asynchronous when I’m joining in so late.  Except for one thing, and here I’m going to take a tiny issue with Avik’s first post.  It was based on (as far as I can tell) a meeting abstract.

I have a long-standing beef with promoting research that is presented in abstract form at scientific meetings.  It makes for great press and lots of splash, but I think it’s a real problem.  So much so that I have refused to participate in media events or press releases that about my work unless it has already appeared in the peer-reviewed literature.  Why?  It’s not rigorously reviewed.  Here is the total amount that we are able to know about the methods of the study Avik cites:

Methods: From 2003-2007, 893,658 major surgical operations were evaluated using the Nationwide Inpatient Sample (NIS) database: lung resection, esophagectomy, colectomy, pancreatectomy, gastrectomy, abdominal aortic aneurysm repair, hip replacement, and coronary artery bypass. Patients were stratified by primary payer status: Medicare (n=491,829), Medicaid (n=40,259), Private Insurance (n=337,535), and Uninsured (n=24,035). Multivariate regression models were applied to assess outcomes.

That’s it.  Was it a good study?  Valid?  How can you tell?

This ticked me off so much as a fellow, that I actually studied it.  Specifically, we looked at abstracts presented at the Pediatric Academic Societies meeting, which is the largest pediatric research meeting.  You can read the full paper, but here’s the abstract:

OBJECTIVE: The validity of research presented at scientific meetings continues to be a concern. Presentations are chosen on the basis of submitted abstracts, which may not contain sufficient information to assess the validity of the research. The objective of this study was to determine 1) the proportion of abstracts presented at the annual Pediatric Academic Society (PAS) meeting that were ultimately published in peer reviewed journals; 2) whether the presentation format of abstracts at the meeting predicts subsequent full publication; and whether the presentation format was related to 3) the time to full publication or 4) the impact factor of the journal in which research is subsequently published.

METHODS: We assembled a list of all abstracts submitted to the PAS meetings in general pediatrics categories in 1998 and 1999, using both CD-ROM and journal publications. In each year, we chose up to 80 abstracts from each presentation format (“publish only,” “poster,” “poster symposium,” “platform presentation”). We chose either 1) all abstracts in each format or 2) when there were >80 abstracts, a random selection of 80 of them. We assessed each selected abstract for subsequent full publication by searching Medline in March 2003; if published, then we recorded the journal, month, and year of publication. We used logistic and linear regression to determine whether publication, time to publication, and the journal’s impact factor were associated with the abstract’s presentation format.

RESULTS: Overall, 44.6% of abstracts presented at the PAS meeting achieved subsequent full publication within 4 to 5 years. There were significant differences between the rates of subsequent full publication of abstracts submitted but not chosen for presentation at the meeting (22.2%) and those that were chosen for presentation in poster sessions (40.0%), poster symposia (44.1%), and platform presentations (53.8%). There were no meaningful differences between the presentation formats in their mean time to publication and their mean journal impact factor.

CONCLUSIONS: PAS meeting attendees and the press should be cautious when interpreting the presentation format of an abstract as a predictor of either its subsequent publication in a peer-reviewed journal or the impact factor of the journal in which it will appear.

I know that can be overwhelming, so here’s the gist.  We looked at a sample of all abstracts sent in to the meeting, and whether they were ever published in peer-review journals.  The first thing I always remind people is that 87% of abstracts that were sent in were presented.  That’s a lot; very few were refused.  So I wouldn’t necessarily assume that just because an abstract is presented, it’s totally valid.  Second, less than 45%of the research presented was published in a peer-reviewed journal in the next four to five years.  So over half of what was presented at the meeting never was “really” published.

I’m not saying the results of Avik’s discussed study aren’t valid.  I’m saying I can’t tell.  And neither can you, without more information.  The peer review for a meeting just isn’t the same as for full publication.  You have less time, different criteria, and almost nothing by which to judge the work.  Ideally, meetings would stop publicizing abstracts as if they were full studies, but neither they, nor the press, seem likely to do so.

Go read this post by Austin Frakt.  He’s more forgiving than I am and takes a kinder tone, but I agree.

Somehow I forgot to post this when it went up:

One of the stories we’re watching in Washington this week has to do with something called “the doc fix.”

It has to do with how doctors are paid when they care for Medicare patients. It’s a problem that’s been around for years — without a permanent solution.

Healthcare-policy wonk Dr. Aaron Carroll provides his take on the latest developments in government health care reform.

This stuff is still relevant, regardless of the short attention span of the media.