Jump to content

Wikipedia talk:Identifying reliable sources (medicine)

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Meta-analyses need careful scrutiny, peer-reviewed or not.

[edit]

This

is a cautionary tale. 𝕁𝕄𝔽 (talk) 19:15, 28 April 2024 (UTC)[reply]

Thanks, that's interesting.
I have been wondering recently why peer review doesn't involve putting each article through an evidence-based checklist (e.g., the Newcastle–Ottawa scale for non-randomized studies; the most relevant checklist for the subject matter could be agreed upon by the reviewers and the editor) and the results published alongside the article. It might make it easier to discover junk science before publication. WhatamIdoing (talk) 21:47, 28 April 2024 (UTC)[reply]
A number of journals now require (or at least strongly encourage) articles to follow appropriate guidelines including completed checklists (usually as part of Supplemental Material). EQUATOR has guidelines/checklists that cover many types of articles. For example, here is part of my boilerplate for rejecting bad MR papers:
"As you prepare your manuscript for submission to another specialty journal, you may be able to improve it by following one or more of the EQUATOR Guidelines's checklists (https://www.equator-network.org). Mendelian randomization studies must adhere to the best practice as described in the following guideline https://wellcomeopenresearch.org/articles/4-186/v3 and be accompanied to MR-STROBE checklist (https://www.equator-network.org/reporting-guidelines/strobe-mr-statement/)." Jaredroach (talk) 21:27, 12 July 2024 (UTC)[reply]

Reconsidering a blanket-ban of primary sources

[edit]

The express reason given is essentially a re-hash of the "reproducibility crisis," but I have to say, should we even be trusting these "fact-checking"-like organizations for whom we essentially defer the power of keeping out bad knowledge?

Did you know, for example, the American Psychological Association, known for such works as the DSM-V and numerous textbooks likely to pass as verified work through here without a second thought, is actually a trade organization with the express intent of lobbying on behalf of practicing psychologists, in exchange extracting registration dues, and licensing fees?

Or that the American Pain Society, this time a non-profit society that nominally advocated on behalf of patients by publishing clinical practice guidelines, actually instead acted on behalf of pharmaceutical companies to propagate a treatment mandate to prescribe more products, in essence acting as a marketing channel where physicians were not primed to regard it critically?

I think it might be beneficial to re-investigate the ultimate authority of the organizations/publication guidelines we choose to let pass uncritically with more ability afforded to well-informed individuals to make their case that enough primary research exists to effectively include a consensus on matters of pathology and lines of inquiry related to potential treatment paradigms. We do ourselves a disservice when we shut out promising potential, and similarly to our readers too when academic literature acts more like a thicket than it does pathway. Additivefreesb (talk) 17:52, 12 July 2024 (UTC)[reply]

Which blanket-ban on primary sources are you referring to? Firefangledfeathers (talk / contribs) 17:56, 12 July 2024 (UTC)[reply]
Wikipedia:Identifying reliable sources (medicine)#Avoid primary sources isn't a blanket ban. Wikipedia:Identifying reliable sources (medicine)#Avoid over-emphasizing single studies, particularly in vitro or animal studies is just common sense.
We have had multiple problems with primary sources, including:
  • editors cherry picking the one source that says something completely different from all the others (e.g., cigarettes don't cause lung cancer)
  • editors using obviously bad primary sources (e.g., the patent claiming that colloidal silver cures HIV, but it didn't test whether people had HIV in the first place)
  • editors believing the media hype (can result in bad content [because newspapers don't always get technical details right] and unbalanced articles [because it's all about what's in today's news, and next week it'll be some other vegetable that everyone's supposed to eat to prevent cancer])
  • authors spamming their own publications into as many articles as possible (this happens much less often with review articles)
plus, of course, all the problems with the reproducibility crisis and the general difficulty of figuring out which primary source to 'believe in', if the data is conflicting.
Primary sources are more likely to be tolerated in veterinary content or for very rare conditions. WhatamIdoing (talk) 18:38, 12 July 2024 (UTC)[reply]

I think something needs to be said of primary sources often also overwhelming both the average reader or editor, owing to both their sheer number, and the fact that even many well-intentioned editors are not deeply knowledgeable about all issues they write about.

Further, there is simply no blanket ban on primary sources. For instance, there is actually not even a recommendation to refrain from using secondary summaries from within primary sources (i.e. background sections or well chosen parts of discussion sections). One of the problems to allay is keeping Wikipedia from reading: xx et al. found 80% mortality, while xy found 79%, and zy found 81% - with WP:OR prohibiting us from summarizing these (which would be a problem due to evidence grading). This becomes less of a problem upon listing authoritative secondary sources, as they already do summation for us, and readers are likely to want to know what, for instance, both the CDC and WHO think about a specific issue.

This guideline already discusses pitfalls of relying on industry and industry-funded sources, and I would not object to a well-thought out extension on issues of industry influence on practice guidelines or biases in professional associations. WP:MEDORG (part of this guideline, which I worked extensively on) points to industry guidelines or guidelines from patient advocacy groups being considered below the threshold of MEDRS.

Perhaps, in the spirit of giving background to a well-though-out question, Colin has input on considerations when MEDRS was originally drafted. CFCF (talk) 17:54, 13 July 2024 (UTC)[reply]

Lots of people worked on the early drafts of MEDRS. My significant part was realising the medical project's guideline, which was becoming MEDMOS, needed the RS stuff pulled out of it into a new page. But the key battles we had then were editors who thought they knew better than these secondary sources or who thought journalists on their favourite paper did better. Citing the secondary literature was something academics are taught to avoid so it didn't come naturally.
Coming back to the original question about our favoured MEDORG guidelines having a malign influence behind them. Consider then if we let editors build our medical articles much like someone might write their own review from the primary research studies. That same malign influence would appear on Wikipedia as editors cherry picking primary sources. There isn't a mechanism whereby Wikipedia might be expected to do better, and a fair amount of evidence that it would do worse.
Wikipedia has a unique editing model where anyone can edit but the consequence of this is we agreed to make the selection and summarising of primary research studies into "somebody else's problem". We found alignment with Wikipedia's preference for "secondary sources" with readily available reviews, guidelines and textbooks. Our model, of relying on existing publications and a crude grading system for those publications, isn't perfect. It is too easy to find oneself reading a journal by a bad publisher. As you point out, it is possible that some apparent authorities are merely fronts for vested interests. But this surely also malignly affects medicine as a whole, so is something the real world needs to fix for itself, rather than us hope we can work around the problem. -- Colin°Talk 09:47, 14 July 2024 (UTC)[reply]
I think that there is sound reason for Wikipedia:Identifying reliable sources (medicine)#Avoid primary sources. For example, consider the following:
The article highlights cardiologist Don Poldermans's use of apparently fake data in studies that then led to a faulty conclusion that beta blockers were beneficial before heart surgery.
The article then goes on to state:
  • [E]xperts who study scientific misconduct believe that thousands of people may be dead because of him.
  • After the revelations, [of the falsified data] a new meta-analysis was published in 2014, evaluating whether to use beta blockers before cardiac surgery. It found that a course of beta blockers made it 27 percent more likely that someone would die within 30 days of their heart surgery.
  • Tens of millions of heart surgeries were conducted across the US and Europe during the years from 2009 to 2013 when those misguided guidelines were in place. One provocative analysis from cardiologists Graham Cole and Darrel Francis estimated that there were 800,000 deaths compared to if the best practices had been established five years sooner. While that exact number is hotly contested, a 27 percent increase in mortality for a common procedure for years on end can add up to an extraordinary death toll.
Note the clarifying & corrective effects of secondary sources in this situation. Peaceray (talk) 18:02, 23 August 2024 (UTC)[reply]

Editorials and op-eds

[edit]

This guideline lacks the equivalent of what WP:RSOPINION is to WP:RS. Having a section on how to use opinion pieces would enhance the accuracy and reliability of medical articles. It is not uncommon for editors of scientific journals to provide their own commentary on various issues; however, these editorials are distinct from the peer-reviewed articles that form the core content of the journals. 124.104.164.12 (talk) 21:13, 24 July 2024 (UTC)[reply]

Can you give me a couple of examples of opinion pieces that you'd like to cite in a Wikipedia article? WhatamIdoing (talk) 00:36, 25 July 2024 (UTC)[reply]

Question about need for MEDRS in Rikishi (sumo wrestlers) health section

[edit]

Hi all, looking for some guidance on what, if any content in Rikishi#Health effects requires MEDRS? This is for a Good Article nomination that is currently ongoing. Rollinginhisgrave (talk) 14:59, 14 August 2024 (UTC)[reply]

@Rollinginhisgrave, the most important thing to do is to use the best sources that you can. Most of MEDRS is directed towards questions about whether a particular medication is safe and effective for a particular condition, and is not relevant to content about whether people in a small profession have shorter lifespans. There is relatively little formal research available.[1]
You might consider whether some of the sources used in that section are out of date. For example, the sentence about body fat percentage is cited to a 30-year-old book, so it might not have up-to-date statistics. (20% body fat is probably an appropriate obesity cutoff for East Asian people.) WhatamIdoing (talk) 00:50, 15 August 2024 (UTC)[reply]
As always, thankyou WhatamIdoing for your help. Rollinginhisgrave (talk) 00:55, 15 August 2024 (UTC)[reply]
Statements about life expectancy are getting into the realm of MEDRS, but the following phrase about as the diet and sport take a toll on the wrestler's body is where it is making medical claims (although vague). The source used there[2] is a primary source and looks to be a short communication rather than a full research article. I checked Web of Science, and it only has 5 citations, but one of them is a review that may be worth using where it mentions sumo, especially in the context of the previous source.[3]
The second paragraph is where I'd be looking for more medical related sources though. Really anything that says that X results in higher incidence of Y for health issues is something pretty squarely needing medical sourcing. That said, this can be a gray area where you might range from commentary from coaches to those trained in sports medicine. At a glance I can't really assess the sources used there, but it's probably worth searching for more up to date sourcing while looking for medical sources there. KoA (talk) 01:45, 15 August 2024 (UTC)[reply]

Talk:Imane Khelif

[edit]

There are currently two RFCs at Talk:Imane Khelif. Interested editors are invited to participate at Talk:Imane Khelif#RfC lead and Talk:Imane Khelif#RfC on weight of "misinformation" in lead. TarnishedPathtalk 10:45, 23 August 2024 (UTC)[reply]

You might get more responses if you post it at Wikipedia talk:WikiProject Medicine. WhatamIdoing (talk) 17:14, 23 August 2024 (UTC)[reply]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Identifying_reliable_sources_(medicine)&oldid=1241964127"