Jump to content

Wikipedia talk:Identifying reliable sources (medicine)

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Discussion on Electronic Harassment

[edit]

Hello, there is a discussion on Talk:Electronic harassment about whether declaring a group 'delusional' falls under WP:MEDRS at Talk:Electronic harassment#Introduction Violates WP:MEDRS and WP:NPOV. This could use some editors more familiar with Wikipedia's standards. Amranu (talk) 13:39, 12 December 2024 (UTC)[reply]

Discussion at RSN that may include a medical claim

[edit]

Help with WP:RSN#Vice Media (again) would be appreciated. -- LCU ActivelyDisinterested «@» °∆t° 16:54, 19 December 2024 (UTC)[reply]

I suggest changing "medicine-articles" to "medical articles" - what do you think?

[edit]

The following sentence appears in the first paragraph of this content guideline (WP:MEDRS):

Sourcing for all other types of content – including non-medical information in medicine-articles – is covered by the general guideline on identifying reliable sources.

I want to solicit others' opinions, but at this point I believe medicine-articles should be changed to medical articles because, unless I'm missing something, medicine-articles is incorrect usage.

I also asked for feedback over at the Guild of Copy Editors talk page, where I explain in more detail problems such as lack of parallel structure and making readers work harder than necessary "when a noun is used adjectivally in place of the more usual adjective." [Bryan A. Garner, Garner's Modern English Usage (4th ed. 2016) at 416–417.]

What do you think? -- Mark D Worthen PsyD (talk) [he/him] 03:38, 3 January 2025 (UTC)[reply]

That shouldn't be hyphenated. @Tony1, could you tell us the best way to say this? WhatamIdoing (talk) 05:26, 3 January 2025 (UTC)[reply]
Definitely no hyphen. "Medical articles" is fine. The only other alternative is "medicine-related articles", but why make it three words rather than two? Tony (talk) 10:08, 5 January 2025 (UTC)[reply]
Years ago, the text previously said "sourcing for all other types of content – including non-medical information in medicine-related articles – is covered by...". Since "medical" means "relating to the science or practice of medicine" then I guess that's a fine replacement for the clunky "medicine-related" and the incorrect "medicine". -- Colin°Talk 19:29, 5 January 2025 (UTC)[reply]
Thanks for fixing that, Colin. WhatamIdoing (talk) 05:30, 6 January 2025 (UTC)[reply]
Much appreciated Colin. ¶ This is a great example of Wikipedia at its best. A moderately experienced Wikipedian (me) posts a suggestion, and within two days very experienced Wikipedians efficiently discuss and promptly make an agreed-upon edit. Little things like this keep me coming back to contribute as I can and outweigh the discouragement I sometimes feel when encountering internecine conflict insigated by a minority of problematic editors. -- Mark D Worthen PsyD (talk) [he/him] 11:20, 8 January 2025 (UTC)[reply]

Improving the "referencing a guideline" illustration

[edit]

Check the quality range illustration used in the WP:MEDORG section.

CFCF posted this in 2016. I do not think it has been discussed anywhere.

I question that it puts "Health technology assessments" as the highest quality guideline. I live in the United States and am unfamiliar with this concept. I know this concept is not used much in Wikipedia. At Talk:Health technology assessment I asked for an example of one of these assessments.

I propose deprecating or updating this image. If we were to re-make it or present this guide in any other form, I support keeping everything else in place and just removing HTA.

I might also switch "national guidelines" with "professional societies", because in the past 10 years or so, multiple major governments have recommended health policies which conflict with medical professional society recommendations. My feeling is that when this happens, Wikipedia editors have prioritized expert physician statements over political statements when they differ. Examples of areas where politicians and medical professional societies take differing positions include on health issues for infectious disease like COVID, labor rights and occupational health and safety, LGBT+ health issues, health effects related to climate change, and access to healthcare in poverty conditions when patients cannot pay for recommended care.

It is fairly common for guidelines like these to be established without completely connecting to the WP:MEDRS standard of evidence, which is a meta analysis published in a peer reviewed journal. Often guidelines either interpret a study beyond what the paper about a study actually says, or they may not clearly even connect to research at all. They are still consensus statements from groups of experts at authoritative institutions, and pass WP:MEDORG, but they are not what usually comes into Wikipedia.

This image is Wikipedia's editorial policy and not much discussed. When something is proposed without evidence or discussion, then we can update and change it just as casually, and I think that is the situation we have here.

I was just having a conversation with @Zefr: about this at Wikipedia:Reliable_sources/Noticeboard#Patient-Centered_Outcomes_Research_Institute_(PCORI) regarding PCORI statements. I am not proposing any ideological changes to what is already established here, but we do need updates sometimes. Bluerasberry (talk) 20:19, 11 January 2025 (UTC)[reply]

In the United States, the Agency for Healthcare Research and Quality (AHRQ) manages a Technology Assessment Program, which produces Health technology assessments. Although AHRQ has produced relatively few assessments, and many of these are now outdated, the available assessments look very thorough and reliable. I don't see any problem with mentioning these assessments in the guideline and using them as sources if relevant and up-to-date. Boghog (talk) 21:23, 11 January 2025 (UTC)[reply]
@Boghog: Thanks, I know AHRQ. If I am reading that page correctly, they have done one assessment since 2020, and it is about one particular treatment for sleep apnea. They have a list of reports between 1990-2020, and by line count there are 160 of those reports. These may be good reports but given that there are so few of these, I question whether we should recommend these as the best kind of report from a medical or scientific organization, which is what that image is doing.
This section MEDORG is supposed to be about when we accept medical info from outside the usual scholarly review articles. Right now it is framed for what kinds of organizations make claims we can accept. Could there be another dimension for what kinds of documents they produce, like guidelines, health technology assessments, and any other claims? Do you have any insight into what other categories of high-quality, non-scholarly-journal, expert publications exist besides these two? Bluerasberry (talk) 22:15, 11 January 2025 (UTC)[reply]
Health_technology_assessment#By_country lists agencies outside the U.S. that do these types of assessments.[1][2] Since this is the English language Wikipedia, the most likely to be used sources are:
  • Australia
  • Canada
  • Ireland
  • New Zealand
  • Norway
  • Sweden
  • United Kingdom
  • Singapore
  • United States
Boghog (talk) 07:58, 12 January 2025 (UTC)[reply]

References

  1. ^ "Countries with National agency/unit/committee that produces HTA reports for the Ministry of Health" (PDF). WHO. 2021-04-08. Archived (PDF) from the original on 2024-02-03. Retrieved 2024-02-03.
  2. ^ "INAHTA Members List". International Network of Agencies for Health Technology Assessment (INAHTA).
There are also several journals devoted to publishing Health Technology Assessments (HTAs):
Boghog (talk) 10:45, 12 January 2025 (UTC)[reply]

References

  1. ^ "About the Journal". Global and Regional Health Technology Assessment. AboutScience Srl.
  2. ^ "Aims & Scope". Health Technology Assessment in Action. Tehran University of Medical Sciences.
@Boghog: Wow, thanks! I will think of a way to add some of this to health technology assessments.
Let me take some time to think about all this. I am still skeptical. I clicked on a few of these, and in the ones I checked, I still see the same thing - major national organizations typically publish fewer than 1 of these per year.
One interesting one that I found was an HTA on HTAs, and it even has a CC license, so maybe I can archive that in Commons and share it in the wiki article. Thanks for the great reactin, will get back. Bluerasberry (talk) 14:28, 12 January 2025 (UTC)[reply]
Bluerasberry - I think a few points are relevant to mention here.
1) A lack of volume of HTAs doesn't really detract from stating that they are generally the most reliable. Inherent to the HTA process is conducting meta-analysis, systematic review, and combining this with ethical and often economic perspectives using a standardized protocol. It is inherently time-consuming, and there will therefore not be as many as other source-types.
2) The fact that AHRQ in particular publishes so few HTAs is because funding has been successively cut as their findings were not appreciated by certain congressional groups. This has happened under multiple administrations, both because of a personal spat that a congressman had with one of the reports in the 1990s/2000s, and then again in the late 2010s with some health care lobby groups. This says absolutely nothing about the quality.
3) I agree that the diagram is somewhat simplistic, but this should be elaborated upon in text. The fact is that not all HTAs, like any source have the same quality. Something by NICE and something by a regional HTA body will not be the same, just like every review isn't the same. This doesn't detract from how the methodology is generally the best.
There is also the University of York - Centre for Reviews and Dissemination - HTA Database [1], which allows you to search for HTAs.
CFCF (talk) 16:02, 22 January 2025 (UTC)[reply]
That website points users to https://database.inahta.org/ It looks like about a thousand HTAs are listed for each year recently, with 23,000 total listed. At a glance, they are mostly not on hot-button politicized topics. For example, searching for transgender finds only one (on phalloplasty), and the MeSH term gender dysphoria finds only three (from UPenn, Sweden, and Spain).[2] There are 20 on cannabis use.[3] There are only 13 on abortion (nine of which are not in English),[4] and another 13 on miscarriage.[5] There are none on assisted suicide and only one on euthanasia.
As an example of filtered results, there are 34 about COVID-19 from the US.[6] That's just 15% of the total for that category. WhatamIdoing (talk) 20:07, 22 January 2025 (UTC)[reply]
The issues with AHRQ haven't been on topics that are controversial among the general population, rather specific findings which call into question either lucrative treatment practices, or treatments that are near-to-heart for specific congressional delegates due to personal views (such as on back pain). This NY Times article is a decent overview, if not very in-depth [7].
If anything the concept is thoroughly "unsexy" and uninteresting for the general public - which means that cuts can be made without much concern. The issue has been ongoing for quite some time [8]
Also, you are right the INAHTA database has replace the York University one (even though the latter still works to some degree). CFCF (talk) 13:06, 23 January 2025 (UTC)[reply]
P.S. Also strange that the database only found those three on gender dysphoria, as there are at least two more that I know of by the Swedish SBU: A systematic review of hormone treatment for children with gender dysphoria and recommendations for research (Hormone treatment of children and adolescents with gender dysphoria) & Hormonbehandling vid könsdysfori – vuxna (in Swedish)
CFCF (talk) 13:14, 23 January 2025 (UTC)[reply]
Those two might not have the keywords that I searched for.
I think that editors might be happy to use these if they were more aware of how easy it is to find them. For example, @Oolong would probably appreciate https://database.inahta.org/article/19028 which says that early intervention Applied behavior analysis is probably not cost effective. WhatamIdoing (talk) 19:37, 23 January 2025 (UTC)[reply]
P.S.WhatamIdoing - it seems that MeSH-search doesn't work so well in the database, and that those two were not registered as HTAs because they intentionally left out ethical and economic aspects - so they are just, if you will, high quality systematic reviews. CFCF (talk) 21:58, 23 January 2025 (UTC)[reply]
Before I get too involved in the details here - let me be clear also in this discussion that I have worked with and for SBU, with funding from Wikimedia Sweden as well as being a Wikipedian in Residence. (This was formerly very clear on my userpage, before my hiatus - and nothing I have made attempts to hide. I have now reinstated this disclosure).
Adapting the guideline to be clearer would seem a good idea. In general I think the use of visual aids, including the multitude that I made years ago - remain okay. One can discuss rearranging the order - but as a rule I think it holds. As for the comment by you Bluerasberry, that both national and international guideliens have been shown fallible through the pandemic, yes I agree. However, I think any change needs to be very carefully thought through. To some extent content by national expert agencies remains a cornerstone of Wikipedia - and most material is still very high quality. With regard to HTAs, I really don't see these as impacted by any controversy from the past years. In essence the strength of the HTA process lies in the method (e.g. SBU method). As long as we are also using the pyramids for hierarchies of evidence, and clarify in text that all of that is just a general guide - I don't see why we would remove this specific visual aid from MEDRS either.
In general I find HTAs tend to fly under the radar from the general public, because the questions they raise are often very technical, somewhat narrow, and mostly of value for policy-makers and health financing groups.
As a first point of clarity, I would probably suggest highlighting the rise in the past 5+ years, of commercial HTAs (such as legit vendors as far as I can see such as IQVIA and perhaps Clarivate, but also some commercial actors stating that they do HTAs without a clear process). Mostly this seems benign, with the bigger issue being that these can be very hard to access, costing in the range of 5-50.000 USD to read. I don't really think this poses a problem for Wikipedia, but it could potentially in the future if misuse of the HTA-term becomes an issue.
Some good guides for language are from the EU Commissions regulation 2021/2282 on HTAs and the WHO page on HTAs, which also has several resolutions on HTAs. The language there, including definitions, should be CC-BY or PD.
CFCF (talk) 21:49, 23 January 2025 (UTC)[reply]
P.S. A terminological point - this is the EU definition:
(1) [...] Health technologies encompass medicinal products, medical devices, in vitro diagnostic medical devices and medical procedures, as well as measures for disease prevention, diagnosis or treatment.
(2) Health technology assessment (HTA) is a scientific evidence-based process that allows competent authorities to determine the relative effectiveness of new or existing health technologies. HTA focuses specifically on the added value of a health technology in comparison with other new or existing health technologies.
The point to be made here is that technology in this sense is a very broad term which should include pretty much any treatment.
CFCF (talk) 21:54, 23 January 2025 (UTC)[reply]

CFCF, I've heard several editors express concerns recently that are less about "controversy from the past years" and more about "the NIH and CDC during the next four years", though for editors specifically working in trans-related topics, the Cass Review seems to have caused a lot of pain and fear last year. WhatamIdoing (talk) 00:06, 24 January 2025 (UTC)[reply]

I've changed the indenting for clarity here - and intend to respond. This is an intricate question, and I will get to it. CFCF (talk) 07:17, 24 January 2025 (UTC)[reply]
Fundamentally, you raise important questions, and I’ve spent some time thinking about it. Recent developments—specifically the unprecedented gag order on federal agencies—are noteworthy and may signal a need for adressing the issue in the future. However, we should be cautious about reacting too quickly. This particular restriction is significant but also fairly limited in scope, and it is still too early to understand the broader effects of what follows.
If I may elaborate on a few points, especially regarding why I view this as potentially relevant for MEDRS more broadly but less so for HTAs: By definition, these assessments aim to be apolitical and methodologically robust. Their credibility rests on the quality of the systematic review and other established processes involved. Unless there is evidence of compromised methodology, I see no immediate need to reassess their reliability. Of course, if anything changes substantially, we can address it then.
With regard to the Cass Review: Although I am not intimately familiar with it, the very existence of an article dedicated to it suggests two things: (1) it was influential, and (2) it was controversial. Both factors are indicated by how editors found sufficient motivation to produce a detailed entry. At a glance, it seems appropriately discussed at Puberty blockers, where its findings—and the criticisms—are laid out.
This brings me to what I view as the core duty—of Wikipedia: to present the best possible sources in an unbiased way. The current coverage of the Cass Review appears to meet that standard, detailing both the findings and criticisms. Whether we agree with its findings remains slightly beside the point, and steers close to criticism of WP:NPOV - which I disagree with.
Regarding concerns about political interference in expert agencies like the CDC or NIH: I remain cautiously optimistic about the independence of these institutions, given their long administrative traditions. While budget cuts could certainly constrain the scope of their work (as happened with AHRQ), there is little indication that their existing outputs would be twisted to say something other than what evidence-based methodologies would conclude. A more likely chilling effect, if it does occur, might be the decision not to investigate certain issues in the first place.
On a more general note, not all government outputs are created equal. In the U.S., for instance, there is a distinction between: Politically shaped outputs, such as Congressional Oversight and Investigations reports (often produced by offices staffed by political appointees), and Technocratic or expert-driven outputs, such as reports by the Congressional Research Service or by independent agencies.
A similar setup exists in many other countries. Sweden’s Government Public Inquiries (SOU) come in different flavors: some are penned by political appointees, while others come from state agencies and are less likely to be politically slanted. The UK also has a multi-layered system involving Quangos, agency reports, commissions, etc.
I think covering these distinctions may be helpful, and I'm happy to discuss how we could weight them. CFCF (talk) 16:37, 25 January 2025 (UTC)[reply]
Another point, getting back to you Bluerasberry - NIHR and SBU are both partners with PCORI - so there is interest among HTA bodies for patient-centered outcomes through something called EViR (The Ensuring Value in Research Funders’ Collaboration and Development Forum).
I don't think this should change the fact that patient organizations are below the cut-off for MEDRS - rather I wonder whether generic descriptions of patient-centered-outcomes actually constitutes biomedical statements, and whether MEDRS at all needs to apply unless there is a statistical/treatment statement (or similar). Just stating that people find a specific outcome relevant doesn't need MEDRS in my book. I don't see that question being raised in Wikipedia:Reliable_sources/Noticeboard/Archive_464#Patient-Centered_Outcomes_Research_Institute_(PCORI).
With regard to the specific statement about incidence of side-effects there - I do find that the it probably needs a MEDRS source. I am on the wall as to whether PCORI should be authoritative editorial board or patient-interest-group. However, I think we are right to be very sceptical of patient-interest-groups in general when it comes to statistics or treatment. Not least from personal experience in research and government work. The groups are often very well intentioned, but lack statistical and medical knowledge, as well as often being unable to understand priorities and crowding-out effects, and sometimes not even questioning whether their statements make much sense. (For example, I had to stop one patient-interest group from stating in a summary of my paper that 50% of the sample had poor outcomes, because we had binarized the outcome variable to use it in a specific regression model and had set 50% as worse and 50% as better, by definition. They had missed that their statement, which they wanted to relay to a news agency really read "The half of the sample with worse outcomes, made up 50% of the sample."). Of course there are better and worse organizations, but as a general rule - I think that for MEDRS, the heuristic still holds. CFCF (talk) 19:00, 25 January 2025 (UTC)[reply]
Had one more thought - maybe you're right that PCORI is a research group - and this is where the distinction lies. I also got thinking about CRUK, which I think is authoritative whereas some professional bodies are less reliable, such as the U.S. ME/CFS Clinician Coalition. It's not straightforward... that is without even getting into psuedo-professional bodies, that only say they are made up of professionals, but really aren't. And then there are organizations that are professional interest groups in fields they don't work in, such as International Physicians for the Prevention of Nuclear War (an admirable cause, but likely not MEDRS). CFCF (talk) 19:11, 25 January 2025 (UTC)[reply]
Another thing that strikes me is that we do not consider Umbrella reviews at all. I am leaning towards redrafting and updating some parts of the guideline - what do you think Bluerasberry, WhatamIdoing? CFCF (talk) 18:16, 26 January 2025 (UTC)[reply]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Identifying_reliable_sources_(medicine)&oldid=1271999682"