It can be debated about whether that’s a logical fallacy. If you ask ten experts for their studies on something and you find a reasonable correlation (eg all the ones promoting smoking are funded by Tabacco lobbyists while the rest have a more natural distribution of results and don’t show any correlation of potential undue influence), it’s not a logical fallacy to suggest “hey, this argument is being made in good faith and the reason I think this is because of the funding sources for this person”. Indeed, we know this was an explicit strategy employed by Tabacco with lung cancer (they knew it was a problem but intentionally funded research to get the contrary results to muddy the waters) and oil companies with climate change (same thing).
Only if you believe that work should not be evaluated on its own merits. If the methodology is good, the sample sizes big enough, the statistics correctly done, etc., I don't care if Mickey Mouse wrote the study.
The climate change example is bad because oil companies have been actively suppressing correct evidence showing the existence of climate change since the 1980s. I'm less familiar with the tobacco studies, but I wouldn't be surprised if they've been doing the same thing, just on longer time scales.
> If the methodology is good, the sample sizes big enough, the statistics correctly done, etc., I don't care if Mickey Mouse wrote the study.
I’d agree with this but in the real world, we usually don’t and can’t really know if the methodology and statistical analysis was solid and rigorously followed as described in the paper – if it they were even adequately described. The only way to be sure is to replicate the study and that costs time and money (as per Brandolini's Law).
Evaluating incentives (e.g., follow the money) is a useful heuristic to evaluate the creditability and trustworthiness of a particular study.
Most people tend to try to engage the world with how it is rather than how they’d like it to be. It’s definitely not as simple as you make it out to be. They employ both strategies - they try to suppress real studies and they try to generate bullshit on top to confuse them situation / overwhelm the capacity of the system to deal with it.
Sure it’s an ad hominem attack to use past behavior to judge someone’s current or actions statements and taking actions as a result (eg ignoring them). And yet, is that a logical fallacy? I don’t think so but you seem to have an opposite conclusion.
I'm tempted to say that claiming it's naivete to expect people to do work to evaluate a research paper is naive, as well as ad hominem. If you don't have the time or expertise to do so, then do the rest of us a favor and don't try.
Now, if you're talking about using such criteria as a heuristic to decide which papers to put effort into evaluating, then we have another discussion to have.
That’s exactly what I’m saying. I don’t bother trying to evaluate a paper because a) my area of expertise only extends so far b) papers are largely worthless in their ability to communicate where the research done was valid. The replication crises and many many fraudulent papers surely would convince anyone that there’s a real problem with our current system. Maybe you’re a brilliant mind that can separate the wheat from the chaff. Most people, even leading researchers in the field, have trouble though so I don’t feel like I’m in bad company.
I am sympathetic with your point of view, but I find myself in a position of what Scott Alexander calls “epistemic learned helplessness”[0]. My powers of understanding the correct method of scientific research are limited. I understand that there must be some studies that are done correctly, but I don’t know which ones. I assume that these are the older studies that haven’t been refuted or retracted after a long time. So I end up trying to trust new research that doesn’t seem to stem from a conflict of interest… although it makes sense to me that things that researchers have some interest in, somehow, are the things that they study. It’s a very neat trap I find myself in! Nothing for it but to become intimately familiar with the methods of scientific research?
