Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

>You can make the data say almost anything.

A coworker had a saying:

"Data is like a prisoner of war. If you beat it around enough, you can make it tell you almost anything."



Yep. I actually am a co-author of a paper that has been published and while I was working on it, I realized how easy it would have been to make the data agree or disagree with the hypothesis. Based on the feedback of reviewers, you see they don't dig into how you get your data and it seems as if its based on "good faith."

I could have made a mistake, or I could have been malicious. I don't think they would have caught it because it would have involved hours and hours of work on their part.


With current complex data processing pipelines it is almost trivial to add e.g. a wrong sign to some variable to get "results" from data that doesn't contain any. I've had many "results" disappear after I found a bug. I could have probably gotten papers quite easily by ignoring the bug.

I'm quite certain a huge share of "results" are due to bugs. Probably many of my own too even though I stress about this constantly.

An intentional bug would be practically impossible to show to be intentional. With notebook/REPL style analysis there wouldn't necessarily even be any documentation of the bug. I'd wager it actually happens, and even surprisingly often. We only know of fabricators who are bad at fabrication.


This goes back to "The nature of things betrays itself more readily under the vexations of art than in its natural freedom.” — Francis Bacon




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: