Skeptic's list

Published on December 16, 2013

Skeptic's list

    Loud statements require convincing evidence.
    Karl Sagan ( original )




    Preamble


    One evening, many years ago, in my first year in the glorious city of Dolgoprudny, walking with friends in the corridors of the university, when suddenly by chance, we saw an announcement on the door: “Lecture: Slavic runes. Chudinov". It is worth noting that this is not the most popular topic of lectures for the physics and technology department. Out of curiosity, we looked at the light and even honestly tried to understand what they were trying to tell us. After about ten minutes, we could no longer restrain our laughter and tried to ask in a soft form whether the author really believes in what he is saying and whether a number of scientific principles violate his theories. The author honestly admitted that they are not really trying to follow some principles there, but are trying to understand the essence of the issue and in general everything that he says is pure truth - look at these photos, they prove everything - here it is for sure “ Mara "is written.

    What is the bottom line from that lecture? Not the last university holds a lecture within its walls of dubious quality (with the sale of books by the speaker, of course), and not only does it serve as a free platform for such personalities, it also beats on its status. It would seem that this is perhaps an isolated case. For example, the same lecture was delivered in the lecture hall of the Polytechnic Museum. Subsequently, the administration recognized this lecture and a number of others as pseudoscientific and even organized an excellent lecture " What is amateur linguistics ", delivered by academician Zaliznyak .

    However, it turned out that this is no exception.

    This has never happened and here again


    Today I was surprised to find the next post.


    To even greater surprise, I found a corresponding post on Habré. Of course, I cannot say that this work is pseudoscientific. But as in many other cases, we have reason to doubt.

    To illustrate the general idea, consider a few examples found on Habré and beyond. As a basic example, we will analyze several articles on the well-known and popular in computer science SAT problem . The task has a rather simple formulation and therefore has received sufficient attention among non-specialists who offer a variety of solutions. As a rule, most of these works do not satisfy the basic scientific principles of writing articles and therefore are an excellent example for parsing errors.

    At the same time, this task is widely known and was covered several times on Habré to be a good illustrative material for the general idea. We will try not only to analyze these works, but also to identify some common features between the “dubious” works and try to use the obtained rules for the Planetary Energy Transmitter project mentioned above.

    Solution of the problem P? = NP


    You don’t need to go far for examples: here , here and here , thousands of them! The search yields even more results, but these are enough to illustrate.

    For an expert in SAT, most likely everything will be clear right away, as soon as he sees an article with proofs, but imagine that we have absolutely no knowledge in the field of computational complexity and we are trying to figure out how much we should trust this work.

    The first thing to think about is what the experts say. The Wikipedia article leads us to the following work: " Reasons to believe by Scott Aaronson ", which is really an expert, we can see by looking at his CV. The author consistently argues that not one of the modern mathematical techniques can give any answer to the question about NP vs. P.

    Second question, where was this article published? Even a superficial search by query: “computational complexity conference” gives a good idea that a number of outstanding experts (as well as organizations such as IEEE, ACM etc) work on this task and in the subject area, who are able to evaluate the work in detail.

    The third question, how is this work fundamentally different from others? This is a rather interesting question, once I was lucky enough to attend a lectureDonald Knuth at a summer school dedicated to SAT / SMT and in an informal conversation, the phrase sounded that by the way the list of links in the work looks and the literature review, you can understand how deep the article is. The intuition here is about the following: if the author gives a qualitative detailed review of the literature, including contemporary works from top conferences / magazines, then he has a good understanding of the subject area and the positioning of his work.

    The list goes on and on, but we are actually interested in a more general question than the NP vs. issue P.

    Abstract Petrick in a vacuum

    On the hub, as on many other resources, there are a huge number of articles about the results of which only experts can give a definite answer. Let me give you an example of several works from the closet: here , here and of course the masterpiece "Belarusian military are exploring torsion fields."

    Of course, no one is a specialist in each field, and therefore, if you publish technical calculations on a specific subject area on Habré, 99% will not be able to delve into the details and conduct substantive conversation at the level of specific formulas, evidence and theories. This does not mean that the community cannot evaluate the quality of work according to some formal criteria.

    We come to the old problem of determining the quality of popular science material presented to the general public. And here I absolutely would not want to reinvent the wheel. Many organizations face a similar problem and we can evaluate their solutions and try to understand whether we can adapt them for Habr.

    TEDx letter


    The most appropriate and competent solution, in my opinion, is the letter TEDx A letter to the tedx community on tedx and bad-science.

    Why TEDx? In many ways, TEDx and Habr are similar: the main value of the resource is user-created content. On the one hand, the thematic resource (various topics in TEDx and Hubs), on the other hand, the material is presented to the general public. Neither TEDx nor Habr conduct and are not intended for the publication of original scientific research and suggest that the material should be understandable to a layman at least at the level of ideas, an example of an excellent article in this format. It cannot be said that the resources are similar in everything and therefore it is impossible to mindlessly copy the solution and apply it left and right, but using the specifics of the resource, we can take the first step in adapting and creating such a solution.

    Next, we will provide a short list of questions from the letter (more precisely, their adaptation for the Habr) that a skeptic should ask when he has a new “breakthrough” job and at the same time give a short comment on these issues on the project “Planetary Energy Transmitter”.
    • This work did not convince a sufficient number of experts: the only confirmation of the correctness that we see comes from the academician of the RAAS and the Russian Academy of Natural Sciences
      image
    • The work has no experimental confirmation repeated by other scientists ( Mozart effect )
    • The authors make incredibly self-confident statements: “This is an endless source of natural energy for all of humanity”
    • They simplify the picture so much that it is impossible to refute the content: Popper's criterion , there must be a feasible experiment that could refute this theory (preferably before 10 ^ 6 rubles are transferred to the authors)
    • R&D: infinite energy, time travel, alchemy, etc. see the comment above about infinite energy.
    • The work contains a huge amount of general information that does not allow the reader to understand the work: this item, like the rest, is subjective, you can get a personal assessment by going over the material
    • There is no serious scientific reputation: if this work is published in Physical Review Letters, then at the same time I will transfer 10k + to the development of the project
    • The only authors have the same knowledge: how so for 100 years, no one else has come up with how to assemble the Tesla Tower?
    • I have no third-party sources of information about the project: I could not find a single third-party resource giving an expert assessment of this project
    • The original letter is labeled “GREAT RED FLAG”: sells related products, services or a plan
    • The authors show excessive importunity in accessing the site: from my position it is difficult to analyze, but it is known that the administration has already deleted the article from the resource


    As in any other case, all these statements and questions cannot clearly say whether the work is of high quality or not, but they allow the reader to get some general assessment of the quality of the work from the point of view of basic scientific principles.

    Instead of a conclusion


    This article is primarily aimed at developing a discussion about what criteria should be applied to articles on Habré. Should the community have a list of questions that need to be answered before someone can use Habr as a platform for promoting their ideas?

    Are there any general principles that we must follow? The community regulates interest in articles through voting, but does it mean that we should not have any standard, what should be the articles? Will it be censorship and pressure if, before the publication of such articles, some requirements will be imposed on them to comply with certain standards?

    Of course, there is no silver bullet that will solve all problems, but the problem itself will not disappear if ignored.