Chapter 13: Scientific Reasoning

  

Chapter 13: Scientific Reasoning

•   Science is regarded as one of the greatest achievements of human beings, alongside art, music, and literature. Technology is a product of science, and it has a huge impact on our lives. But the core of the scientific methodology is hypothesis testing, an essential part of critical thinking. 

•   Hypothesis testing is a matter of gathering evidence to select the best hypothesis. Hypothesis testing is not just for scientists: in any type of career, we have to solve problems, and hypothesis testing helps us find the best solutions to our problems. 

•   Suppose your mobile phone is not working. Is the battery dead or is the phone broken? You try to recharge it to see if it works. If it does the phone wasn't broken. This is hypothesis testing. 

•   Or think about how to improve your health. What should you eat and what exercises should you do? You need to gather information and evaluate different theories before coming up with a plan. This also involves hypothesis testing. 

•   There are two noteworthy features of hypothesis testing. First, it is based on evidence, not on feelings, tradition, popularity, authority, or personal preferences. Second, hypothesis testing is often difficult to prove that a theory must be correct.

•   This does not mean we should give up scientific reasoning. We do our best to identify the theory that has the highest probability. It is too bad if we turn out to be wrong, but such is the uncertainty of life. This is like investing in the stock market. Nobody can predict the future accurately all the time. But someone who is correct 70% of the time will already be doing very well. 

1.       THE DEAR METHOD:

•   We now look at the four main steps in hypothesis testing. We call this the DEAR method, which reminds us of the first letter of the keyword in each of the four steps. 

a)      Define the hypothesis to be tested. 

b)      Collect the evidence for and against the hypothesis. 

c)      List all the alternative hypotheses. 

d)      Rank them and pick the best one to accept.

•   Where do these hypotheses come from in the first place? The answer is that they can come from anywhere. They might arise from the problems that we are trying to solve or from observations that we have made. 

•   For example, we might have seen lots of white swans and so we wonder whether it is true that all swans are white. However, what makes a hypothesis scientific is not how it comes about. A scientific hypothesis is a clearly specified statement that can be tested in principle. Many scientific theories have been inspired by dreams or wild speculations. They can still be acceptable if we have good evidence showing that they are true. 

1.      Step 1: Define the theory to be tested: The first step of hypothesis testing is to define clearly the hypothesis that is to be evaluated and make sure we know what it means. If the meaning of a hypothesis is unclear, it will be difficult if not impossible to test it. Here are a few things to bear in mind: 

•   Clarify keywords: Some people think that everyone is surrounded by an aura field, a field of energy. To test this hypothesis, we need an explanation of what an aura energy field is. Is it the same as the electromagnetic energy that is studied in physics? If so then there are ways to test its presence. In fact, this is probably true since our bodies have warmth and so they emit heat, which is a form of electromagnetic energy. But then the aura field is not something very remarkable. On the other hand, if this is not what is meant by an aura field, then further clarification is needed. Otherwise, there is no way to test the hypothesis and we have no reason to believe it.

•   Be precise: A more precise hypothesis is less likely to be misunderstood. Take the claim that gold is a good investment. Its meaning is not obscure, but more precision will provide better guidance. Are we supposed to buy real physical gold or stocks that are linked to gold? Is this a good investment for the short or long term? What kind of return are we talking about? Taking these concerns into account might give us a more concrete claim, such as investment in physical gold or gold stocks will beat inflation and perform better than major stock markets in the next five years. 

•   Clarify the scope of the hypothesis: The scope of a claim is the range of things the claim is supposed to be true of. Take the claim "swans are white." Is this true of all swans, most of them, or just some of them? The scope of the claim makes a big difference to the evidence we need to check whether the claim is true. "All swans are white" is false because there are black swans in Australia. But if the claim is changed to "some swans are white," the existence of black swans becomes irrelevant and what matters is whether you can find at least a few white swans. But consider also "most swans are white." Knowing that there are white swans and black ones will not help us decide whether it is true. We need a detailed statistical survey to find out. As you can see, the scope of a theory makes a big difference to the evidence needed to test it. 

2.      Step 2: Gather the evidence for and against the theory: To evaluate a hypothesis, we gather all relevant evidence.

•   There are two types of evidence: Supporting evidence is facts that increase our confidence in a hypothesis. Counter-evidence is facts that decrease our confidence.  Generally speaking, a piece of supporting evidence provides a reason for thinking that the hypothesis is true. This happens when some fact obtains which is what we should expect given the hypothesis. Counter-evidence is the opposite. For example, the hypothesis that all swans are white implies that the next swan we see will be white. So if we do see a white swan, that counts as supporting evidence, and if we see a black swan, that would be counter-evidence. Or take a different example. Lots of dark clouds are supporting evidencefor the hypothesis that it will rain soon. If the air pressure is low, that is another piece of supporting evidence. But a bright and clear sky will be counter-evidence instead. What if it is a windy day? This is neither supporting evidence nor counter-evidence since wind makes no difference to the likelihood of rain. 

•   Evidence can differ in strength: Seeing a single white swan is weak supporting evidence that all swans are white. Finding lots of white swans in different countries would be much stronger evidence. But this is not conclusive evidence, evidence that proves or disproves a hypothesis beyond a reasonable doubt. Unless you have seen all the swans there are, you can never be sure that they are all white. On the other hand, seeing a single black swan does count as conclusive counter-evidence against the hypothesis that all swans are white. So when we gather the evidence we have to decide two things: first, whether it is supporting evidence or counter-evidence, and second, whether the evidence is weak, strong or conclusive.

•   The more evidence the better: Finding more evidence in support of a hypothesis means we can be more confident that it is true. So avoid relying on a single piece of evidence. But remember that a hypothesis can be wrong even if we have lots of supporting evidence that is not conclusive. Furthermore, human beings are prone to pay more attention to evidence that agrees with their own opinions. So if you agree with a hypothesis, make a special effort to come up with counterexamples, and seek out people who disagree with you to see if they know of counter-evidence that you do not. 

3.      Step 3: List all the alternative theories: The world is a complicated place and things are often not what they seem. When we have a theory that seems to explain the evidence, we should actively consider whether there are alternative theories that provide even better explanations. If you have a severe stomachache, it might be due to something you just ate. But it could also be acute appendicitis, which can be life-threatening. 

•   An alternative theory is one that is (1) distinct from the theory you are considering and (2) broadly consistent with the evidence you have observed. For example, it is now widely acknowledged that the Earth's temperature is increasing, and this global warming is caused by pollution and other human activities. But an alternative theory to consider is that the temperature increase is only part of the natural fluctuation in climate. Sometimes the Earth gets cooler and sometimes it gets hotter. 

•   Sometimes we can rule out an alternative theory by getting more evidence. To decide whether global warming is due to natural climatic fluctuations, scientists look at historical records and ice core samples to measure the extent of natural temperature variation in the past and see whether this accounts for recent global warming, and the conclusion is negative—global warming is due to recent human activities. 

•   Coming up with alternative theories requires knowledge and imagination, and the truth might not be obvious. Human beings are often affected by biases, and they view the world through the perspectives they are most attached to. Some people like to invoke the supernatural whenever there is something that is puzzling— for example, a butterfly refused to fly away after Daddy passed away so it must have been his reincarnation. Others like to resort to a divine command, such as it is God's will. Still, others like to blame things on their favourite target, saying it is the fault of the government/society/my teacher/ my parents/my girlfriend, and so on. Good scientific reasoning requires us to actively challenge our default explanation. This is not just a matter of being open-minded. We need the courage to accept that our favourite or most comfortable point of view might not be the correct one. 

4.      Step 4: Rank the theories and pick the best one: Once we have come up with a list of alternative theories, we can evaluate them carefully and pick the one that is most plausible. This method of reasoning is known as inference to the best explanation, which is of the following form: 
We have a set of evidence E.
X, Y, Z, … are all theories compatible with E.
X provides the best explanation of E.
X is most likely to be true.

•   But how do we find out which theory provides the best explanation? The answer is that we need to appeal to more general theoretical considerations.

5.      Predictive power: Predictive power is about the quantity and quality of the predictions made by a theory. 

•   A theory that generates no prediction at all fails the minimal requirement for a scientific hypothesis. A claim that can not be tested can perhaps still be meaningful. It might even be true. But if we believe the claim, it only can be a matter of faith and not reason since there is absolutely no evidence to justify the belief. 

•   The quality of prediction is about precision and accuracy. If an astrologist predicts that an old man is going to die within 20 years on the basis of the position of the planets, and the man dies 10 years later, this is not too impressive. But suppose the astrologist predicts that the man will be crushed to death by a jet engine falling from the sky exactly 20 years and one day later. If the prediction turns out to be right, this would be a very impressive accomplishment. With a few more such correct predictions, we might even become believers in astrology! 

•   When the predictions of a theory turn out to be wrong, it is possible to save the theory by challenging some of the auxiliary assumptions. These are assumptions we make about the theory or about the experimental setting that helps us generate the prediction. For example, to test the hypothesis that water freezes at 0°C, we can use a thermometer to measure the temperature of ice, but the auxiliary assumption here is that the thermometer is accurate. 

•   When a theory fails to be confirmed by evidence, one way to save the theory is to reject some of these auxiliary assumptions. For example, some people claim they have telepathic abilities that enable them to read other people's minds. When being tested repeatedly in an experimental setting, they inevitably fail to perform better than others who are simply guessing. They might say that the scientists carrying out the experiments have hostile and negative thoughts that interfere with the concentration and abilities of these practitioners. This justification (or excuse) is called an ad hoc hypothesis, one that is introduced solely to avoid disconfirmation of a theory. 

6.      Mechanism: Sometimes two events can be correlated without there being a direct causal link between them. There might be a positive correlation between ice cream sales and the number of shark attacks in Australia, but it does not mean selling more ice cream causes sharks to attack human beings. This correlation might seem strange until we note that shark attacks happen more often in the summer when more people eat ice cream. This underlying explanation allows us to understand the link between the correlated events. 

7.      Fruitfulness: The last point about the mechanism is related to fruitfulness—whether a theory helps us make surprising or unexpected predictions that turn out to be correct and whether the theory helps us detect and explain connections that we would not have noticed otherwise. 

•   Consider the theory of plate tectonics, which says that the surface of the earth is covered by a series of plates floating on a viscous mantle and moving in relation to one another. After the theory was first developed in the 1960s, it generated a host of new predictions and explanations which were subsequently confirmed. For example, geologists were able to gain new insights as to why earthquakes tend to be concentrated along oceanic trenches and spreading ridges (because they correspond to frictional boundaries between plates). 

8.      Coherence: There are two kinds of coherence. First, a theory should be internally coherent— that is, logically consistent. It is possible that a useful theory is not fully consistent when it is first proposed. But inconsistency does tell us it is not completely true, so it should be revised and improved somehow. 

•   The other aspect of coherence is that good theories should be consistent with other well-confirmed theories and facts. For example, people such as Uri Geller claim they can bend metal spoons with their minds. This certainly goes against common sense and science. A simpler explanation is that magic or fraud is involved. To reject this more mundane explanation, we need to carefully test these people under tough conditions, and nobody has managed to pass such tests so far. A lot of alternative medicine and claims about the supernatural should be treated with similar caution. 

9.      Simplicity: Many scientists believe strongly that we should search for simple theories if feasible. But why should we prefer simple theories? 

•   Simple theories have a number of advantages. First, they are often easier to apply, so there is a practical reason to prefer a simpler theory. Second, a complicated theory suggesting lots of different entities would require more evidence to support it. Finally, looking for simple theories coincide with the search for unifying causal mechanisms in our explanations. They help us understand the connections between different areas and offer a deeper explanation of the world. 

2.      RELYING ON EXPERT OPINION

•   We cannot possibly know everything, and inevitably we need to rely on other people's analyses and opinions, especially when it comes to scientific or technical matters. But deferring to an expert does not mean we can stop thinking critically. In particular, we must check whether the expert is credible, accurate, and unbiased. Here is a checklist of questions to think about: 

•   What exactly is the expert’s opinion? Sometimes media headlines can misrepresent an expert's opinion. A scientist might say there is some weak evidence suggesting a correlation between A and B, but the headline might simply say "A causes B." So check the actual evidence carefully.

•   Is the expert in the right field? Information is more reliable when it comes from an expert who knows the topic well. In 2008, Europe's CERN laboratory was about to activate a giant particle collider for studying subatomic physics. But according to German chemist Otto Rössler, the experiment should not proceed because it might create mini black holes that could destroy the Earth. Rössler might be an expert in chemistry, but it does not mean he is equally an expert in particle physics. The collider is now in use and luckily we are still here.

•   Is the expert reliable? Does he or she have a good reputation?Sometimes the expert is anonymous, so we cannot verify the reliability of the expert. So be careful with information from web pages and blog posts with no citations. Furthermore, doctorate degrees and special titles might not mean much. Even experts can often get things wrong, so it will be useful to know the past record of their opinion. 

•   What is the context in which the opinion is made? We have seen that quotes can be taken out of context. And sometimes people are joking, being emotional, or not being serious for some other reason. An opinion expressed informally might be more of a hunch (crook) than a judgment made in a public forum. An opinion expressed by an official and reputable professional organisation is more weighty than one from an unknown scientist.

•   Do other authorities in the same field agree with the opinion? Weshould be cautious if authoritative experts disagree with each other. The credibility of an opinion increases when it is free from any serious dispute among independent experts in the same area. This is one reason why it is a good idea to get a second opinion when you are deciding what to do with a serious medical condition. Of course, with about seven billion people in the world, you can expect disagreement on just about every issue. We would need to evaluate the extent of the disagreement and the arguments on both sides. 

•   Is there any conflict of interest? If a tobacco company says smoking is healthy, the claim does not have a lot of credibility since the company would benefit from lying. Similarly, if a software company commissions an expert who then publishes a report saying that the company's operating system is the most secure one in the market, we should be somewhat sceptical of the report. However, it does not mean we should totally discount an opinion totally whenever a vested interest is involved. We need to consider each case individually. First, we should evaluate the evidence given for the opinion carefully and see if independent experts agree. Second, sometimes people also have a vested interest in telling the truth because they might lose more in the long run when being caught lying.

•   Is there any other source of bias? An opinion can be biased even if the person making the opinion does not stand to gain any material benefit. For example, if someone knows a person involved in a dispute, the relationship might affect his judgment as to who is right or wrong in the dispute. Objectivity can also be affected when someone is being emotional or in a heated discussion. Sometimes academics align themselves with a particular school of thought, and they might be dismissive of alternative perspectives. This can also be a source of bias that affects their reliability. 

END OF THE PART