Critical Thinking, by Adam Wiggins
Table of Contents
 

2. Data is put into context, i.e., "compared to what?" Data in a void is useless, regardless of its correctness.

This one is tricky because it is not intuitive to the mind of man. When a piece of data without context is encountered, it can seem like overwhelming evidence in support of a particular conclusion. For example: "robberies in our city tripled last month, therefore we have to increase funding to the police department." In the absence of context, this makes perfect sense. A 200% increase in the number of robberies does seem pretty dire. But there's not enough context to make a conclusion about whether increasing police funding makes sense. It could be the case that there was one robbery last month, and three robberies this month. If the historical average for the city is around two robberies per month, then the 1 -> 3 increase is not really an increase at all, but just the normal fluctuation of statistics. An average of two per month doesn't mean that there is always exactly two per month, just that there is usually about that many each month.

But there are other things to consider. What is the national average for number of robberies per month for comparably-sized cities? If the average is much higher than even our new rate of three in one month, it seems unlikely that there is cause for alarm. What qualifies as a robbery? Often statistical oddities can occur when the sources collecting the information change their criteria. For example, perhaps previously only robberies in homes or places of business were counted, but this month the police department started grouping muggings into the robbery category as well.

Sometimes looking at the numbers a different way will produce a different result. For example, what is the total value of items stolen? It may be that last month we only had one robbery, but it was a bank holdup and the perpetrators made off with hundreds of thousands of dollars. This month we may have had three, but two of them were high school bullies demanding lunch money from their smaller classmates, and one was a back-alley mugging at knifepoint where the perpetrator only got the $40 that was in the victim's wallet. In this case the number may have increased, but the total severity has decreased.

The importance of context cannot hardly be overstated. Without context, all the possibilities will not be visible. In fact, you should be suspicious of a simple piece of data which seems to rule out other possibilities. One approach in this situation is to start creatively speculating as to other theories which could explain the data, and then look go and look for additional data which will confirm or deny the new theories. Here's a real-world example of such an occurrence.

For some time it has been speculated that Google, the wildly successful Internet search engine company, would enter the web browser market. Supporting data includes Google's financial support of the development of the open-source web browser Firefox, and the fact that such a move is an obvious and viable strategy given their existing business model. Data supporting the reverse conclusion is that their representatives state that they have no plans to introduce a browser of their own, and (on a ongoing basis) simply the fact that they have have not done so yet. At one point, the following simple piece of data was interjected into the argument: Google had registered the domain name gbrowser.com.

On the face of it, this is the perfect piece of information. It is easily verified by anyone with an internet connection by checking the whois record for gbrowser.com. And what other purpose could Google have with such a domain other than introducing their own "G"-branded browser?

Here's the answer: if you check the domain records, you will see that Google has registered hundreds of domain names starting with the leter G, including: galbums.com, gbeam.com, ggovernment.com, ggroups.com, gsend.com, and gtaxes.com. In this context, it becomes immediately obvious that the gbrowser.com domain registration has only minor significance, because it appears to be only a cautionary land-grab move by a company that is hedging its bets for the long term. Although still relevant, this new data hardly sways the argument much.


Table of Contents