Unearthing archaeological treasures of the past. A grave humanitarian crisis is unfolding in Ethiopia. An icy world is in meltdown, amid penguin population shifts. This is what I've learned. Magazine From the Editor. See Photos. One-on-one with Dr. Bees bed down in blooms, and more wonders. A visual timeline of every animal ever sent into space. Nature makes a comeback in Mozambique. These 12 tools help solve avian mysteries. Parisians want to recover a legendary river now buried under the city.
Latest Issues. Magazine November Issue. Meanwhile, the book is long on examples to convince you that there are great reasons to learn this stuff. The idea for this book was born not terribly long after my unfortunate experience in Mrs. I went to graduate school to study economics and public policy. For three weeks, we learned math all day in a windowless, basement classroom really. On one of those days, I had something very close to a career epiphany. Our instructor was trying to teach us the circumstances under which the sum of an infinite series converges to a finite number.
Stay with me here for a minute because this concept will become clear. The three dots means that the pattern continues to infinity. This is the part we were having trouble wrapping our heads around. One of my classmates, Will Warshauer, would have none of it, despite the impressive mathematical proof. To be honest, I was a bit skeptical myself. How can something that is infinite add up to something that is finite?
Then I got an inspiration, or more accurately, the intuition of what the instructor was trying to explain. I turned to Will and talked him through what I had just worked out in my head. Imagine that you have positioned yourself exactly 2 feet from a wall. Now move half the distance to that wall 1 foot , so that you are left standing 1 foot away.
And so on. You will gradually get pretty darn close to the wall. But you will never hit the wall, because by definition each move takes you only half the remaining distance. In other words, you will get infinitely close to the wall but never hit it. Therein lies the insight: Even though you will continue moving forever—with each move taking you half the remaining distance to the wall—the total distance you travel can never be more than 2 feet, which is your starting distance from the wall.
For mathematical purposes, the total distance you travel can be approximated as 2 feet, which turns out to be very handy for computation purposes. The point is that I convinced Will. I convinced myself. And when I do, it will probably make sense. In my experience, the intuition makes the math and other technical details more understandable—but not necessarily the other way around.
The point of this book is to make the most important statistical concepts more intuitive and more accessible, not just for those of us forced to study them in windowless classrooms but for anyone interested in the extraordinary power of numbers and data.
The problem is that if the data are poor, or if the statistical techniques are used improperly, the conclusions can be wildly misleading and even potentially dangerous.
Imagine that headline popping up while you are surfing the Web. According to a seemingly impressive study of 36, office workers a huge data set!
Clearly we need to act on this kind of finding—perhaps some kind of national awareness campaign to prevent short breaks on the job.
Or maybe we just need to think more clearly about what many workers are doing during that ten- minute break. My professional experience suggests that many of those workers who report leaving their offices for short breaks are huddled outside the entrance of the building smoking cigarettes creating a haze of smoke through which the rest of us have to walk in order to get in or out. Statistics is like a high-caliber weapon: helpful when used correctly and potentially disastrous in the wrong hands.
This is not a textbook, which is liberating in terms of the topics that have to be covered and the ways in which they can be explained. The book has been designed to introduce the statistical concepts with the most relevance to everyday life.
How do scientists conclude that something causes cancer? How does polling work and what can go wrong? How does your credit card company use data on what you are buying to predict if you are likely to miss a payment? Seriously, they can do that. If you want to understand the numbers behind the news and to appreciate the extraordinary and growing power of data, this is the stuff you need to know. But I have even bolder aspirations than that. I think you might actually enjoy statistics.
The underlying ideas are fabulously interesting and relevant. The key is to separate the important ideas from the arcane technical details that can get in the way. That is Naked Statistics. Students will complain that statistics is confusing and irrelevant. Then the same students will leave the classroom and happily talk over lunch about batting averages during the summer or the windchill factor during the winter or grade point averages always. The same data completion rate, average yards per pass attempt, percentage of touchdown passes per pass attempt, and interception rate could be combined in a different way, such as giving greater or lesser weight to any of those inputs, to generate a different but equally credible measure of performance.
Is the quarterback rating perfect? Does it provide meaningful information in an easily accessible way? I am a Chicago Bears fan. During the playoffs, the Bears played the Packers; the Packers won. There are a lot of ways I could describe that game, including pages and pages of analysis and raw data. But here is a more succinct analysis. Chicago Bears quarterback Jay Cutler had a passer rating of In contrast, Green Bay quarterback Aaron Rodgers had a passer rating of That tells you a lot of what you need to know in order to understand why the Bears beat the Packers earlier in the season but lost to them in the playoffs.
That is a very helpful synopsis of what happened on the field. Does it simplify things? Yes, that is both the strength and the weakness of any descriptive statistic. The curious thing is that the same people who are perfectly comfortable discussing statistics in the context of sports or the weather or grades will seize up with anxiety when a researcher starts to explain something like the Gini index, which is a standard tool in economics for measuring income inequality.
As such, it has the strengths of most descriptive statistics, namely that it provides an easy way to compare the income distribution in two countries, or in a single country at different points in time. The statistic can be calculated for wealth or for annual income, and it can be calculated at the individual level or at the household level. All of these statistics will be highly correlated but not identical.
A country in which every household had identical wealth would have a Gini index of zero. As you can probably surmise, the closer a country is to one, the more unequal its distribution of wealth. The United States has a Gini index of. Once that number is put into context, it can tell us a lot.
For example, Sweden has a Gini index of. We can also compare different points in time. The Gini index for the United States was. The most recent CIA data are for This tells us in an objective way that while the United States grew richer over that period of time, the distribution of wealth grew more unequal.
Again, we can compare the changes in the Gini index across countries over roughly the same time period. Inequality in Canada was basically unchanged over the same stretch. Sweden has had significant economic growth over the past two decades, but the Gini index in Sweden actually fell from.
Is the Gini index the perfect measure of inequality? Absolutely not—just as the passer rating is not a perfect measure of quarterback performance. But it certainly gives us some valuable information on a socially significant phenomenon in a convenient format. We have also slowly backed our way into answering the question posed in the chapter title: What is the point?
The point is that statistics helps us process data, which is really just a fancy name for information. Sometimes the data are trivial in the grand scheme of things, as with sports statistics. Sometimes they offer insight into the nature of human existence, as with the Gini index. How does Netflix know what kind of movies you like? How can we figure out what substances or behaviors cause cancer, given that we cannot conduct cancer-causing experiments on humans?
Does praying for surgical patients improve their outcomes? Is there really an economic benefit to getting a degree from a highly selective college or university? What is causing the rising incidence of autism? Statistics can help answer these questions or, we hope, can soon. The world is producing more and more data, ever faster and faster.
Here is a quick tour of how statistics can bring meaning to raw data. Description and Comparison A bowling score is a descriptive statistic. So is a batting average. Most American sports fans over the age of five are already conversant in the field of descriptive statistics. We use numbers, in sports and everywhere else in life, to summarize information. How good a baseball player was Mickey Mantle? He was a career. To a baseball fan, that is a meaningful statement, which is remarkable when you think about it, because it encapsulates an eighteen-season career.
We evaluate the academic performance of high school and college students by means of a grade point average, or GPA.
A letter grade is assigned a point value; typically an A is worth 4 points, a B is worth 3, a C is worth 2, and so on. By graduation, when high school students are applying to college and college students are looking for jobs, the grade point average is a handy tool for assessing their academic potential. Someone who has a 3. That makes it a nice descriptive statistic.
The GPA does not reflect the difficulty of the courses that different students may have taken. How can we compare a student with a 3. This caused its own problems. Instead, they paid to send me to a private driving school, at nights over the summer. Was that insane? But one theme of this book will be that an overreliance on any descriptive statistic can lead to misleading conclusions, or cause undesirable behavior.
Descriptive statistics exist to simplify, which always implies some loss of nuance or detail. Anyone working with numbers needs to recognize as much. Inference How many homeless people live on the streets of Chicago? How often do married people have sex? These may seem like wildly different kinds of questions; in fact, they both can be answered not perfectly by the use of basic statistical tools.
One key function of statistics is to use the data we have to make informed conjectures about larger questions for which we do not have full information. It is expensive and logistically difficult to count the homeless population in a large metropolitan area. Yet it is important to have a numerical estimate of this population for purposes of providing social services, earning eligibility for state and federal revenues, and gaining congressional representation.
One important statistical practice is sampling, which is the process of gathering data for a small area, say, a handful of census tracts, and then using those data to make an informed judgment, or inference, about the homeless population for the city as a whole. Sampling requires far less resources than trying to count an entire population; done properly, it can be every bit as accurate. A political poll is one form of sampling.
A research organization will attempt to contact a sample of households that are broadly representative of the larger population and ask them their views about a particular issue or candidate. This is obviously much cheaper and faster than trying to contact every household in an entire state or country.
The polling and research firm Gallup reckons that a methodologically sound poll of 1, households will produce roughly the same results as a poll that attempted to contact every household in America. In the mids, the National Opinion Research Center at the University of Chicago carried out a remarkably ambitious study of American sexual behavior.
The results were based on detailed surveys conducted in person with a large, representative sample of American adults. If you read on, Chapter 10 will tell you what they learned. How many other statistics books can promise you that? That does not mean that they are making money at any given moment. When the bells and whistles go off, some high roller has just won thousands of dollars.
The whole gambling industry is built on games of chance, meaning that the outcome of any particular roll of the dice or turn of the card is uncertain. At the same time, the underlying probabilities for the relevant events—drawing 21 at blackjack or spinning red in roulette—are known. This turns out to be a powerful phenomenon in areas of life far beyond casinos. Many businesses must assess the risks associated with assorted adverse outcomes.
However, any business facing uncertainty can manage these risks by engineering processes so that the probability of an adverse outcome, anything from an environmental catastrophe to a defective product, becomes acceptably low. Wall Street firms will often evaluate the risks posed to their portfolios under different scenarios, with each of those scenarios weighted based on its probability.
The financial crisis of was precipitated in part by a series of market events that had been deemed extremely unlikely, as if every player in a casino drew blackjack all night. I will argue later in the book that these Wall Street models were flawed and that the data they used to assess the underlying risks were too limited, but the point here is that any model to deal with risk must have probability as its foundation. The entire insurance industry is built upon charging customers to protect them against some adverse outcome, such as a car crash or a house fire.
The insurance industry does not make money by eliminating these events; cars crash and houses burn every day. Sometimes cars even crash into houses, causing them to burn.
Instead, the insurance industry makes money by charging premiums that are more than sufficient to pay for the expected payouts from car crashes and house fires. The insurance company may also try to lower its expected payouts by encouraging safe driving, fences around swimming pools, installation of smoke detectors in every bedroom, and so on. Probability can even be used to catch cheats in some situations. The mathematical logic stems from the fact that we cannot learn much when a large group of students all answer a question correctly.
But when those same test takers get an answer wrong, they should not all consistently have the same wrong answer. If they do, it suggests that they are copying from one another or sharing answers via text. Of course, you can see the limitations of using probability. A large group of test takers might have the same wrong answers by coincidence; in fact, the more schools we evaluate, the more likely it is that we will observe such patterns just as a matter of chance.
A statistical anomaly does not prove wrongdoing. We cannot arrest Mr. Kinney for fraud on the basis of that calculation alone though we might inquire whether he has any relatives who work for the state lottery. Probability is one weapon in an arsenal that requires good judgment. We have an answer for that question—but the process of answering it was not nearly as straightforward as one might think. The scientific method dictates that if we are testing a scientific hypothesis, we should conduct a controlled experiment in which the variable of interest e.
If we observe a marked difference in some outcome between the two groups e. We cannot do that kind of experiment on humans.
If our working hypothesis is that smoking causes cancer, it would be unethical to assign recent college graduates to two groups, smokers and nonsmokers, and then see who has cancer at the twentieth reunion.
Smokers and nonsmokers are likely to be different in ways other than their smoking behavior. For example, smokers may be more likely to have other habits, such as drinking heavily or eating badly, that cause adverse health outcomes. If the smokers are particularly unhealthy at the twentieth reunion, we would not know whether to attribute this outcome to smoking or to other unhealthy things that many smokers happen to do. We would also have a serious problem with the data on which we are basing our analysis.
Smokers who have become seriously ill with cancer are less likely to attend the twentieth reunion. As a result, any analysis of the health of the attendees at the twentieth reunion related to smoking or anything else will be seriously flawed by the fact that the healthiest members of the class are the most likely to show up. The further the class gets from graduation, say, a fortieth or a fiftieth reunion, the more serious this bias will be.
We cannot treat humans like laboratory rats. As a result, statistics is a lot like good detective work. The data yield clues and patterns that can ultimately lead to meaningful conclusions. You have probably watched one of those impressive police procedural shows like CSI: New York in which very attractive detectives and forensic experts pore over minute clues—DNA from a cigarette butt, teeth marks on an apple, a single fiber from a car floor mat—and then use the evidence to catch a violent criminal.
The appeal of the show is that these experts do not have the conventional evidence used to find the bad guy, such as an eyewitness or a surveillance videotape.
So they turn to scientific inference instead. Statistics does basically the same thing. The data present unorganized clues—the crime scene. Statistical analysis is the detective work that crafts the raw data into some meaningful conclusion. After Chapter 11, you will appreciate the television show I hope to pitch: CSI: Regression Analysis, which would be only a small departure from those other action-packed police procedurals. When you read in the newspaper that eating a bran muffin every day will reduce your chances of getting colon cancer, you need not fear that some unfortunate group of human experimental subjects has been force-fed bran muffins in the basement of a federal laboratory somewhere while the control group in the next building gets bacon and eggs.
Instead, researchers will gather detailed information on thousands of people, including how frequently they eat bran muffins, and then use regression analysis to do two crucial things: 1 quantify the association observed between eating bran muffins and contracting colon cancer e.
Of course, CSI: Regression Analysis will star actors and actresses who are much better looking than the academics who typically pore over such data. What individuals are most likely to become terrorists?
Olympic beach volleyball team. When she gets the printout from her statistical analysis, she sees exactly what she has been looking for: a large and statistically significant relationship in her data set between some variable that she had hypothesized might be important and the onset of autism. She must share this breakthrough immediately!
The researcher takes the printout and runs down the hall, slowed somewhat by the fact that she is wearing high heels and a relatively small, tight black skirt. She finds her male partner, who is inexplicably fit and tan for a guy who works fourteen hours a day in a basement computer lab, and shows him the results.
Together the regression analysis experts walk briskly to see their boss, a grizzled veteran who has overcome failed relationships and a drinking problem. Just about every social challenge that we care about has been informed by the systematic analysis of large data sets. In many cases, gathering the relevant data, which is expensive and time-consuming, plays a crucial role in this process as will be explained in Chapter 7. I may have embellished my characters in CSI: Regression Analysis but not the kind of significant questions they could examine.
There is an academic literature on terrorists and suicide bombers—a subject that would be difficult to study by means of human subjects or lab rats for that matter. One such book, What Makes a Terrorist , was written by one of my graduate school statistics professors. The book draws its conclusions from data gathered on terrorist attacks around the world.
A sample finding: Terrorists are not desperately poor, or poorly educated. Well, that exposes one of the limitations of regression analysis. We can isolate a strong association between two variables by using statistical analysis, but we cannot necessarily explain why that relationship exists, and in some cases, we cannot know for certain that the relationship is causal, meaning that a change in one variable is really causing a change in the other.
In the case of terrorism, Professor Krueger hypothesizes that since terrorists are motivated by political goals, those who are most educated and affluent have the strongest incentive to change society. These individuals may also be particularly rankled by suppression of freedom, another factor associated with terrorism. This discussion leads me back to the question posed by the chapter title: What is the point? The point is not to do math, or to dazzle friends and colleagues with advanced statistical techniques.
The point is to learn things that inform our lives. As a result, there are numerous reasons that intellectually honest individuals may disagree about statistical results or their implications. At the most basic level, we may disagree on the question that is being answered.
As the next chapter will point out, more socially significant questions fall prey to the same basic challenge. What is happening to the economic health of the American middle class? Nor can we create two identical nations —except that one is highly repressive and the other is not—and then compare the number of suicide bombers that emerge in each.
Even when we can conduct large, controlled experiments on human beings, they are neither easy nor cheap. Researchers did a large-scale study on whether or not prayer reduces postsurgical complications, which was one of the questions raised earlier in this chapter. We conduct statistical analysis using the best data and methodologies and resources available. Statistical analysis is more like good detective work hence the commercial potential of CSI: Regression Analysis.
Smart and honest people will often disagree about what the data are trying to tell us. But who says that everyone using statistics is smart or honest? As mentioned, this book began as an homage to How to Lie with Statistics, which was first published in and has sold over a million copies.
The reality is that you can lie with statistics. Or you can make inadvertent errors. In either case, the mathematical precision attached to statistical analysis can dress up some serious nonsense.
This book will walk through many of the most common statistical errors and misrepresentations so that you can recognize them, not put them to use. So, to return to the title chapter, what is the point of learning statistics? To summarize huge quantities of data. To make better decisions. To answer important social questions. To recognize patterns that can refine how we do everything from selling diapers to catching criminals. To catch cheaters and prosecute criminals.
To evaluate the effectiveness of policies, programs, drugs, medical procedures, and other innovations. And to spot the scoundrels who use these very same powerful tools for nefarious ends. If you can do all of that while looking great in a Hugo Boss suit or a short black skirt, then you might also be the next star of CSI: Regression Analysis. In that case, the United States would have a Gini Index of The first question is profoundly important.
It tends to be at the core of presidential campaigns and other social movements. The second question is trivial in the literal sense of the word , but baseball enthusiasts can argue about it endlessly. What the two questions have in common is that they can be used to illustrate the strengths and limitations of descriptive statistics, which are the numbers and calculations we use to summarize raw data.
That would be raw data, and it would take a while to digest, given that Jeter has played seventeen seasons with the New York Yankees and taken 9, at bats. Or I can just tell you that at the end of the season Derek Jeter had a career batting average of.
It is easy to understand, elegant in its simplicity—and limited in what it can tell us. Baseball experts have a bevy of descriptive statistics that they consider to be more valuable than the batting average. I called Steve Moyer, president of Baseball Info Solutions a firm that provides a lot of the raw data for the Moneyball types , to ask him, 1 What are the most important statistics for evaluating baseball talent?
Ideally we would like to find the economic equivalent of a batting average, or something even better. We would like a simple but accurate measure of how the economic well-being of the typical American worker has been changing in recent years. Are the people we define as middle class getting richer, poorer, or just running in place?
Per capita income is a simple average: total income divided by the size of the population. Congratulations to us. There is just one problem. My quick calculation is technically correct and yet totally wrong in terms of the question I set out to answer. Yah right! Additionally many of these websites are very salesy and they sell crap and dupe people.
My information is to help people not screw them into buying some inflated worthless trinkets on the Internet. I am fed up with this BS. And anyone who is legitimate on the Internet selling quality and informational eBooks ought to be upset as well.
If you are assisting folks who sell eBooks in anyway you need to be sure that the people you associate with, do not associate with other people who are scam artists. I hate scammers, they piss me off and sell crap on the Internet and trick people making it hard for real decent people on the Internet. You should be as pissed as I am, even though I am not selling anything on the Internet.
I am sick and tired of worthless, spineless, weakness, scammers, promoters of trash on the Internet. Everywhere I look, more scam artists, selling crap, worthless junk. Is it a crime to help people? Unfortunately for every one of the good folks out there with real world and worthy knowledge selling their eBooks or giving them away, there are 10 scammers selling hype and crap to anyone stupid enough to pay them. It seems everyone has some stupid profit motive in everything you do?
What is deal here? Is that the way all everyone online thinks? Why has everyone got me categorized for some worthless promoter of crap like them, simply because I write articles and submit them online?
Many eBook promoters have attempted to recruit my writing talents, assuming that I would lower my standards of ethics and join them in some sort of online hype to scam people. I am not like them, how dare they assume that. I want these people to stay away from me. I do not want them around or associated with my noble efforts to help people, make them think or give them free information.
For example, the government has made it difficult for Christian and Islamic groups to gain permission to repair or build new places of worship. For many years this mining area has been protected by the Kachin Independence Army. However, in the Myanmar government dropped leaflets from helicopters informing the population in northern Kachin that civilians and Kachin militants who remain in the region will be considered hostile opposition to the government military forces.
The government then forced more than inhabitants from their homes and villages, as well as from the amber mines. This hostile takeover of the profitable Kachin amber mines ensures that amber purchases from researchers and private collectors will help fund the government side of the Myanmar ethnic civil war.
While some researchers and universities feel as though they should refrain from making such amber purchases, their failure to participate enables many private collectors to remove collections from the public or to charge researchers an exorbitant fee for access.
Furthermore, many of the miners in the Kachin region, on both sides of the conflict, are not fully aware of the value of the amber that they are selling and are therefore being exploited by the wholesalers who purchase from them. Myanmar classifies amber as a gemstone, not a fossil, so it can be legally removed from the country, unlike fossils that have restrictions on removal.
If you were a university scientist, how would you decide whether it is ethical for you to buy amber from Myanmar? If you took a deontological approach, what would your reasoning look like? What moral principles would you take into account? If you took a utilitarian approach to answering this question, what would your reasoning look like? What facts would you weigh in making the decision? In deciding whether it is ethical for you to buy amber from Myanmar, do you need to guard against the self-serving bias unduly affecting your decision?
If so, how would you go about guarding against it have a deleterious impact? Are there other cognitive biases and heuristics that might adversely affect your moral reasoning if you are not careful? Which ones? Is this ever a consideration that buyers should take into account? Is it always a consideration that buyers should take into account? Xing, Linda et al. American football is a rough and dangerous game. Football rules intentionally create high speed collisions between human beings, making such injuries inevitable and the sport controversial.
And new knowledge about brain injuries have caused many people to call football immoral [2] and to advocate its abolition.
0コメント