Call number
Publication
New York : Penguin Press, 2012.
Pages
534
Description
Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair's breadth, and became a national sensation as a blogger. Drawing on his own groundbreaking work, Silver examines the world of prediction.
Media reviews
The first thing to note about The Signal and the Noise is that it is modest – not lacking in confidence or pointlessly self-effacing, but calm and honest about the limits to what the author or anyone else can know about what is going to happen next. Across a wide range of subjects about which
Show More
people make professional predictions – the housing market, the stock market, elections, baseball, the weather, earthquakes, terrorist attacks – Silver argues for a sharper recognition of "the difference between what we know and what we think we know" and recommends a strategy for closing the gap. Show Less
What Silver is doing here is playing the role of public statistician — bringing simple but powerful empirical methods to bear on a controversial policy question, and making the results accessible to anyone with a high-school level of numeracy. The exercise is not so different in spirit from the
Show More
way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War. Except that their authority was based to varying degrees on their establishment credentials, whereas Silver’s derives from his data savvy in the age of the stats nerd. Show Less
A friend who was a pioneer in the computer games business used to marvel at how her company handled its projections of costs and revenue. “We performed exhaustive calculations, analyses and revisions,” she would tell me. “And we somehow always ended with numbers that justified our hiring the
Show More
people and producing the games we had wanted to all along.” Those forecasts rarely proved accurate, but as long as the games were reasonably profitable, she said, you’d keep your job and get to create more unfounded projections for the next endeavor....... Show Less
In the course of this entertaining popularization of a subject that scares many people off, the signal of Silver’s own thesis tends to get a bit lost in the noise of storytelling. The asides and digressions are sometimes delightful, as in a chapter about the author’s brief adventures as a
Show More
professional poker player, and sometimes annoying, as in some half-baked musings on the politics of climate change. But they distract from Silver’s core point: For all that modern technology has enhanced our computational abilities, there are still an awful lot of ways for predictions to go wrong thanks to bad incentives and bad methods. Show Less
Mr. Silver reminds us that we live in an era of "Big Data," with "2.5 quintillion bytes" generated each day. But he strongly disagrees with the view that the sheer volume of data will make predicting easier. "Numbers don't speak for themselves," he notes. In fact, we imbue numbers with meaning,
Show More
depending on our approach. We often find patterns that are simply random noise, and many of our predictions fail: "Unless we become aware of the biases we introduce, the returns to additional information may be minimal—or diminishing." The trick is to extract the correct signal from the noisy data. "The signal is the truth," Mr. Silver writes. "The noise is the distraction." Show Less
Awards
LA Times Book Prize (Finalist — Science & Technology — 2012)
The Economist Best Books (2012.22)
Phi Beta Kappa Award in Science (Winner — 2013)
ALA Outstanding Books for the College Bound (Science and Technology)
Language
Original language
English
Original publication date
2012
Physical description
534 p.; 9.8 inches
ISBN
9781594204111
Similar in this library
User reviews
LibraryThing member bragan
Nate Silver, best known for his remarkably accurate predictions of the results of the 2012 US elections, takes in-depth look at how people attempt to make predictions about everything from stock prices to earthquakes to an opponent's next chess move. He explains in very clear, very carefully
If I have one complaint about this book, it's that there is almost too much of a good thing. What he's saying is great, well-expressed, and very much worth saying, and his examples are good, but there are so many of them, and he discusses them so thoroughly, that by the end my brain was getting a little tired. Then again, maybe that's my fault as much as Silver's, as one thing I did notice is that my level of brain fatigue depended a lot in my own interest levels in whatever area he was currently considering. Attempts to predict natural disasters or terrorist attacks, or to sort the noise of weather from the signal of climate? Fairly exciting stuff to me, and I zipped through it. But economics makes my eyes glaze over and baseball makes me yawn, and there was an awful lot of both.
Still. I definitely recommend it to anyone who is willing to take a bit of a dive into this subject, whether or not you happen t be a baseball fan. Although you'll probably enjoy parts of it much more than I did if you are.
Show More
thought-out ways how people in various fields attempt to assign probabilities to future events, the importance of not underestimating your level of uncertainty, the dangers of overconfidence, and all kinds of other interesting and important aspects of the subject.If I have one complaint about this book, it's that there is almost too much of a good thing. What he's saying is great, well-expressed, and very much worth saying, and his examples are good, but there are so many of them, and he discusses them so thoroughly, that by the end my brain was getting a little tired. Then again, maybe that's my fault as much as Silver's, as one thing I did notice is that my level of brain fatigue depended a lot in my own interest levels in whatever area he was currently considering. Attempts to predict natural disasters or terrorist attacks, or to sort the noise of weather from the signal of climate? Fairly exciting stuff to me, and I zipped through it. But economics makes my eyes glaze over and baseball makes me yawn, and there was an awful lot of both.
Still. I definitely recommend it to anyone who is willing to take a bit of a dive into this subject, whether or not you happen t be a baseball fan. Although you'll probably enjoy parts of it much more than I did if you are.
Show Less
LibraryThing member jcbrunner
Like an artist's well composed album should enrich the listening experience of a hit single track, the longer form of a book should allow prominent bloggers to present their ideas in a more complex and deeper way. Alas, in reality, both albums and books are stuffed with filler material, dumbing
Nate Silver's book includes very good parts. Usually, these are found whenever he talks about his core areas of experience - baseball, poker and polling. When he ventures far from his topics, the errors begin to pile up. Statistics is a powerful tool in helping to solve problems. Unfortunately, not every problem is solved with a proverbial statistical hammer. The 2008 economics crisis was not, as Silver implies, a prediction problem. Goldman Sachs knew very well that it was selling toxic sludge to its customers. Rating agency bosses sent their underlings back to work on their rating models until they matched their clients' expected AAA. This was not a failure of the models used but a lack of professionalism, regulation and ethics. Peddling his statistical tools as the underutilized wonder weapons is cheap. The worst part, however, is his accommodating discussion of the views of climate change deniers. There is simply no there there. Would a late 19th century Nate Silver found pro slavery activists worthy of discussing their viewpoints? It is sad that selling books in the United States requires so much pandering (and self-censorship) in order not to upset the small-minded.
Barack Obama's win was not difficult to predict - given his sad mean lying toxic joke of an opponent. Nate Silver's compilation and cleaning up of polling data was a welcome corrective to the US media misinformation and entertainment complex which insisted on a horse race despite Romney jockeying a dead horse. Silver's public service of raising awareness of the elements and boundaries of polling are both necessary and timely. He unfortunately only lightly touches upon one of my pet peeves: Statistically significant results, which should be mostly used to not talk about not significant data relationships. Obtaining a certain level of statistical significance is a question of how many resources one wants to invest to attain it. The strength of the link and its explanation are much more important aspects for discussion. The highlight of the book is the introduction to Bayesian statistics which updates and improves upon prior estimates. This part should have been extended. Overall, an average read.
Show More
down the message instead of deepening it and often pandering to the weird mindset of the current crop of American Republicans. Nate Silver's book includes very good parts. Usually, these are found whenever he talks about his core areas of experience - baseball, poker and polling. When he ventures far from his topics, the errors begin to pile up. Statistics is a powerful tool in helping to solve problems. Unfortunately, not every problem is solved with a proverbial statistical hammer. The 2008 economics crisis was not, as Silver implies, a prediction problem. Goldman Sachs knew very well that it was selling toxic sludge to its customers. Rating agency bosses sent their underlings back to work on their rating models until they matched their clients' expected AAA. This was not a failure of the models used but a lack of professionalism, regulation and ethics. Peddling his statistical tools as the underutilized wonder weapons is cheap. The worst part, however, is his accommodating discussion of the views of climate change deniers. There is simply no there there. Would a late 19th century Nate Silver found pro slavery activists worthy of discussing their viewpoints? It is sad that selling books in the United States requires so much pandering (and self-censorship) in order not to upset the small-minded.
Barack Obama's win was not difficult to predict - given his sad mean lying toxic joke of an opponent. Nate Silver's compilation and cleaning up of polling data was a welcome corrective to the US media misinformation and entertainment complex which insisted on a horse race despite Romney jockeying a dead horse. Silver's public service of raising awareness of the elements and boundaries of polling are both necessary and timely. He unfortunately only lightly touches upon one of my pet peeves: Statistically significant results, which should be mostly used to not talk about not significant data relationships. Obtaining a certain level of statistical significance is a question of how many resources one wants to invest to attain it. The strength of the link and its explanation are much more important aspects for discussion. The highlight of the book is the introduction to Bayesian statistics which updates and improves upon prior estimates. This part should have been extended. Overall, an average read.
Show Less
LibraryThing member Maya47Bob46
Of course Nate Silver has hit the election squarely once again. This book, while not explaining his formula, does explain his thinking. If you have ever wondered about political pundits, earthquake predictors, sports betters, chess and poker players and how each calculates - or doesn't calculate -
Show More
the odds, read this book. Even if you know nothing about math or statistics, this is well worth reading. Show Less
LibraryThing member haig51
In the movie 'The Graduate', Dustin Hoffman's character is given career advice in the form of one word, 'plastics'. If a remake of the movie were to be done in present times, that word might just be 'statistics'. Nate Silver embodies this worldview and through a hodgepodge of disparate but
Show More
interrelated vignettes he explores how statistics and probability play out in such areas as sports, economics, chess, poker, climate, politics, and terrorism. He is an admitted bayesian, a follower of the probabilistic approach based on the works of the 18th century reverend and mathematician Thomas Bayes, and a proponent of thinking like a fox as opposed to thinking like a hedgehog. While hedgehogs have one defining ideology or model of the world, foxes have multiple competing models, and using Bayesian probability, constantly update their predictions when new data comes in. Bayesian probability is based on subjective predictions, not on objective statistical distributions like those found in the frequentist school founded by Fisher, a 195h century biologist. It remains to be seen whether the bayesian approach is the 'true' way, but it certainly does seem more effective in practice. Silver spends a lot of time delving into the nuances of each of the example fields he explores, with just enough detail to offer insightful commentary without overwhelming the lay reader with advanced mathematics. In an increasingly unpredictable and complex world, Silver gives us a glimpse of the skills necessary to wrangle the messy reality which we find ourselves living in. Show Less
LibraryThing member JeffV
Statistics are fun, it's why a lot of us are enamored with sports. While a player's OBP might be useful for your fantasy league, statistics and probability are far more useful in the science of prediction. This includes such subjects as political polling, poker, weather and earthquake forecasting.
Silver teaches us how to evaluate such predictions -- whether to be skeptical because they are based on specious data or sample size, or when we should trust it because of sound modeling, such as present weather forecasting. This is an interesting look at the current state of prediction in many fields...and the gains in accuracy made in just the past 20-30 years. Still, no matter how good you are at crunching numbers, you are unlikely to consistently beat Wall Street or the local bookie. Those who make a living gambling do so with slim margins and a large bank roll.
Silver has made a living both gambling and working for sports statistics companies. Hearing his take is interesting...and a wake up call for anyone who thinks they have a system to beat Vegas. I always enjoyed seeing Silver on TV (he's been on The Daily Show) and hearing him on radio. I hope he writes additional books on the topic. as a sports fan, I have a natural affinity for statistics and the meaning of.
Show More
Sports statistician Nate Silver covers all of these and more in The Signal and the Noise. While entertaining us with stories about sports book, disease probability, and devastating hurricanes, Silver also teaches us about the science of prediction and how it differs from basic probability and other statistics-based sciences. We learn how sample size can skew results...just recently a sports statistician claimed the Blackhawks record-setting season start in hockey was a 1-in-750 year event. How can he say such a thing when hockey has been with us less than a century? Well, it seems predictions can be made based on available statistics...but we should also not be surprised when models based on limited data don't pan out as predicted.Silver teaches us how to evaluate such predictions -- whether to be skeptical because they are based on specious data or sample size, or when we should trust it because of sound modeling, such as present weather forecasting. This is an interesting look at the current state of prediction in many fields...and the gains in accuracy made in just the past 20-30 years. Still, no matter how good you are at crunching numbers, you are unlikely to consistently beat Wall Street or the local bookie. Those who make a living gambling do so with slim margins and a large bank roll.
Silver has made a living both gambling and working for sports statistics companies. Hearing his take is interesting...and a wake up call for anyone who thinks they have a system to beat Vegas. I always enjoyed seeing Silver on TV (he's been on The Daily Show) and hearing him on radio. I hope he writes additional books on the topic. as a sports fan, I have a natural affinity for statistics and the meaning of.
Show Less
LibraryThing member rivkat
Like many people I know, I was entranced by Silver’s election blog, and I love him for popularizing some important facts about survey data and probability. That said, this book was a bit of a letdown. Silver is a Bayesian, and he does a decent job explaining the basic principles, but there’s
Show More
not nearly enough math. While I liked finding out that Gary Kasparov basically psyched himself out against Big Blue in the chess chapter, and I learned something about why earthquakes are so hard to predict, overall I found the treatments of his various topics (also including poker, climate change, terrorism, and the stock market, among others) too superficial to be really enjoyable. And his statement that, dire warnings notwithstanding, we shouldn’t worry too much that lower Manhattan would be covered by storm surge was amusing, in a bitter sort of way—though I should hurry to point out that the occurrence of an event deemed low-probability by a model doesn’t by itself mean that the model is wrong! Show Less
LibraryThing member ntgntg
I read this in October, 2012. The ideas are very good and resonate with my own thinking -- use sound mathematics to improve predictions and decision making. He advocated Bayes, but doesn't provide any specific examples. I will reread from time to time.
I wondered whether Silver (a liberal working at
I wondered whether Silver (a liberal working at
Show More
the NY times) would fall prey to his biases and shade his predicitons. But no, he predicted O would win easily and that is what happened. Conservatives take note. Show Less
LibraryThing member tgraettinger
I liked the book, but maybe not as much as I'd hoped to. My expectations were pretty high - maybe unrealistically so - given the author's success predicting the 2008 and 2012 elections (and his appearances on "The Daily Show with Jon Stewart"). He is at his best describing his own work with the
Show More
elections, baseball, and poker - though I felt like he was really getting too far down in the weeds with his descriptions of poker. The portions of the book where he ventured into other people's work (in weather, earthquakes, and terrorism, for instance) were weaker. One central message that really resonated with me: estimate/quantify your uncertainty around your predictions. Another: think about the "priors" and how your predictions will/should change in the light of new evidence. All in all, recommended for any practitioner of data mining or predictive analytics. Show Less
LibraryThing member jasonlf
The Signal and the Noise was a really great read. It's one of those books that lets you annoy your friends by tediously repeating facts, many of which they already have picked up from reading the book, reading reviews, or other tedious friends. Like when the Weather Channel says it is a 30% chance
There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.
I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.
And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.
But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
Show More
of rain it rains 30% of the time, but that when it says a 10% chance of rain it really means a 3% chance of rain (they would rather people be pleasantly surprised). Or that earthquakes are unpredictable. Or that minor league baseballs players are difficult to predict and give a good advantage to very good scouts, while in the majors it is different. Or that a particularly great professional sports better will win 56 percent of the time. Or other examples drawn from the areas Nate focuses on: political predictions, elections, the macroeconomy, financial markets, epidemics, earthquakes, terrorism, baseball, chess, and poker.There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.
I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.
And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.
But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
Show Less
LibraryThing member FKarr
Good, but not great. Was either overlong for the simplistic message he was trying to get across, or he just concentrated too much time & too many words on too few examples. Does he ever really explain how to filter the signal from the noise?
LibraryThing member heindl
Silver understands how statistic works, and so he uses this hammer for any nail.
It is not exactly the way but sometimes I had the feeling.
Another strange part for a reader from Germany are the details of the football league and the poker play.
Both not well known here in Germany.
But I learned a
Especially helpful was the chapter about market prediction, earth quakes and climate change. It gave me new insights, how people misunderstand signals and noise. And exactly this chapters, I wish, many more would read.
It is not exactly the way but sometimes I had the feeling.
Another strange part for a reader from Germany are the details of the football league and the poker play.
Both not well known here in Germany.
But I learned a
Show More
lot, how to have a reality check, what has to be done, if new information drops in. The Bayesian seems to be very efficient in prediction problems, even if it doesn't look that clear in the first glance.Especially helpful was the chapter about market prediction, earth quakes and climate change. It gave me new insights, how people misunderstand signals and noise. And exactly this chapters, I wish, many more would read.
Show Less
LibraryThing member timtom
This is probably one of the best nonfiction books I've ever read. Predictions are everywhere in our lives: we choose mates, jobs, our clothes in the morning, a school for our children based on our possibly poorly informed, influenced yet usually overly confident forecasts. And of course, there are
It is a very dense text, though. The theory is very well explained and expertly illustrated, but there remains an awful lot of it. I also had trouble with the arcane baseball terminology that is used repeatedly to illustrate key concepts. Whereas all other examples are nicely introduced to help the reader understand, a prior knowledge of baseball seems to be a prerequisite for this book... I didn't like that, and it cost the book its otherwise well-deserved fifth star!
Show More
those who make a living out of predicting the weather or the evolution of the stock market (and then bet somebody else's money). Nate Silver very efficiently proves that nearly every prediction we make is flawed because we mistake noise for signal, dismiss low probabilities as "impossibilities" and tend to prefer listening to over-confident forecasters rather than cautious ones. It is a vital reality check: this book should be mandatory reading to every professional forecasters, especially economists and policy makers.It is a very dense text, though. The theory is very well explained and expertly illustrated, but there remains an awful lot of it. I also had trouble with the arcane baseball terminology that is used repeatedly to illustrate key concepts. Whereas all other examples are nicely introduced to help the reader understand, a prior knowledge of baseball seems to be a prerequisite for this book... I didn't like that, and it cost the book its otherwise well-deserved fifth star!
Show Less
LibraryThing member AlexEpstein
I'm a huge fan of Nate Silver's blog FiveThirtyEight.com. It's the most interesting and convincing and reliable blog for election poll analysis.
This book is good-ish, but it's no FiveThirtyEight. I feel like the blog entries are sharper and more on point than the book.
That said, I read the whole
This book is good-ish, but it's no FiveThirtyEight. I feel like the blog entries are sharper and more on point than the book.
That said, I read the whole
Show More
book, and I was rarely bored. Show Less
LibraryThing member Opinionated
In which Nate Silver argues for better forecasting and lionises Baynesian method. Its all neatly set out and impossible to argue with - or rather to do so would be like arguing against the law of gravity.
But I would have liked a broader range of examples; for example is dissection of the poor
He spends some time discussing baseball - a field which is, as he accepts, unusually rich in historical data. This is interesting, and whilst not a baseball fan myself, I could see how baseball is unique. A team game yes, but one where individual clashes predominate. Its unlikely the same approach could be taken to cricket. He also spends time on poker, a topic that personally interests me not at all
The best chapters are those on weather forecasting - where forecasting and predictions are getting better and better , and climate science specifically global warming. And here he nails his argument. Noone disputes the greenhouse effect, few serious people dispute the heating effects of increased greenhouse gases. The problem of course is predicting what will happen; noisy data, patchy historical data, complicating factors galore (sunspots, the cooling effects of the Mt Pinatubo erruption, the cooling effects of sulpher, ENSO etc etc) make predictions difficult. And, as Mr Silver points out, what so called "sceptical" climate scientists disagree about are the predictive models - not the core science
But as Mr Silver points out throughout the book, the goal is not a perfect forecast, but better forecasting. As we get new data we can improve our forecasts. After all, better forecasting is at least an improvement (although in my view, when he tries to use Bayes method to predict terrorist activity, a forecast of the likelihood of an event taking place is not of a great deal of use without knowing in what direction to look for it)
Still - its an excellent book. Should be read in schools. And in government
But I would have liked a broader range of examples; for example is dissection of the poor
Show More
predictions and forecasting of the financial markets is will known by now, true though it is. His successes in the field of political prediction are also well known - and as he admits himself, competing methods had set a pretty low bar for him to beat. And as he admitted in a trip to Australia, he wouldn't be able to replicate his success there (at least not on a state by state basis) due to lack of data at the state levelHe spends some time discussing baseball - a field which is, as he accepts, unusually rich in historical data. This is interesting, and whilst not a baseball fan myself, I could see how baseball is unique. A team game yes, but one where individual clashes predominate. Its unlikely the same approach could be taken to cricket. He also spends time on poker, a topic that personally interests me not at all
The best chapters are those on weather forecasting - where forecasting and predictions are getting better and better , and climate science specifically global warming. And here he nails his argument. Noone disputes the greenhouse effect, few serious people dispute the heating effects of increased greenhouse gases. The problem of course is predicting what will happen; noisy data, patchy historical data, complicating factors galore (sunspots, the cooling effects of the Mt Pinatubo erruption, the cooling effects of sulpher, ENSO etc etc) make predictions difficult. And, as Mr Silver points out, what so called "sceptical" climate scientists disagree about are the predictive models - not the core science
But as Mr Silver points out throughout the book, the goal is not a perfect forecast, but better forecasting. As we get new data we can improve our forecasts. After all, better forecasting is at least an improvement (although in my view, when he tries to use Bayes method to predict terrorist activity, a forecast of the likelihood of an event taking place is not of a great deal of use without knowing in what direction to look for it)
Still - its an excellent book. Should be read in schools. And in government
Show Less
LibraryThing member johnnyapollo
I'm currently re-reading this book and am a fan of Silver's FiveThirtyEight blog on the WSJ website. I was introduced to Silver's work as the Republican primaries were tracking towards a candidate. I remember running across a reference to Silver's statistical models while working for a Fantasy
The book does some interesting things from the very beginning, exploring the concept of Big Data and how that affects analysis and continuing to apply those concepts to real world scenarios - including a very good description of how the financial melt-down of 2008 happened and the events that lead to that point in history. It's fairly finance heavy but still understandable - actually I think I got more from those first couple of chapters than all the other reading I did during the collapse and now via historical perspective. Kudos to Silver for doing a good job in peeling that apart and making it approachable.
I especially like the main premise, which is how to look and measure relevant data while being exposed to an over-abundance of cruft, then using that data to make strong predictive models. To understand the concept, think about doing simple key-word searches on Google – the number or return-hits can be astronomical. While the first few results may be relevant, as you page through the results the number of insignificant hits decreases proportional to a score, based on how the search engine interprets the number of times your words are used on the websites or documents returned. This actually only implies relevance – the problem is that with so much data available, the user still has to decide what to use as reference. To make matters worse, the engine relies on the frequency of the words and not of the contextual relevance of the sentence built by those words. And at the end of the even-less-relevant spectrum, many sites are created to intentionally misdirect the searcher with bad information. All of this would qualify as Noise as defined by Silver (the reference is old radio, how to get through all the static to the station you want to hear clearly) – so how to get the right Signal with all that going on?
Going back to the election, with so much noise to penetrate, it’s easy to see how facts can be ignored while other facts are emphasized, to support your own views and/or argument. It also explains how the losing party could have been so convinced of victory, picking and choosing only those statistics that support the party while ignoring all else as “media bias” – Silver has proven that by first applying the judicious use of data, then an algorithm that adjusts daily poling numbers based on past performance, a relatively accurate predictive model is possible to produce. I’ll end the review here so there is something for others to discover in this work.
Show More
Sports aggregator so the name had struck a cord, when mentioned by a friend to me. As the race towards the election started to heat up, I would visit the blog daily (just as I would all of the various media sites, both conservative and liberal). What I liked about FiveThirtyEight was the reliance on math as applied to a predictive model, which enticed me to purchase this book.The book does some interesting things from the very beginning, exploring the concept of Big Data and how that affects analysis and continuing to apply those concepts to real world scenarios - including a very good description of how the financial melt-down of 2008 happened and the events that lead to that point in history. It's fairly finance heavy but still understandable - actually I think I got more from those first couple of chapters than all the other reading I did during the collapse and now via historical perspective. Kudos to Silver for doing a good job in peeling that apart and making it approachable.
I especially like the main premise, which is how to look and measure relevant data while being exposed to an over-abundance of cruft, then using that data to make strong predictive models. To understand the concept, think about doing simple key-word searches on Google – the number or return-hits can be astronomical. While the first few results may be relevant, as you page through the results the number of insignificant hits decreases proportional to a score, based on how the search engine interprets the number of times your words are used on the websites or documents returned. This actually only implies relevance – the problem is that with so much data available, the user still has to decide what to use as reference. To make matters worse, the engine relies on the frequency of the words and not of the contextual relevance of the sentence built by those words. And at the end of the even-less-relevant spectrum, many sites are created to intentionally misdirect the searcher with bad information. All of this would qualify as Noise as defined by Silver (the reference is old radio, how to get through all the static to the station you want to hear clearly) – so how to get the right Signal with all that going on?
Going back to the election, with so much noise to penetrate, it’s easy to see how facts can be ignored while other facts are emphasized, to support your own views and/or argument. It also explains how the losing party could have been so convinced of victory, picking and choosing only those statistics that support the party while ignoring all else as “media bias” – Silver has proven that by first applying the judicious use of data, then an algorithm that adjusts daily poling numbers based on past performance, a relatively accurate predictive model is possible to produce. I’ll end the review here so there is something for others to discover in this work.
Show Less
LibraryThing member MBDudley
A worthwhile read, with lots of examples from his career as a poker player, pollster and sports fan. Could have been subtitled "Why Bayesian Thinking Matters." I bought this for work purposes but enjoyed it on a personal level too.
LibraryThing member albertgoldfain
A remarkably clear and comprehensive explanation and defense of Bayesian reasoning. Should be required reading for anyone who thinks there is a simple "system" to beat the market, stop a terrorist, win the World Series of Poker, or even predict the weather. Includes useful anecdotes from Silver's
Show More
life experiences. I particularly liked the watered-down (but more accurate) version of the efficient market hypothesis. Show Less
LibraryThing member ashergabbay
I bought "The Signal and the Noise" after I saw a recommendation (I think it was in "The Economist"). As everyone knows by now, Nate Silver became famous after correctly predicting the outcome of the US election, down to the last state. In this book, he strives to separate the chaff from the wheat
Show More
when it comes to predictions about the future. In Silver's words, "the difference between what we know and what we think we know". He urges us to think in probabilistic terms, explaining Bayes Theorem of conditional probability and why he thinks it's relevant to almost any prediction: baseball scores, politics, stock market, etc. He calls for more modesty from TV pundits, and that is hardly a call one can argue with. Show Less
LibraryThing member kcshankd
I first encountered Nate Silver at baseballprospectus.com where he developed player statistical projections for upcoming seasons. When he branched into politics with 538.com prior to the 2008 election I was ecstatic. Finally a push against the empty pomposity that makes up most political coverage
This is an excellent, readable approach to understanding probability and prediction. The 'gets' Silver's success at 538 garnered are tremendous. He discusses the housing bubble with Nobel-winnings economists, baseball with Bill James, Deep Blue's IBM creators, known unknowns and unknown unknowns with Dick Cheney, and concludes with a strong chapter on what to do about it.
Fascinating.
Show More
in the U. S.! I wasn't disappointed. This is an excellent, readable approach to understanding probability and prediction. The 'gets' Silver's success at 538 garnered are tremendous. He discusses the housing bubble with Nobel-winnings economists, baseball with Bill James, Deep Blue's IBM creators, known unknowns and unknown unknowns with Dick Cheney, and concludes with a strong chapter on what to do about it.
Fascinating.
Show Less
LibraryThing member ehousewright
If only it were fashionable to state your initial belief and then set about learning things that will help you modify it in a meaningful way instead of only interacting with people and ideas that solidify it. His examples are fascinating, he actually considers other interpretations of data and he
Show More
provides a plan for coming to conclusions that are certainly more considered than they might have been before applying his Bayesian approach. What an interesting and practical book—I read the news differently because of it. Show Less
LibraryThing member Lyndatrue
Excellent work on predictions, and forecasting. I'd have liked the author to spend a lot less time on the intricacies of sports, but otherwise it's well done.
LibraryThing member cyclismotron
It's a bit late to be writing this review, but at least we are still in the same calendar year!
This was an extremely enjoyable book to read, especially for someone who has worked in an environment where forecasts, both quantitative and qualitative, have been important. One of the key lessons for me
"when the facts change, I change my opinion, but only to a degree weighted by the quantity and quality of the new facts".
The only instances when it consistently fell over were ironically when Silver is writing about the things he knows most about: sports and politics. An inordinate amount of time is spent describing the world baseball statistics and performance prediction. While the inclusion of this much detail is partly to exhibit Silver's extensive knowledge of the area, it bulks out what is already a rather dense paperback. The sections on politics are similar.
This was an extremely enjoyable book to read, especially for someone who has worked in an environment where forecasts, both quantitative and qualitative, have been important. One of the key lessons for me
Show More
was how to adopt the application of Bayesian forecasting in everyday life and also how to communicate it to others. Incorporating it in to one of my favourite quotes, Bayesian forecasting can be summarised in the words of Keynes:"when the facts change, I change my opinion, but only to a degree weighted by the quantity and quality of the new facts".
The only instances when it consistently fell over were ironically when Silver is writing about the things he knows most about: sports and politics. An inordinate amount of time is spent describing the world baseball statistics and performance prediction. While the inclusion of this much detail is partly to exhibit Silver's extensive knowledge of the area, it bulks out what is already a rather dense paperback. The sections on politics are similar.
Show Less
LibraryThing member MarkGFaust
Fascinating and well written book on use of statistics and Bayesian inference in a variety of applications by the creator of FiveThirtyEight.com.
LibraryThing member nosajeel
The Signal and the Noise was a really great read. It's one of those books that lets you annoy your friends by tediously repeating facts, many of which they already have picked up from reading the book, reading reviews, or other tedious friends. Like when the Weather Channel says it is a 30% chance
There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.
I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.
And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.
But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
Show More
of rain it rains 30% of the time, but that when it says a 10% chance of rain it really means a 3% chance of rain (they would rather people be pleasantly surprised). Or that earthquakes are unpredictable. Or that minor league baseballs players are difficult to predict and give a good advantage to very good scouts, while in the majors it is different. Or that a particularly great professional sports better will win 56 percent of the time. Or other examples drawn from the areas Nate focuses on: political predictions, elections, the macroeconomy, financial markets, epidemics, earthquakes, terrorism, baseball, chess, and poker.There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.
I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.
And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.
But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
Show Less
LibraryThing member vguy
Among the best non-fiction i've read recently, up there with Shapiro's 'Contested Will'. Like Shapiro, he is at ease ranging over a huge range of stuff and bringing it to light. Sets the issues of data-handling, prediction and uncertainty in historical and scientific contexts.
The core is Bayes'
I was apprehensive of his background in baseball and poker (two fields that I abhor) and confess to skimming those chapters.
The core is Bayes'
Show More
theory which is fairly mathematical for me to grasp fully but the surprise key seems to be: subjective sense of what is probable must be built into the overall projection; total objectivity is an illusion. RA Fisher comes in for a basting for advocating the elimination of all subjectivity; the fact that he was a smoking/cancer denier in the pay of tobacco companies while puffing away provides an extra coffin-nail! The fun part of the book is how Silver explores implications in many fields: - politics, weather and climate change, epidemiology, earthquakes, terrorism, flight safety and more - throwing anecdotal and analytical light as he goes. Weather forecasters do better when they look out of the window, not just at their charts, just as generals do better if they visit the front-line. Earthquakes remain highly impervious to prediction largely because the essential data is hidden below ground. "Big data", is nothing new;it's been expanding ever since the invention of writing. But now as it grows exponentially, there is a danger in expecting more data (noise) to provide better understanding. What we need are better tools for analysis and pattern discernment, that is, for detecting signal. And, Silver tells us, Bayes theorem is one of the most important.I was apprehensive of his background in baseball and poker (two fields that I abhor) and confess to skimming those chapters.
Show Less