Statistics for the win.
I won't go into to much explanation since Nate does, but check this out. Benford's Law says that, with remarkably few assumptions, an extremely wide range of types of data all exhibit similar patterns when it comes to their values' first digit.
So whether you're talking about the number of people living in each of America's counties or the distance between Earth and each of the Universe's objects, if you took your entire list and only kept the first digits, the proportion of 1s would be about the same on both lists, and the same for 2s, etc. (This sort of makes sense - there are probably more counties with 10-19 people than 90-99, more with 100-199 than 900-999, more with 100,000-199,999 than 900,000-999,999, etc.)
Long story short, Nate's analysis appears to show that one of Ahmadinejad's opponent's vote totals per county(?) behaved extremely well according to Benford's Law - except there were far too many 7s ...
Showing posts with label 538. Show all posts
Showing posts with label 538. Show all posts
June 18, 2009
June 08, 2009
Nate Silver confounds me again
A pretty fascinating fivethirtyeight post from yesterday: The Palin Paradox: Women More Likely to Elected in Male-Dominated Districts.
His findings are exactly what it sounds like they are. Even when he isolates Democratic-leaning (and thus more likely to elect a woman at all) districts, "the most male-dominated from among these strongly Democratic districts elected women in 10 out of 15 instances. The 15 most female districts elected just 3 women."
So, "all told, after controlling for the district's partisan affiliation, male-dominated districts were more than twice as likely to elect a Congresswoman as were female-dominated districts."
What he doesn't have, of course, is explanation for this seemingly - counter intuitive phenomenon. But he has some interesting thoughts:
"It's possible, and maybe even somewhat likely, that there is some sort of latent variable affecting both the sex ratios and elections to the Congress that I haven't accounted for .... Perhaps in male-dominated areas, women are more likely to violate traditional sex roles including something like running for political office, which has traditionally been a male-dominated occupation -- the Sarah Palin frontierswoman caricature works well here. It would be interesting to know whether there more women actually running for office in male-dominated areas, or rather, whether they are winning more often when they do run. Or perhaps this is a phenomenon that goes beyond politics, and career growth is retarded for the dominant gender when there is an insufficient number of the opposite one. Or perhaps there is even something more Freudian: a lack of female companionship (or vice versa) triggers a yearning for it that is manifested in the way we vote."
His findings are exactly what it sounds like they are. Even when he isolates Democratic-leaning (and thus more likely to elect a woman at all) districts, "the most male-dominated from among these strongly Democratic districts elected women in 10 out of 15 instances. The 15 most female districts elected just 3 women."
So, "all told, after controlling for the district's partisan affiliation, male-dominated districts were more than twice as likely to elect a Congresswoman as were female-dominated districts."
What he doesn't have, of course, is explanation for this seemingly - counter intuitive phenomenon. But he has some interesting thoughts:
"It's possible, and maybe even somewhat likely, that there is some sort of latent variable affecting both the sex ratios and elections to the Congress that I haven't accounted for .... Perhaps in male-dominated areas, women are more likely to violate traditional sex roles including something like running for political office, which has traditionally been a male-dominated occupation -- the Sarah Palin frontierswoman caricature works well here. It would be interesting to know whether there more women actually running for office in male-dominated areas, or rather, whether they are winning more often when they do run. Or perhaps this is a phenomenon that goes beyond politics, and career growth is retarded for the dominant gender when there is an insufficient number of the opposite one. Or perhaps there is even something more Freudian: a lack of female companionship (or vice versa) triggers a yearning for it that is manifested in the way we vote."
May 08, 2009
A fascinating look at some banks.
Update: The link to the WSJ graphic is now included. Sorry about that.
The Wall Street Journal has posted a great "interactive graphic" that compares the 19 stress-tested banks on a number of criteria including new capital needed, tier 1 common capital ratio, and amount of TARP funding received.
A related aside: I love complex graphics along these lines and don't think that the value of such graphics can be overstated when one considers the graphics' ability to convey different depths of information for those of different intelligence and interest levels. Nate Silver does a great job with his graphics, for example "A Starry-Eyed Look at the 2012 Republican Field." The most cursory glance tells you nothing more than which candidates are present and which candidates might be similar to one another simply because there is little distance between them (basically the level of focus shown by someone who just wants to pretend to be smart around others, e.g. "I'd say Huckabee, Romney, Jindal, and Palin are the favorites."). The next level of information gets into actually looking at those axes and understanding what they imply. The third level takes into account the size of each bubble: the larger the bubble the greater the public support for that candidate. Finally, one can, as Nate suggests, consider the implications of the political 'gravity' exerted by each of these 'planets.' That is, one can imagine that when one politician gains support (that is, his planet grows) it must be pulling that support from elsewhere (that is, exerting a political gravity on the supporters of the other planets, pulling them away).
Honestly, this post did begin as a quick post only focused on that banks graphic; it spiraled out of my control once I started talking about the always riveting topic of graphics! I hope it all made sense.
The Wall Street Journal has posted a great "interactive graphic" that compares the 19 stress-tested banks on a number of criteria including new capital needed, tier 1 common capital ratio, and amount of TARP funding received.
A related aside: I love complex graphics along these lines and don't think that the value of such graphics can be overstated when one considers the graphics' ability to convey different depths of information for those of different intelligence and interest levels. Nate Silver does a great job with his graphics, for example "A Starry-Eyed Look at the 2012 Republican Field." The most cursory glance tells you nothing more than which candidates are present and which candidates might be similar to one another simply because there is little distance between them (basically the level of focus shown by someone who just wants to pretend to be smart around others, e.g. "I'd say Huckabee, Romney, Jindal, and Palin are the favorites."). The next level of information gets into actually looking at those axes and understanding what they imply. The third level takes into account the size of each bubble: the larger the bubble the greater the public support for that candidate. Finally, one can, as Nate suggests, consider the implications of the political 'gravity' exerted by each of these 'planets.' That is, one can imagine that when one politician gains support (that is, his planet grows) it must be pulling that support from elsewhere (that is, exerting a political gravity on the supporters of the other planets, pulling them away).
Honestly, this post did begin as a quick post only focused on that banks graphic; it spiraled out of my control once I started talking about the always riveting topic of graphics! I hope it all made sense.
May 01, 2009
In life, there are no coincidences
OK, so that's a really stupid title. But it's pretty interesting that the same week that the Dems secure their filibuster proof majority (almost - damn you, Coleman!), we have the announcement that Justice Souter is going to retire. So, on to the biggest story that isn't a story, as six months from now the Court will be exactly the same, just younger and woman-ier.
Politically, this is bad news for the Democrats. As laughably horrible as the Republicans were politically in last year's presidential campaign, the Democrats have gotten to just sit back and laugh as they top themselves in the incompetence department this year. As trite as the saying is, the Republicans are a party divided.
But if there's one issue that can unite (some) people under any circumstances, it's nominating people to the Supreme Court - namely, whether or not they are baby-killers.
This is likely to cause some unity amongst Republicans and is good for them in the short term (by short term, I mean possibly long enough to give them a boost in the 2010 elections). Unfortunately for them - and I can't believe I'm saying this - reigniting the culture war is not a good long-term strategy for them, at least until the economy warms up. (Morality is an issue best left for times of peace and prosperity - are no Republicans getting this memo?) And even then - they've got to be careful - (wow, I went to 538 to find an archived article, but it turns out there's Breaking News!) people are starting to get behind gay marriage (please, no jokes). And that's going to be hard to leave out of a culture war, but including it will only move their party farther to the fringes.
OK, I've been meaning to do a separate post on this for a while, but I'll never get to it, so here we go. Long story short, Newt Gingrich is talking about the possibility of trying to create a moderately conservative third party that would move in the Republicans' space (from the left). I can't believe I'm saying this, but Newt Gingrich and Meghan McCain are the best two political strategists on the entire right of American politics right now (they have the best ideas, anyway).
And I have to say - it makes me sad. I would really like to see a stronger Republican party, because one-party rule is never good, no matter who the party is. But I'd like to see a sensible Republican party as well. I guess I'll have to keep waiting ...
Politically, this is bad news for the Democrats. As laughably horrible as the Republicans were politically in last year's presidential campaign, the Democrats have gotten to just sit back and laugh as they top themselves in the incompetence department this year. As trite as the saying is, the Republicans are a party divided.
But if there's one issue that can unite (some) people under any circumstances, it's nominating people to the Supreme Court - namely, whether or not they are baby-killers.
This is likely to cause some unity amongst Republicans and is good for them in the short term (by short term, I mean possibly long enough to give them a boost in the 2010 elections). Unfortunately for them - and I can't believe I'm saying this - reigniting the culture war is not a good long-term strategy for them, at least until the economy warms up. (Morality is an issue best left for times of peace and prosperity - are no Republicans getting this memo?) And even then - they've got to be careful - (wow, I went to 538 to find an archived article, but it turns out there's Breaking News!) people are starting to get behind gay marriage (please, no jokes). And that's going to be hard to leave out of a culture war, but including it will only move their party farther to the fringes.
OK, I've been meaning to do a separate post on this for a while, but I'll never get to it, so here we go. Long story short, Newt Gingrich is talking about the possibility of trying to create a moderately conservative third party that would move in the Republicans' space (from the left). I can't believe I'm saying this, but Newt Gingrich and Meghan McCain are the best two political strategists on the entire right of American politics right now (they have the best ideas, anyway).
And I have to say - it makes me sad. I would really like to see a stronger Republican party, because one-party rule is never good, no matter who the party is. But I'd like to see a sensible Republican party as well. I guess I'll have to keep waiting ...
February 26, 2009
Michael Phelps will be psyched.
The Obama administration has so far shown a willingness to, in many ways, seek out and subsequently ignore controversy, whether the controversy is legitimate or contrived. This willingness has taken many forms, ranging from a nearly $1T stimulus bill to a declaration that the deficit will be halved within the next four year, from the appointment of Judd Gregg to the declared but un-planned closure of Guantanamo. In some cases Obama has argued his case pretty well (think Guantanamo) whether or not he was arguing the correct point; in others he has demonstrated a brashness and confidence unexpected by many observers despite being on the wrong side of the argument (think Judd Gregg). It seems as if Obama has chosen to make the most of the impossible situation he faces by attempting to handle not only the immediate crises (e.g. Iraq, Afghanistan, and the credit market), but also many less-pressing issues that other presidents would have left for later (e.g. the elimination of the Global Gag Rule, the institution of limits on congressional pay, and the recommendation of increased fuel efficiency standards). Obama has thus used the cover of “the worst recession since the Great Depression” to slide through some policies that would have brought down hellfire on the White House during past administrations. (This argument is related to Chris’ point that the stimulus was passed in a manner similar to the Patriot Act.)
It seems as though Obama and Attorney General Eric Holder have decided to open another front in the Obama administration’s backdoor war on outdated, inappropriate, and absurdly expensive policies: Holder declared that federal agents will no longer raid marijuana clubs in states that have passed legislation legalizing medical use of marijuana. [On an embarrassing note: without “The first black […]” leading his name I almost feel as if Holder’s title is missing something.]
Chalk one up for states’ rights. While this is far from legalization of marijuana federally it represents a significant deviation from past policies, and I can only hope that this indicates a shift in focus in the War on Drugs. As the above chart from 538 illustrates, public opinion is slowly moving in favor of the legalization of marijuana (both from the support and opposition sides), a trend I interpret (while hindered in no way by data or fact) to mean that the public as a whole is gradually coming to understand the differences between marijuana, which can lead to mental addictions, and more serious drugs that form both mental and physical addictions and lead to very serious health and general welfare problems (read: crime). As Nate Silver points out, though, it seems likely that, “[…] we'll see other some other once-unthinkable things like legalized gay marriage [before we see legalized marijuana].”
I guess my main point is that the War on Drugs needs to be refocused in two major ways. First of all, the government should exert different efforts for the control of different drugs, with the distribution of funding and resources determined by an honest assessment of the impact of each drug on individuals and the country as a whole. Stated another way, marijuana related arrests should not account for nearly half of all drug arrests (see table below) in a country in which 97 million people have ‘admitted’ to having tried pot.
Source.
The other major change I support is less likely to be implemented, but who knows what this crazy Obama administration will do next? That change is a shift from a supply-side focus to a demand-side focus. Let’s try a mental experiment: imagine if the DEA made a bust in which it confiscated 80% of the heroin in the USA overnight. The next day, as junkies head to the corners, what will happen? Will the ‘drought’ lead users to rehabilitate, to realize the dangers of being a drug addict, to avoid crime and seek out Christ? Or will price go through the roof, likely leading to more drug-related crime as users attempt to scrounge up the necessary cash to buy some of the tiny supply? I think the latter is significantly more likely.
It is certainly easier to publicly justify supply-side programs to limit drug use: the publicity of drug busts; the visibility of border patrol agents; the high recidivism rates of addiction treatment programs; etc. Simply because a course of action is easier and less controversial, though, does not mean it is the best choice. I believe that if the United States wants to seriously attempt to combat the use of dangerous drugs it needs to rethink its classification system and the mechanisms by which it combats drug purchases.
There are a number of potential problems here, I admit. First, assuming demand for a drug falls, prices will fall. When prices fall the suppliers have less incentive to be there (less profit per unit sold), but the users have a greater incentive to use (more units of drug per dollar spent). I think these effects would interact differently based on the drug in question and the level of addiction it creates. Second, drug treatment programs are expensive. I would argue that some of this cost would be offset by the lowered costs associated with the reclassification of some drugs (e.g. if marijuana were not a drug for which one could be jailed, the country would no longer have to pay to jail those arrested with marijuana, or may not even have to arrest those people at all). Again, the net effect is ambiguous in this hypothetical world I have imagined. Lastly, and I am sure I am missing arguments for both sides, opponents of these reforms may argue that by enforcing treatment programs rather than jail time the country is choosing to give criminals a second chance at the expense of those who may have never committed a crime. I imagine an opponent thinking, “Why is that criminal allowed to walk near my child? Why do I pay for his wrong decisions with my fear over my own property and safety?” I have no rebuttal for this except for my own belief that even the best of men can fall prey to addiction and that each and every person who does deserves at least a second chance, if not more.
I hope to one day live in a country in which one mistake on the order of taking a drug does not ruin one’s life permanently. If he feels compelled to continue sneaking legislation through based on false pretenses, let’s hope Obama chooses some of the right legislation to sneak through. I apologize for the length of this ramble. I hope it received at least one drug-addled, "Hooray!" from the crowd.
It seems as though Obama and Attorney General Eric Holder have decided to open another front in the Obama administration’s backdoor war on outdated, inappropriate, and absurdly expensive policies: Holder declared that federal agents will no longer raid marijuana clubs in states that have passed legislation legalizing medical use of marijuana. [On an embarrassing note: without “The first black […]” leading his name I almost feel as if Holder’s title is missing something.]
Chalk one up for states’ rights. While this is far from legalization of marijuana federally it represents a significant deviation from past policies, and I can only hope that this indicates a shift in focus in the War on Drugs. As the above chart from 538 illustrates, public opinion is slowly moving in favor of the legalization of marijuana (both from the support and opposition sides), a trend I interpret (while hindered in no way by data or fact) to mean that the public as a whole is gradually coming to understand the differences between marijuana, which can lead to mental addictions, and more serious drugs that form both mental and physical addictions and lead to very serious health and general welfare problems (read: crime). As Nate Silver points out, though, it seems likely that, “[…] we'll see other some other once-unthinkable things like legalized gay marriage [before we see legalized marijuana].”
I guess my main point is that the War on Drugs needs to be refocused in two major ways. First of all, the government should exert different efforts for the control of different drugs, with the distribution of funding and resources determined by an honest assessment of the impact of each drug on individuals and the country as a whole. Stated another way, marijuana related arrests should not account for nearly half of all drug arrests (see table below) in a country in which 97 million people have ‘admitted’ to having tried pot.
Source.
The other major change I support is less likely to be implemented, but who knows what this crazy Obama administration will do next? That change is a shift from a supply-side focus to a demand-side focus. Let’s try a mental experiment: imagine if the DEA made a bust in which it confiscated 80% of the heroin in the USA overnight. The next day, as junkies head to the corners, what will happen? Will the ‘drought’ lead users to rehabilitate, to realize the dangers of being a drug addict, to avoid crime and seek out Christ? Or will price go through the roof, likely leading to more drug-related crime as users attempt to scrounge up the necessary cash to buy some of the tiny supply? I think the latter is significantly more likely.
It is certainly easier to publicly justify supply-side programs to limit drug use: the publicity of drug busts; the visibility of border patrol agents; the high recidivism rates of addiction treatment programs; etc. Simply because a course of action is easier and less controversial, though, does not mean it is the best choice. I believe that if the United States wants to seriously attempt to combat the use of dangerous drugs it needs to rethink its classification system and the mechanisms by which it combats drug purchases.
There are a number of potential problems here, I admit. First, assuming demand for a drug falls, prices will fall. When prices fall the suppliers have less incentive to be there (less profit per unit sold), but the users have a greater incentive to use (more units of drug per dollar spent). I think these effects would interact differently based on the drug in question and the level of addiction it creates. Second, drug treatment programs are expensive. I would argue that some of this cost would be offset by the lowered costs associated with the reclassification of some drugs (e.g. if marijuana were not a drug for which one could be jailed, the country would no longer have to pay to jail those arrested with marijuana, or may not even have to arrest those people at all). Again, the net effect is ambiguous in this hypothetical world I have imagined. Lastly, and I am sure I am missing arguments for both sides, opponents of these reforms may argue that by enforcing treatment programs rather than jail time the country is choosing to give criminals a second chance at the expense of those who may have never committed a crime. I imagine an opponent thinking, “Why is that criminal allowed to walk near my child? Why do I pay for his wrong decisions with my fear over my own property and safety?” I have no rebuttal for this except for my own belief that even the best of men can fall prey to addiction and that each and every person who does deserves at least a second chance, if not more.
I hope to one day live in a country in which one mistake on the order of taking a drug does not ruin one’s life permanently. If he feels compelled to continue sneaking legislation through based on false pretenses, let’s hope Obama chooses some of the right legislation to sneak through. I apologize for the length of this ramble. I hope it received at least one drug-addled, "Hooray!" from the crowd.
January 27, 2009
A few quick links.
Let's get the blogging week started with a few interesting pages to check out:
- Nate Silver's writeup of "So Just Who Did Vote For The Bailout?" - A lot of interesting stuff in this one, including a statistical analysis of factors contributing to voting patterns on the first bailout bill last year and a brief discussion of the implications of writing one's own definitions of terms like Progressive.
- The Daily Beast - "The Best of Blago" - He is not only crazy; he has the ability to take legitimate literary quotations and warp them to suit his purposes. Wonderful spin, Rod.
- Bit.ly - An alternative to the Tiny URL service, Bit.ly has all of the same features, but is also copying (in their entirety), cataloging, and organizing all sites that users create shortened URLs for. It will take a while or the benefits to pan out (beyond the link-shortening, that is), but it should be interesting to see what researchers can pull off with that amount of information.
- Red Bull Snowscrapers - On February 5 a bunch of professional snowboarders will be jumping off a 90' ramp built in East River Park. Carly and I drove past this last week and saw the main ramp being built. The ramp is pretty cool, with the main support being provided by stacked shipping containers (see photo below). I kind of want to figure out a way to see this from afar so I don't have to hang out with a bunch of snowboarders to see one person jump this. Check out this site for more pictures of the actual ramps.
- Update: I also meant to include the fact that Texas dealt a blow to creationism in a recent schoolboard ruling.
December 08, 2008
The Democratic Senate Majority, Prisoner's Dilemma Style
Nate's got a particularly good article on how it's possible for 58 seats to be better than 59.
(In response to the article: of course, the moderate Republicans in question could work out deals with each other ahead of the vote)
(In response to the article: of course, the moderate Republicans in question could work out deals with each other ahead of the vote)
November 25, 2008
Quick Back-Pat Update
November 17, 2008
November 10, 2008
Nate Silver = Doogie Howser (but with math)?
"By kindergarten, he could multiply two-digit numbers in his head. By 11, he was conducting multivariate analysis to figure out if the size of a baseball stadium affects attendance (it doesn’t). By age 13, he was using statistics to manage a fantasy baseball team. When his parents refused to buy him computer games, he taught himself the Basic programming language and created his own."
Daa-yum. Go Nate. Not so much Keith.
Franken's chances
I really enjoyed this post on FiveThirtyEight. I know Chris checks every post on that site, but how about everyone else? Is it pointless or helpful to highlight particularly good posts from there?
Subscribe to:
Posts (Atom)
Posts
