Skip to main content


Showing posts with the label big data

Accounting for bias when analyzing public data

We tend to overestimate the reliability of authority figures, and this impacts how we should analyze data for public policy. Public data is an intrinsic appeal to authority The CDC's WONDER database keeps track of causes of death within the United States . When a death certificate is created for a person in the United States, the certificate includes a special code indicating the cause of death. Through a lengthy process, that information makes its way from the funeral home or hospital to a state registry to the National Vital Statistics System and finally to the CDC. CDC tracks that information in WONDER, which can be partially queried by the public. WONDER is used by scientists, researchers and journalists for all sorts of reasons. It was data from WONDER that largely provided the justification for the claim the the United States has been undergoing an epidemic of heroin addiction. And by any measure, the US has a serious problem with heroin and abuse of other opiate drugs. But W

The New York Times on Data Convergence and Privacy Concerns

As more of day to day life and communications takes place on IP based networks, and as the regulation  surrounding that convergence removes barriers to owners of those networks providing information gleaned from those communications to third parties (including the State) without fear of tort or criminal  reprisal, it becomes more and more clear that tomorrow's world is one with much less privacy than that envisioned during, say, the writing of the Federalist Papers. The New York Times just published a discussion on the topic that is worth perusing (even if you are not particularly a fan of the Times).