All posts by admin

New years entity resolutions

Just in case you think that entity resolution problems (matching up names appearing in multiple data sources, while not falsely assuming that everyone named “John Smith” is the same person) are purely an academic concern, I recently got an email from an airline announcing TSA’s new Secure Flight program and asking me to provide them with birth date and gender information when making a reservation:
Continue reading New years entity resolutions

Concurrency and Reachability movie

Animated gif clip of concurrency movie
View as: QuickTime | YouTube | other formats.

A movie premier! Yup, this week we are releasing a scare-thriller by the name of Concurrency and Reachability: transmission in a dynamic network. Don’t let the title fool you, the topic is a bit sexier than it sounds, as the underlying network model used to simulate disease transmission was derived from data on real-world sex contact networks.
Continue reading Concurrency and Reachability movie

CorpWatch API lauch!

For the past several months, Greg and I have been working on project to scrape corporate subsidiary ownership relations from Securities Exchange Commission filings. The first part of the project launched today! So now you can pull down company names and relationships for more than 200,000 publicly traded U.S. corporations and their subsidiaries from http://api.corpwatch.org. If writing code is not your thing, we also built an interactive browser for the data at http://croctail.corpwatch.org.

croctail_screenshot
Continue reading CorpWatch API lauch!

Oilmoney Redux

The main conference on Social Network Analysis was is in San Diego this year, so I decided to make a trip down. Was nice to step away from the screen and see old and new faces from the far-flung research community. Amusingly, the conference landed in the middle of spring break celebrations, so there were bearded academics wandering geekily around in crowds of drunken sunburnt 20-something revelers.

prezoilmoney movie placeholderI gave a presentation at the very tail end of the conference to demonstrate some features of the oilmoney website—including a presidential contribution movie, and bit of analysis on the data. Much of this will be familiar to anyone who has read these earlier posts, but the stat stuff is new.
Warning: the rest of this post is pretty geeky, read at your own risk ;-)

Continue reading Oilmoney Redux

Digging into MAPLight.org’s Bill Endorsement Data

Diagram of bill supportersDan Newman, director of the money in politics watchdog/transparency site MAPLight.org kindly shared some of their bill endorsement data for me to explore. In addition to providing an elegant interface for accessing California and U.S. Federal campaign contribution data and voting records, MAPLight’s interns do extensive research to determine various organizations positions on bills that are being voted on in Congress. These endorsement and opposition relationships can be thought of as ties linking the organizations to the various bills they take a position on. The ties can then be assembled to form—yup, you guessed it—networks of bills and their supporters. My hope is that giving the bill data a relational treatment might reveal some of the coalitions and give additional context for each organization’s position.
Continue reading Digging into MAPLight.org’s Bill Endorsement Data

San Francisco Political Contributions

Closeup of SF 2008 political contribution networkI’m very interested in trying to figure out ways to map the political landscapes and power structures that are operating around us. I’d like to be able to see various organizations and political actors in the context of their allies, enemies, and supporters in order to understand where the political boundaries are between various factions.
Continue reading San Francisco Political Contributions

Network of ballot measure endorsements

Closeup of CA Proposition Endorsement network

It is election day! Fingers crossed…. ;-) Before today I was searching for various organizations’ endorsements of California ballot measures. Finally located some data, and was curious how it would appear as a network showing the organizations and the propositions they support. Was able to scrape data for University of Berkley’s IGS Library Ballot Measure Endorsement page (for Nov. 2008) and create a few network images.

Continue reading Network of ballot measure endorsements