I re-read a short paper of Andrew Gelman’s yesterday about multilevel modeling, and thought “That would make a nice example for PyMC”.  The paper is “Multilevel (hierarchical) modeling: what it can and cannot do, and R code for it is on his website.

To make things even easier for a casual blogger like myself, the example from the paper is extended in the “ARM book”, and Whit Armstrong has already implemented several variants from this book in PyMC.

There is really nothing left for me to do but collect up the links and find a nice graphic. I guess I’ll go the extra distance and condense the code to as few lines as possible and paste it in here, too. Here’s my github, if you want to get my reformed csv files and follow along at home.

household_data = [d for d in csv.DictReader(open('household.csv'))]
county_data = [d for d in csv.DictReader(open('county.csv'))]

# hyper-priors
g = Uniform('gamma', [0,0], [100,100])

s_a = Uniform('sigma_a', 0, 100)

# priors
a = {}
for d in county_data:
    @stochastic(name='a_%s'%d['county'])
    def a_j(value=0., g=g, u_j=float(d['u']), s_a=s_a):
        return normal_like(value, g[0] + g[1]*u_j, s_a**-2.)
    a[d['county']] = a_j

b = Uniform('beta', 0, 100)

s_y = Uniform('sigma_y', 0, 100)

# likelihood
y = {}
for d in household_data:
    @stochastic(observed=True, name='y_%s'%d['household'])
    def y_i(value=float(d['y']), a_j=a[d['county']], b=b,
            x_ij=float(d['x']), s_y=s_y):
        return normal_like(value, a_j + b*x_ij, s_y**-2.)
    y[d['household']] = y_i

mc = MCMC([g, s_a, a, b, s_y, y])
mc.sample(10000, 2500, verbose=1)

I might have mentioned that I got to do some world traveling for my work recently. Seeing rural Tanzania was an experience that I still don’t really have good words to describe. But this is not a post about that. This is a post about a sticky idea I got stuck on in some science fiction I was reading during my multi-day to and fro travel.

On my around-the-world-in-4.5-days journey, I read the Jewish feminist sci-fi novel He, She, and It by Marge Piercy. It’s got a classic hard AI theme, about a robot that is so, so human… I’d recommend it. But dilemmas of whether a robot can make a minyon in the reform tradition of 2059 has not stuck in my mind the way this one line about whales has:

The great whales—we had just about killed off the last of them before we began to translate their epic and lyric poetry.

Okay, I’m a little embarrassed by it when I re-read it, but seriously, could we do it? That is, does a serious attempt to translate whale songs into english have a chance in this modern age? I once had a dusty book about an effort by Carl Sagan and his buddies to learn to communicate with dolphins in the 1960s, but technology has seriously advanced since then.

The last talk I saw on statistical machine translation was an effort to do arabic-to-english translation without telling the computer anything about the structure of sentences in either language.

Whale-to-english translation is at least one step harder, since there is a whale-speech-to-whale-text component that needs to precede the machine translation part (and, I suppose, there is the possibility that whale songs cannot be translated into english).

A few questions: Has it already been done/proven impossible? Do you think we could do it? Do you have a vast collection of whale songs available to aid in the quest?

Regarding question 3, after a little searching, I’ve found the lab at Cornell that probably has the necessary data set. They seem more interested in counting and tracking whales than translating them, but I’ve seen many a health researcher be protective of this sort of precious data. I wonder if the Bioacoustics Research Program could share a few thousand hours of recordings.

Too bad for me, my first global health paper will have to be revised and resubmitted. In addition to some more substantive objections, the negative reviewer said “It is unclear what software was used to carry out the Bayesian estimation by MCMC. This is not possible in STATA and would be extremely difficult in the scripting language, Python.” It was difficult in Python! I doubt that any software would make it much easier, though.

To cheer myself up, I’ve been looking into the newest fads in pets: robotic hamsters and teacup pigs.

Would you like to work with me applying computational algorithms to challenges in global health metrics? Then apply for the IHME post-graduate fellowship. Deadline is Feb 15.

(There is also a “pre-graduate” version, for those who have not started graduate school yet.)

I’m updating my CV, and that reminded me that I meant to promote this cool clustering technique that I was a little bit involved in, Clustering With Shallow Trees.

This goes way back to about half-way through my post-doc at MSR, when statistical physicist Riccardo Zecchina was visiting for a semester, and was teaching me about all of the “intractable” optimization problems that he can solve using his panoply of propagation algorithms. In particular, he was working on algorithms for certain types of steiner tree optimization, and he had discovered that adding an extra constraint on the depth of the tree didn’t make the problem harder. (All variants of the problem he considers are NP-hard, but some are NP-harder than others.)

On the bus to work the next day, this depth constraint clicked with some complaints I had heard recently about the failures of single-linkage clustering in practice, that the algorithm produces long, stringy clusters, which are very sensitive to noise. Could having  a knob to tune the depth of the spanning tree could be a way to address this? Riccardo worked hard on it for a long time, and brought a bunch of collaborators into the mix, and eventually they figured out how to make it work really well. They also proved that this approach interpolates between single-linkage (when the depth is unbounded) and the popular new affinity propagation technique (when the depth bound is 2.

It turns out that something between SL and AP is the thing to do in many instances. Here is the home-run example from the paper, clustering people based on their SNPs:

Compare with the results of single linkage and affinity prop:

(What use is clustering people based on their genetic information? It’s important and scary to think about that…)

I got them to try applying it to a public health dataset as well, and the results are promising, but it needs more careful attention to be useful.

That reminds me: Riccardo and Team Survey Propagation, is the code for this available? We need to let other researchers try it on their own data if we want to give the technique a chance to take off.

I’m preparing for my first global travel for global health, but the net is paying attention to a paper that I think I’ll like, and I want to mention it briefly before I fly.

Computational Complexity and Information Asymmetry in Financial Products is 27 pages of serious TCS, but it is so obviously applicable that people outside of our particular ivory tower, and even outside of academia entirely are blogging and twittering about it, and even reading it!

Freedom to Tinker has a nice summary of this paper, if you want to know what it’s about in a hurry.

Mike Trick makes the salient observation that NP-hard doesn’t mean computers can’t do it. But the assumption that this paper is based on is not about worst-case complexity; it is, as it should be, based on an assumption about the average-case complexity of a particular optimization problem over a particular distribution.

As it turns out, this is an average-case combinatorial optimization problem that I know and love, the densest subgraph problem. My plan is to repeat the problem here, and share some Python code for generating instances of it. Then, you, me, and everyone, can have a handy instance to try optimizing. I think that this problem is pretty hard, on average, but there is a lot more chance of making progress on an algorithm for it than for cracking the P versus NP nut.

First, the Densest subgraph problem (bottom of p. 5):

Fix M; N; D; m; n; and d to be some parameters. The (average case, decision) densest subgraph problem with these parameters is to distinguish between the following two distributions P and D on (M;N;D) graphs, where R is obtained by choosing for every top vertex D random neighbors on the bottom; and P is obtained by first choosing random hidden subsets S from [N] and T from [M] with |S| = n and |T| = m, and then choosing D random neighbors for every vertex outside of T, and D-d random neighbors for every vertex in T. We then choose d random additional neighbors in S for every vertex in T.

Then the Densest subgraph assumption (middle of p. 6) is:

Let (N; M; D; n; m; d) be such that N = o(MD), , then there is no and poly-time algorithm that distinguishes between R and P with advantage .

Or, to say the same thing in Python, with a little help from networkx:

import random
from networkx import Graph

def planted_dense_subgraph(M=1000, N=1000, D=500, m=25, n=25, d=15):
    """ Generate a bipartite graph with a planted dense subgraph
    (distribution P)

    Parameters
    ----------
    M, N, D, m, n, d : int, optional
      M and N are the sizes of the bipartitions and m and n are the
      size of the planted node sets.  D is the degree of the M-vertices
      and d is the number of edges from an m-vertex to n-vertices

    Output
    ------
    G : Graph
      A bipartite graph, with vertices T_1, ..., T_M and B_1, ..., B_M
    T_hidden, B_hidden : lists
      The vertex sets of size m and n that are hidden in the T and B
      vertices
    """

    T = ['T_%d'%i for i in range(M)]
    B = ['B_%d'%i for i in range(N)]

    T_hidden = random.sample(T, m)
    B_hidden = random.sample(B, n)

    G = Graph()
    G.add_nodes_from(T)
    G.add_nodes_from(B)

    for t in T:
        if t in T_hidden:
            G.add_star([t] + random.sample(B, D-d))
            G.add_star([t] + random.sample(B_hidden, d))
        else:
            G.add_star([t] + random.sample(B, D))

    return G, T_hidden, B_hidden

def random_graph(M=1000, N=1000, D=500):
    """ Generate a bipartite graph without a planted dense subgraph
    (distribution R)

    Parameters
    ----------
    M, N, D : int, optional

      M and N are the sizes of the bipartitions and D is the degree of
      the M-vertices

    Output
    ------
    G : Graph
      A bipartite graph, with vertices T_1, ..., T_M and B_1, ..., B_M
    """

    T = ['T_%d'%i for i in range(M)]
    B = ['B_%d'%i for i in range(N)]

    G = Graph()
    G.add_nodes_from(T)
    G.add_nodes_from(B)

    for t in T:
        G.add_star([t] + random.sample(B, D))

    return G

If I give you the graph produced by of one of these functions, you can’t tell me which function I used with any more accuracy than if you flip a coin to decide.

As the authors say, this is an assumption. It could be proven false by a clever algorithm tomorrow.

Wow, where does the day go? I spent all my non-meeting time debugging something. At least I fixed it before 5 PM.

The details of the problem are boring, but the whole ordeal could have been avoided if I had just followed the two rules of optimizing software in my Generic Disease Modeling System. What are they?

First Rule of Program Optimization: Don’t do it
Second Rule of Program Optimization (for experts only!): Don’t do it yet

Maybe next week I’ll get a second to write about the good kind of optimization; my statistical physics friends have posted an article on the arxiv which I am a co-author on, about an application of bounded-depth minimum spanning trees, Clustering with Shallow Trees.

This weekend marks the submission of my first “Global Health” paper. Congratulations to me! And many, many thanks to all the people who have worked with me to make it happen. I’ll go into details sometime in the future, first let me see how things go in the refereeing process.

While I was over-working on that business, I got an interesting Call-for-Papers forwarded from global health/AI researcher Emma Brunskill. The AAAI Spring Symposium on Artificial Intelligence for Development (AI-D) is an effort to build a community of people applying computer science and artificial intelligence in less-developed settings.

TCS people, don’t let the “AI” in their title turn you off. Eric Horvitz says that this is for all of us.

From the CfP:

There has been great interest in information and communication technology for development (ICT-D) over the last several years. The work is diverse and extends from information technologies that provide infrastructure for micropayments to techniques for monitoring and enhancing the cultivation of crops. While efforts in ICT-D have been interdisciplinary, ICT-D has largely overlooked opportunities for harnessing machine learning and reasoning to create new kinds of services, and to serve a role in analyses of data that may provide insights about socioeconomic development for disadvantaged populations. The unprecedented volume of data currently being generated in the developing world on human health, movement, communication, and financial transactions provides new opportunities for applying machine learning methods to development efforts, however. Our aim is to foster the creation of a subfield of ICT-D, which we refer to as artificial intelligence for development (AI-D), to harness these opportunities.

It’s great to see Computer Science trying to address the social issues of our time.

I don’t feel like having that post about how big things are brewing in US health care reform on the top of my blog anymore, so here is a quick replacement: a ranking paper that caught my eye recently on arxiv, where computer scientists is applied to politics: On Ranking Senators By Their Votes, by my fellow CMU alum, Mugizi Rwebangira (@rweba on twitter).

Whoops, I got busy again and didn’t have time to make new pictures of TFR vs HDI for Rif and Tanja, let alone fix the Bayes factor estimation code or implement the nested sampling version (which I think will be the cool way to estimate evidence). But coming soon: How MCMC is tying my new work in Health Metrics to my education in Operations Research. That will be in two weeks, at best.

Until then, here is some light reading to get ready for a big week of US healthcare reform debate: Get Sick, Get Out, a survey conducted by lawyers interested in catastrophic medical payments and their connection to housing forclosures. It’s 40 pages long, but it’s in legal-journal format, where they have like 10 words per page if you skip the footnotes. From the abstract:

Half of all respondents (49%) indicated that their foreclosure was caused in part by a medical problem, including illness or injuries (32%), unmanageable medical bills (23%), lost work due to a medical problem (27%), or caring for sick family members (14%).

I’m excited for the next week of healthcare reform debates. When my most jaded friends are forwarding me Moveon.org videos (and I’m listening to 4 minutes of recent REM), I know something unusual is going on.

Happy labor day weekend!