Tech Notes

Contradictions of the MPH

But the more time I spent as a student of public health, the more my worries of impracticality gave way to a funny feeling of being left out. Our professors were trained as statisticians, economists, and sociologists; what was I being trained as? Was public health a discipline? An area of expertise? An employment category? After years of being a quantitative researcher, I still hesitate to call myself a statistician or an econometrician; I suspect those who work in qualitative methods have similar identity crises with respect to anthropology and ethnography. My courses still adhered to conventions of being observers, not practitioners; but my training was intentionally discipline agnostic. As a result I never quite feel at home; too dispassionate to be a practitioner, too invested to be an academic.

Yours truly writing for the Brooklyn Quarterly.

Link: Digging Into Data

Determinants of rural Liberians’ confidence in health care

I have a new paper out analyzing the results of a 2008 survey in Liberia, focusing on respondents’ confidence in their ability to obtain needed care. Though this particular paper has no relation to recent troubling events in Liberia, the issue of health system confidence is surely playing a role in the progression of events today.


Background: Following a protracted civil war, Liberia is rebuilding its health system.  One of the aims of reconstruction is to expand access to health care to a previously underserved rural population.

Objective: This study analyzed the determinants of Liberians’ confidence in their ability to obtain needed care for themselves or their children in case of serious illness.

Methods: A cross-sectional survey of 1,435 adults in Nimba County, Liberia was conducted. Logistic regression models were estimated with reported ability to obtain needed health services for serious illness as the dependent variable, and demographics, health need, health system characteristics, and informal health care as independent variables.

Results: Overall, 50.56% of respondents reported that they could obtain needed services for themselves or their children.  Confidence in the ability to obtain care increased with education (odds ratio (OR) 1.62, 95% confidence interval (CI): 1.19 – 2.21) and poor physical health in the past 30 days (OR 1.38, 95% CI: 1.01 – 1.88), and decreased with poverty (OR 0.66, 95% CI: 0.47 – 0.93), exposure to previous trauma (OR 0.50, 95% CI: 0.36 – 0.71), dissatisfaction with respondents’ last formal health visit (OR 0.70, 95% CI: 0.54 – 0.91), and high utilization of the informal health sector (OR 0.84, 95% CI: 0.73 – 0.96). No correlation was found between health system confidence and being female, being 35 years old or younger, formal health sector use, being within an hour of a clinic and the closest clinic having basic capabilities.

Conclusions: Respondents’ experiences with the health care system had a greater correlation with their confidence in obtaining needed health care than proximity or quality of medical equipment in health clinics.  Despite pro-poor policies guiding health system reconstruction, poor and less educated individuals have less confidence that the health system can meet their health needs.

Click here for the article entry (access required).

Click here for the manuscript.


Increasingly Irrelevant Distinctions

When I started graduate school a few years ago, I was terribly concerned about my notetaking setup (in retrospect, I should have been more concerned about statistics). I was about to get hammered by information that took a variety of forms: lots of equations and diagrams, but also enough discussion-oriented material to require flexibility in how I took notes. My main candidates, which I used to varying degrees, were:

  • Notes taken on my iPad, a combination of handwritten (with a stylus) and typed[1];
  • Typing notes using MultiMarkdown, to allow for easily formatted text, equations, footnotes, and other wacky content;
  • Going whole hog and taking notes in LaTeX in real time (using TeXpad on Mac or iOS);
  • Handwriting my notes with a pen, in a notebook made of paper. Probably by candlelight or gas lantern.

I ended up with a combination of these, depending on the course and whatever devices I had available. This caused quite a bit of consternation in my workflow-addled brain. What if I needed an equation and couldn’t find it? What if I had different silos of material that adhered to different organizational systems? Won’t somebody please think of the metadata?

Looking back on my 2+ years of notes (which I actually refer to pretty often), I’ve realized how little the input medium mattered. No matter which of the tools I used, everything that I recorded is now either (a) in PDF format in Dropbox, or (b) stashed in Evernote.

Solution (a) should be familiar to you if you’ve made it this far in my post, but it’s worth stressing: keep every bit of digital information that’s important to you in PDF format. It’s standard, it’s viewable on any device, it’s not tied to a large corporation that constantly changes the meaning of file extensions, it’s searchable (especially with solid OCR software on Mac and iOS)…just do it.

Solution (b) is new to me, even though I’ve used Evernote off and on for many years. I’ve only recently started working with Evernote in a way that clicks for me, but I now rely heavily on Evernote’s solid handwriting recognition, which they do automatically once a note is on their servers. These improvements are reliable enough that I can choose to write on my iPad with a stylus and export to Evernote, or handwrite in a notebook and capture it with the Evernote app’s document camera. Here’s an example of some notes I took on my iPad (left) and on paper (right):

Evernote handwriting recognition
Evernote handwriting recognition

As you can see, both notes recognize the word “treat” despite some pretty terrible handwriting.

I’m still working out the kinks of what to keep as a PDF in Dropbox and what to ‘scan’ into Evernote (FYI, Evernote saves these scans as image files). The short answer is that I keep notes that I want to reference in Evernote, and notes that I want to keep working on (including journal articles that I want to highlight) in Dropbox. It’s still a work in progress, but I’m trying my very best not to stress about it. Both are highly searchable, organizable repositories, and Mavericks tags have added the tagging functionality of Evernote to Dropbox.

I’m sure I’ll expound on this in a later post, but hopefully somebody will find this useful.

  1. If you’re interested, I used Noteshelf with a Wacom Bamboo stylus and now use GoodNotes with an Adonit Jot Script. But I digress.  ↩

On Rigor 2

This is a followup to my previous post on external validity and rigor, and a further attempt to pretend that this blog is not just about productivity, apps, and hacks.

Jed Friedman has a great piece on the Development Impact blog on a working paper by Hunt Alcott (who I cited in my previous post). Alcott describes a concept called “External Unconfoundedness”. This perfectly articulates what I was trying to get at in my previous post, an attempt to bring statistical notions of unbiasedness to questions of external validity. A lot of the conditions for external unconfoundedness have to do with the environment of the original study, and Alcott is particularly interested in site selection bias – the extent to which the setting for a study is chose because of favorable conditions.

Both the working paper and the post are great reads.

Link: Toward a more systematic approach to external validity: Understanding site-selection bias

Wireless Presenting Just Got a Lot Easier

Much has been said about Apple’s updates to Keynote, Pages, and Numbers last year, and much of that has been pretty negative. However, Apple has been rolling out updates to these apps over the past few months and, as of the latest update, has made a wonderful thing possible: it is now easy to wirelessly present and annotate a Keynote presentation with nothing more than an iPad, iPhone, and adapter.

Doing a wireless presentation on the go has been possible, but even the most well-done setups have required purchasing, bringing, and configuring additional devices (MacSparky’s setup, which I have used until now, requires a $99 Apple TV and a $99 Airport Express). Now, I only need three things which I carry around anyway: my iPad mini, my iPhone, and an adapter to hook an iOS device to a projector.

The Setup

The basic setup is to use your iPad as a remote that you carry around, which controls your iPhone that’s connected to the room projector[1]. Here are my steps for setting up a wireless Keynote presentation:

  1. Connect my iPhone to the projector using the adapter
  2. Pair the Keynote apps so that my iPad serves as a remote for my iPhone
  3. Press Play on my iPad
  4. Don’t be nervous and do a really good presentation

I’ll walk you through steps 1–3; you’re on your own for step 4.

Hardware: Connect your iPhone

This is easy. Both the VGA and HDMI adapters also have a power plug, so you can keep your iPhone charged while you project. As for which adapter to purchase, in my experience overhead projectors use VGA, while flatscreens use HDMI. That said, I work in academia, so you private sector folks with the fancy pants might only need HDMI to function.

Software: Pair the Keynote apps

The pairing process can feel a bit tedious, but only needs to be done the first time (your device is remembered for subsequent presentations). Better still, if you’re pairing iOS-to-iOS you don’t even need to be on the same network. Here are the steps:

  1. Make sure Bluetooth is on for both devices
  2. Open Keynote on both devices
  3. On the iPad, tap the remote button on the main screen and tap Continue
  4. On the iPhone, open a presentation
  5. On the iPhone, tap on the wrench icon on the upper right of the screen and navigate to Presentation Tools > Allow Remote Control and turn Enable Remotes on
  6. After taking a few seconds to pair via Bluetooth, your iPad should appear underneath Enable Remotes; tap Link
  7. A passcode should show up on both devices; confirm that they’re the same number by tapping Confirm on your iPhone
  8. Tap Done on your iPhone

Combined, here’s what it looks like on the iPhone:

and on the iPad:


I said I couldn’t help you make an awesome presentation, but here’s a neat tip: as of this week’s Keynote updates, you can use your iPad to draw on your existing slides, and draw attention to parts of your slide using a laser pointer function. Here’s what it looks like in practice:

This means that you can walk around the room with iPad in hand, drawing and pointing to your heart’s content, all while projecting onto the room’s projector through your iPhone.

The best part? This is all I need for my new mobile presentation setup:

Enjoy, and please get in touch if you have ideas or suggestions.

  1. In this post, I’m using the iPad as the remote and connecting the iPhone to the projector; this is because I prefer to use the iPad for annotations and reading notes. Feel free to switch the two if you prefer to have the iPhone in hand.  ↩


MOOC Money

Caroline M. Hoxby recently published a NBER working paper on the role of MOOCs in the future business models of both “selective” and “non-selective” postsecondary institutions. The bulk of the paper would be interesting only to an economist, but the end is where things get interesting.

Hoxby argues that the current way that MOOCs are run (make your courses available publicly) could align with the business of non-selective institutions, in that students would pay to enroll in a course just like they would at a brick and mortar school. For “Highly Selective Postsecondary Education” (HSPE) Institutions (schools that rely heavily on alumni donations), though, Hoxby presents an alterative:

Viable online education for HSPE must deal with two problems: (i) the selectivity necessary for offering advanced education and (ii) the experiences that build the beliefs and adherence that sustain the venture capital-like financial model.

Consider a system in which HSPE institutions created online versions of their courses that could be traded with other institutions whose students had similarly high aptitude and preparation. The exporting institution could maintain the advanced nature of the course by limiting enrollment to those outside students who were best prepared, by disallowing outside students whose home institutions had previously sent students who underperformed, or by insisting that the outside students receive support (interactions and assessment) from an instructor at their home institution who is trusted by the exporting faculty member. Exporting institutions might offer such courses at a sustainable cost.18 A student’s home HSPE institution would continue to set his degree requirements, grant his degree, and be responsible for all other aspects of his PE experience.

I haven’t thought enough about this model to consider its implications, but this is one of the more novel pieces I’ve come across that speaks directly to how higher education business models can incorporate these new modes of teaching. If you’re aware of others, please do let me know!

Link: The Economics of Online Postsecondary Education: MOOCs, Nonselective Education, and Highly Selective Education

On Rigor

Lant Pritchett wrote a piece for the Building State Capacity blog about the notion of “rigorous evidence.” At the risk of putting words in his mouth, my sense is that his argument boils down to this: promoters of evidence-based policy overplay their hands by focusing exclusively on internal validity[1]. He says as much in his post:

Evidence would be “rigorous” about predicting the future impact of the adoption of a policy only if the conditions under which the policy was to be implemented were exactly the same in every relevant dimension as that under which the “rigorous” evidence was generated. But that can never be so because neither economics—nor any other social science—have theoretically sound and empirically validated invariance laws that specify what “exactly the same” conditions would be.

Pritchett raises an important point; our understanding of internal validity and our methods for assessing it are far more developed than that of external validity[2]. However, I can’t help but feel that Pritchett is overplaying his hand as well. We consider a study to be internally valid if our comparison groups are equivalent in expectation, not if they are exactly the same in every relevant dimension. This may seem like mincing words, but there’s a distinction between equivalence and plausibly arguing the absence of bias. The latter is the standard to which we hold studies when assessing internal validity, and we should do the same for external validity. Still, the point remains that our understanding of external validity is far removed from even this weaker definition.

Link: Rigorous Evidence Isn’t

  1. Wikipedia’s entry on external validity vs. internal validity provides a nice overview of the tension for those unfamiliar with the concepts.  ↩

  2. A recent working paper by Hunt Alcott and Sendhil Mullainathan is an interesting foray into developing metrics for external validity. Unfortunately, these metrics seem to require a whole heap of data that is rarely available for a single intervention.  ↩

EZ does it

While discussing research workflows with colleagues, I’ve been surprised to hear that many get the full text of a journal article by coming across an article, navigating to their school library website, searching for the journal under E-Resources, clicking on the journal link, digging down to the article of interest, and downloading it.

It doesn’t have to be this way (usually).

In most cases, links to journal pages through your library’s website are identical to their ordinary link + a suffix (called an EZproxy) that validates your school credentials to check if you have access. Thus, instead of:

Your library’s link will look something like:

Your browser can make this change through what’s called a ‘bookmarklet,’ a bookmark that, when clicked, does something with the existing URL in your browser. For my case, creating a bookmark with the following script as its content will redirect a journal’s site through my school library:


Alternatively, you can simply drag this EZproxy link into your browser bar.

Here’s a list of different schools’ EZproxy URLs. Edit the code above by replacing with your school’s URL.

In practice, this means that if you click this after trying to access a full PDF:


You get this:


Now, the caveat: this will not work for certain journals[1]. This may be because your school accesses that journal through some larger database, in which case you may have to go back to your library website, like an animal. But this works for me most of the time; hope you find it useful.

  1. Note that this may simply be because your school doesn’t have access to that journal. Ugh, I know.  ↩

Teaching Regression Discontinuity

A very enjoyable post on the regression discontinuity study design over at the must-read Development Impact blog. I was lucky enough to introduce this design to a room full of policy master’s students this semester, and I agree completely with all of Evans’ points. In addition to being a conceptually interesting design, RD is a great way to introduce students to the rationale behind quasi-experimental studies.