Imbens and Rubin’s book “Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction” is an excellent reference for the Potential Outcomes approach. There are also several summaries online done by Rubin which do a great job of explaining the core concepts and how they apply in concrete examples. Rubin’s class in grad school was a large factor in my decision to focus my PhD on Causal Inference, and that book is one I return to frequently.
Plug for my friend’s “start-up”/company that I’ve used to find online piano teachers to great success, but they also do vocals as well as many other instruments: ToneRow.com.
She is a piano prof at Juilliard and I believe runs one of their startup/business courses, so she’s put a lot of careful thought into this (but I’m sure would appreciate feedback).
Disclosure: I am completely unaffiliated with her company, but have been using it during this period of SIP. If you have any questions or feedback about TR, feel free to reach out.
For foreign students, the PhD program is a great way to enter the U.S. job market. In fact, I'd go as far as to say that, while American undergrads can get lucrative job offers right out of college, foreign students are left with doing a Masters/PhD first, hence partly explaining a skew towards foreign students in many Computer Science departments around the country. For those students, I'd say the PhD program is "worth it".
Disclosure: foreign PhD candidate at an American university here.
Great article! One nitpick however in the informal definition of a p-value: the p-value is the probability of getting results similar or more extreme to the results we observed if the dice are not biased.
I'm currently in the process of getting my phd on this exact topic and I'm always happy to hear the problem of network interference in A/B tests being tackled at more and more tech companies. Here's a small (incomplete) list of (mainly industrial) references to solutions to this problem.
- Facebook data scientists have an extensive list of publications on the topic: [1][2]. (Dean Eckles is now at MIT).
- LinkedIn's experimentation team has also published papers on the topic: [3][4]
- Data scientists at Google have also worked on this topic. One external publication/collaboration I'm aware of is: [5]
Ah, but a person with preëxisting good luck might be preferentially randomly selected into the nose-touching group, amplifying the effect and making it hard to measure its size.
Interesting. It was my impression that Donald Rubin was the main instigator of propensity-score methods. Do you have some paper references in mind to give me a better sense of the timeline?
Quite possible -- Robins was pointed out to me as one of the pioneers by Sander Greenland, around 2008 or so? I'm more of an experimental/clinical statistician at this point. The work I did with propensity scoring and doubly robust estimation convinced me that shrinkage was a less-bad approach for the things I needed to do. YMMV.
