heart attack surgery

in the previous lecture, we saw that not allstudents that was randomized into the different treatment arms in project star stayed in thegroups to which they were allocated. some students left the different groups forvarious unknown reasons. however as long as reallocation happens randomlyit does not invalidate the randomized controlled trial in terms of estimating the causal effectof the treatment.

heart attack surgery, however, it is very likely that reallocationof individuals after randomization is not random.students allocated to the control group may be unhappy by not receiving the treatmentand might subsequently try to get into treatment group anyway.in the case of small classes, most students

allocated to small classes are probably happyby being assigned to a small class and will not try to reallocate but in many other casessome of those allocated to treatment may subsequently try to defer treatment if possible.for instance training or education for unemployed. here it is often the case that many unemployedallocated to training or education does in fact not want to do this and tries to evadetreatment. all this invalidates randomization and threatensthe possibility to estimate the causal effect of a treatment from a randomized trial.therefore, even if rct’s in theory enables researchers to estimate causal effects, inpractice this may prove difficult. however, surprisingly, even in the case ofnon-random dropout, data from an rct still

enables the estimation of causal effects.the following slides explains how. denote as before t the treatment indicator,taking the value one if the individual is actually treated that is not only allocatedto treatment but also actually treated and zero if not treated.further, define z equal to one if allocated to the treatment group and zero otherwise.z is the randomization indicator. again the observed outcome is outcome eitheras treated or untreated. average outcome for those offered treatmentis the average outcome without treatment for those offered treatment plus the treatmenteffect (the difference between outcome with and without treatment) for those actuallytreated (obtained by multiplying with the

treatment indicator) for those allocated totreatment. because allocation into treatment is randomized,the average baseline outcome for those allocated into treatment is equal to the baseline outcomefor those not allocated into treatment. now write the average gain if treated forthose allocated to treatment (z=1) separately for those who choose not to receive treatment(t=0) and those who choose to receive treatment (t=1).a fraction of those offered treatment declines and one minus this fraction accepts.the gain for those who decline treatment is zero, as they are not treated.the gain for those who accept treatment can be rewritten as the gain conditional on beingoffered treatment (z=1) and accepting treatment

(t=1).we can now rewrite the average gain for those treated (both those accepting treatment andthose declining) as a weighted average of the gain for those accepting and those decliningtreatment. further, we assume that no one can enter treatmentwithout being offered treatment – that is we exclude the possibility that you can sneakinto treatment without being offered treatment. thus we only allow for dropping treatmentif offered. hence, if you accept treatment it must havebeen offered, that is if t = 1, this implies that z = 1 (but not necessarily the otherway around). from this, it also follows that the averagegain for those offered and accepting treatment

is the same as the average gain for thosewho accepted treatment – so we do only need to condition on that people accept treatment.from this, we are now able to derive the main results – the “bloom” equation namedafter its inventor, howard bloom. on the left hand side of the equation, wehave observed entities, that is, stuff that we can calculate from the observed data andon the right we have our object of interest, the average treatment effect for those treated.this implies that we can calculate the average treatment effect for those who accepted treatmenteven if some individuals selectively leaves treatment.now given the algebra on the previous slide, we can now prove the “bloom” equation.first, the average outcome for those offered

treatment can be rewritten as the averagebaseline for those offered treatment plus the average gain for those who accept treatmentconditional on being offered treatment. that is equal to the baseline outcome forthose not offered treatment (due to the rct) plus the gain for those offered and actuallytreated. this is, again, equal to the average baselinefor those not offered treatment plus the weighted average gain for those offered and acceptingand those declining treatment. this is equal to the average baseline outcomefor those not offered treatment plus the average gain for those offered and accepting treatmentweighted with the fraction who accept treatment when offered treatment.going back to the bloom equation at the top

of the slide, we can write the average outcomefor those not offered treatment as the baseline outcome for those not offered treatment – asthis group is not treated and they are thus unaffected by the treatment but otherwiseequal to those offered treatment. collecting everything, we can write the nominatorof the left hand side of the bloom equation as the baseline outcome for those not offeredtreatment plus the weighted gain for those offered and accepting treatment minus theaverage outcome for those not offered treatment. as can be seen everything but the right handside of the bloom equation nicely cancels out leaving the right hand side of the bloomequation. this concludes the proof.so you have just seen that despite non-random

dropout from an rct, we can still estimatethe causal effect of the treatment for those who accept treatment.note that this is not the same as the average causal effect for those offered treatment(including those who rejects treatment and this has zero effect).we will never know what would have happened to those who declined treatment, as this isnot necessarily the same as what happened to those accepted treatment due to selectivedrop out. we now turn to something different.when persons are selected into treatment, they are obviously aware that they are exposedto the treatment. this, by itself may affect behavior.therefore, while we can measure the causal

effect of the treatment, the interpretationis less clear if people respond to merely being observed to a treatment.do people change behavior because they are affected by the treatment or because theyknow they are being observed? this phenomenon is known as the hawthorne effect, names soafter the famous hawthorne plant where researchers try to manipulate productivity by changingthe work environment. however, it was later speculated that workerresponse was more due to being observed than to change in work environment.therefore, change in productivity was not a result of change in work environment butfrom being observed by researchers. therefore, research did not imply that changein work environment affects productivity but

that being observed affects productivity,at least while being observed. later, other researchers has doubted the so-calledhawthorne effect and concluded that the whole research design was flawed and that the datadoes not allow either conclusion. however, to illustrate the idea behind thehawthorne effect, we look at the star data. the table on the slide shows class size bytreatment arm – small classes, regular classes and regular classes with a teacher’s aide.from the table it can be seen that for classes of size 16 to 18 students, there are a numberof classes of equal size in all three treatment arms.thus, if it is the actual class size that matters and not treatment type, outcomes shouldbe the same in all three treatment arms when

actual class size is the same.if it is a hawthorne effect, there should be a difference across treatment arms forthe same class size. some caution should be considered here, though.because, even if students are allocated into treatments by lottery, actual class size couldbe a result of selective attrition and drop out after the lottery.if we are willing to assume that ordinary classes that are observed to be small is aresult of negative selection (a bad teacher for example) and small classes that are inhigh range for small classes is a result of positive selection (a good teacher) we shouldexpect that the causal difference between class types is larger than the observed difference.thus the estimated difference between treatments

arms for comparable class size is a lowerbound for the true difference. with the above caveat in mind the regressionson this slide shows the difference in math achievement in kindergarten for students inthe different treatments arms. students in small classes is the referencegroup. the regression results in the top panel showsresults for classes in the range less than 29 students and larger than 12 students, andthe bottom shows results for classes with less than 19 students and more than 16 students.this is the range from the table on the previous slide, where all treatment arms has classesof comparable size. from the regressions, we find that the effectof treatment arm is the same, irrespective

of whether we look at all class sizes or classeswhere class size is approximately the same across treatment arms.therefore, with the caveat from the previous slide about the causal effect in the lowerpanel probably being larger than the estimated effect, we are inclined to conclude that thecausal effect of being in a small class is more likely to be a hawthorne effect ratherthan being an effect of being taught in a small class.therefore, when teachers and/or students are allocated into a small class in the star study,this induced them to teach/study harder, not because they are in a small class but becausethey are expected to perform better from being in a small class.you should note that this example is made

up for illustrative purposes of this courseand that there is not a general agreement among researchers that the effect of the starproject was a hawthorne effect. until now, we have relied upon randomizationto infer the causal effect from a treatment. the upside of this was that it is the designthat allowed the researcher to infer causality and in principle, causality is undeniable.the downside is the external validity. is the observed causal effect due to the mechanismsof the treatment or is it a hawthorne effect? if a randomized controlled trial is infeasibleor if we want to rule out hawthorne effects, there are alternative designs; one of themost notable, is called the instrumental variables method.the basics of instrumental variables will

be laid out on the following slides and thenthe analogy to the estimator for the randomized controlled trail will be explained.say we want to estimate the return to education by running a regression of log earnings onyears of education. then we have learned that it would be dangerousto interpret the regression coefficient as the causal effect of years of education onlog earnings unless we have either randomized years of education or that we have the fullset of confounders that affect log earnings over and above years of education.so, in the absence of data from an rct on years of education, what to do?imagine that we have available a third variable, z, that affects education but is otherwiseuncorrelated with earnings.

think of z as a variable that when it changes,causes changes in the level of education but it has no direct effect of earnings.one example could be an educational reform that expands the minimum years of compulsoryschool. it certainly affects years of education butit is very unlikely that it affects individual earnings over and above education.obviously other things than a school reform may affect education.this is indicated by the error term u. also, other things than education may affectearnings. this is indicated by the error term e.it is also very likely that e and u are correlated as they both capture the effect of stuff,e.g.

intelligence that determines the level ofeducation and earnings. we may write the figure as equations instead.one equation for the level of earnings, y, and one equation for the level of education,x. the instrumental variable only affects thelevel of education and not earnings. hence, it should not appear in the equationfor earnings. note the resemblance to the treatment indicatorz previously. in an rct, z is the indicator of whether thesubject was allocated to the treatment or control group and t was the indicator of whethertreatment was actually accepted. here t is replaced by years of education,x.

however, the algebra is the same.because we can estimate the causal effect on the treated using randomization into treatment,we can also estimate the causal effect of years of education because z (e.g.the school reform) acts as a randomizer. in order for the instrument to deliver causaleffects we need it to be independent of everything else, just a randomization is independentof everything in the case of the rct. therefore, given years of education, x, theschool reform, z, must not have any direct effect on earnings, y.this in turn implies that z must be uncorrelated with what otherwise affects both years ofeducation as well as earnings. what follows does not seem to relate to howwe derived the bloom equation.

we return to this in a couple of slides.instead, we turn to how we derived the linear regression coefficient, now with the extensionof the instrumental variable equation. we start by working with covariance betweenthe dependent variable and the instrument. inserting the expression of the dependentvariables in terms of the x variable leads to that we can rewrite the covariance betweeny and z as the effect of x on y, b, times the covariance between x and z plus b timesthe covariance between e and z. this implies that we can write the fractionof the covariance between y and z and the covariance between x and z using the aboveexpression of the covariance between y and z and this gives us b plus a term involvingb and the fraction between the covariance

between e and z and the covariance betweenx and z. the denominator, the covariance between eand z is zero by assumption. hence, the fraction between the covariancebetween y and z and the covariance between x and z is equal to b, the causal effect ofx on y. therefore, the availability of an instrumentallows us to estimate the causal effect of x on y even when x and the error term, e arecorrelated. as an example of instrumental variables inthe case of the return to education, we use the well-known case of quarter of birth; seee.g. angrist and krueger (1995).the idea here is that due to the quarter of

birth, there is variation in when a personcan leave compulsory school. all pupils starts in compulsory school atthe same date but might leave when they turn 15.as quarter of birth vary across respondents but school start does not, quarter of birthmight affect the educational level of the respondents as some pupils are allowed toleave compulsory sooner than others. this is in fact the case as the top figureshows. using us panel data on birth cohorts fromthe 1930’s we find clear seasonal patterns in the mean years of education.thus, quarter of birth, in part, affects your level of education.this is a graphical illustration of the covariance

between the instrument (z – quarter of birth)and the independent variable (x – years of education).the next figure shows the covariance between log earnings and quarter of birth.here we also find a clear seasonal pattern. log earnings partly depends on your quarterof birth. if the instrumental variable assumption iscorrect – that the instrumental variable only affects dependent variables through theindependent variables, the reason for quarterly change in earnings is due to an indirect effectthrough earnings. note that there is no empirical way of verifyingthe instrumental variable assumption. it remains an assumption.but if it is true, the ratio between the data

in the two figures yields the causal effectof education on earnings. we can derive the iv estimator in an alternativeway that may be a little more intuitive. in the first stage, we regress x – yearsof education, on the instrument – here quarter of birth.for simplicity think of z as a binary dummy variable, taking the value one if the respondentis born in the first quarter and zero otherwise. from this we can obtain the predicted valuesof x given z. the virtue of the predicted values of x usingz is that the predicted values of x only pertain the part of the variation in x that is commonwith z. because z is independent from the error termu, by the iv assumption, the predicted value

of x using z is also independent from u.in the second stage, we use the predicted values of x instead of the observed valuesof x. note that the predicted values of x are alsoindependent of e, again by the iv assumption. using the covariance operator we can formallyshow what was verbally derived on the previous slide.essentially, we are estimating the slope of x on y using predicted rather than observedvalues of x. so the iv estimator is the covariance betweeny and the predicted values of x divided by the variance of the predicted values of x.replacing the predicted values of x by their expression in terms of the instrument z, weget the regression coefficient of z on x multiplied

by the covariance between y and z dividedby the variance of z. plugging in the definition of the regressioncoefficient of z on y we finally get that the iv estimator, the covariance of y andz divided by the covariance of x and z. note that this is not at proof that the ivestimator consistently estimates b – it just shows that we get the iv estimator ifwe use the two stage estimation procedure from the previous slide.however, we saw that the iv estimator was consistent on slide 10.we now conclude the derivations of the iv estimator by showing the analogy to the bloomequation. imagine that the instrument is binary – asin the example with quarter of birth in either

the first or the last three quarters.it can then be shown that the usual iv estimator, that is the covariance between y and z dividedby the covariance between x and z can be rewritten as the ratio between the difference betweenthe expected values of y across outcomes of the instrument and differences between expectedvalues of x across outcomes of the instrument. this is also known as the wald estimator.if we replace the dependent variable x with actual treatment and the instrument as whetherindividuals are randomized to treatment or control this is exactly the bloom equation.so in the iv case, the iv can be thought of as replacing the lottery in the randomizedcontrolled trial. actually, what the iv does, is that it replacesthe randomization by design with randomization

by nature.somehow, “nature” makes the randomization. to clarify the latter point we show some exampleson the next slide. imbens and van der klaauw (1995) studies theeffect of being having served in vietnam on subsequent earnings.the idea is that having served in vietnam may have reduced the earnings capacity ofveterans due to lost labor market experience on the “ordinary” labor market or psychicdisorders, such as posttraumatic stress disorder due to combat experience.however, one could imagine that those serving in vietnam are not comparable to those stayingat home. even though drafting to serve in the armywas made by lottery, exemptions was possible,

the former us president bush being a notableexample. therefore, imbens and van der klaauw use cohortindicators as instrumental variables. if one is willing to assume that, there areno differences in earnings capacity across cohorts, being born in a different cohort’syields “lottery” like differences in drafting probabilities independent of earnings.mcclellan et al. (1994) study the effect of heart attack surgeryon health. however, people hospitalized for heart surgerymay not be randomly selected. people with health insurance or people withknowledge of health problems may be more likely to be hospitalized than others.to generate lottery like variation in hospitalization

mcclellan et al.uses proximity to cardiac health care facilities. for individuals with the same background characteristicsbut different proximity have different opportunities for getting into surgery in time.evans and ringel (1999) studies the effect of maternal smoking on fetal birth weight.but it is very likely that women who smoke during pregnancy have a different health profilethan women who do not smoke during pregnancy. therefore, it would not be safe to assumethat the entire average difference in birth weight across women who smoke and do not smokeare the causal effect of smoking. to generate lottery like variation in smokingevans and ringel uses state cigarette taxes. as taxes vary across states, this is thoughtto generate differences in smoking habits

across otherwise identical women.in this lecture, you have learned that even though there may be selective dropouts fromrct’s one may still be able to estimate the causal effect of a treatment for thoseactually treated. further, you have learned that it is possibleto estimate causal effects in the absence of data from an rct.what is needed is something else that generate lottery like variation in the exposure ofthe dependent variable. if such variation is available, one can usethe instrumental variable approach to estimate causal effects.in the next and final lecture, you will learn even less restrictive methods that also maybe able to yield causal estimates form non-randomized

data.