- So far in this video series we've talked about how to develop theories and what it means to pose testable hypothesis. We've also discussed important aspects of measurement and data collection but we're not quite done yet. After we've post a research question, developed a theory, identified a testable hypothesis and collected data. We still need to analyze our data to understand what it tells us about our hypothesis. Specifically, we want to answer questions like is there sufficient evidence to support our hypothesis? And more specifically, is there really a relationship between our independent variable and our dependent variable? In this video we'll start talking about how to answer these questions. After watching this video, you should be able to identify the role of hypothesis testing in the scientific method. Outline the steps of hypothesis testing and identify the limitations of bivariate hypothesis testing. To start answering these questions we'll consider an example. A well-known theory in international relations argues that democracies are relatively more peaceful than non democracies. Let's say we've already theorized about why this may be the case and discussed how to measure important concepts in the theory like democracy and peaceful. And let's also say that we've already collected data. So specifically we have a list of some countries in the world today, whether they are democracies or not and a conflict score, which tells us how peaceful or how conflict prone the country is. And let's say that a higher conflict score means that the country experienced more conflict, so it's less peaceful. We hypothesize that democracies will have less conflict than non democracies and we'll test this hypothesis using the conflict score that I just described. When we average the conflict scores of each type of country we see that democracies have a score of 30 and non democracies have a conflict score of 35. So democracies have a lower conflict score on average but in general, the two scores seem pretty close. So how do we know if this difference is real? In other words, how do we know this difference isn't just due to chance? Well, to answer this question we can use a bivariate hypothesis test. What does it mean to conduct a bivariate hypothesis test? What are we trying to do? Well, in general, when we conduct a bivariate hypothesis test we're asking, what would my sample look like if there was no relationship between my two variables in the population? And does my sample look different enough from this that I can conclude it's unlikely that I would get the sample if there really was no relationship in the population? That might sound a little unintuitive at first and that's totally okay. The reason we set up hypothesis testing like this is because as researchers, we want to place the burden on ourselves to provide sufficient evidence supporting our hypothesis. In other words, we wanna make the process of hypothesis testing just a little more difficult so that we can be really confident in our conclusions. Although there are lots of different types of hypothesis and hypothesis tests there are a few main steps that you'll go through to test your hypothesis regardless of what kind of test you're using. In this video, we'll talk briefly about what each step looks like in the context of our example hypothesis. The first step is to identify the null hypothesis. In our case, our hypothesis, which technically is our alternative hypothesis, was that democracies will have a lower conflict score than non democracies. Our null hypothesis then is that there's no difference between the conflict scores and democracies compared to non democracies. So you might be wondering why it makes sense to identify a null hypothesis when what we're really interested in is our alternative hypothesis. But remember that when we started testing our hypothesis we start by thinking about what our data would look like if there was no relationship between our two variables. And in other words, what our data would look like if the null hypothesis was true. And we gain confidence in our hypothesis when our sample looks really different than what we would expect if the null hypothesis were true. After we've identified both the alternative and null hypothesis we then pick what we call a significance level. You can think about this step as deciding how strongly evidence from our sample must be before we're ready to reject the null hypothesis. Technically any number between zero and one can be used as a significance level but we usually choose .01, .05 or .1 by convention. So let's say we chose .05. By choosing 0.05 we're basically saying that we're comfortable with taking a 5% risk that we'll incorrectly conclude that a real difference exists in the conflict scores between democracies and non democracies when there's actually no difference in real life. If we were to choose 0.01 or 1% instead we would be demanding stronger evidence from our sample. The next step is to choose an appropriate hypothesis test given our data and our hypothesis. In our case we could use a difference of means test because we're comparing two means or averages. The mean conflict score of democracies and the mean conflict score of non democracies. A difference of means test asks whether the averages we're interested in, so here the mean conflict score of democracies and the mean conflict score of non democracies, are really different in real life or in the population. Or if instead the differences we see in our sample are probably just due to random chance. After we identify an appropriate type of hypothesis test we use the formula associated with the test to calculate what we call a test statistic and a p value. To do this you'll probably need some other information about your data like your sample size. The final step is to interpret the p value you've calculated and decide whether we have enough evidence to say that there is a relationship between our variables. But technically, we decide whether we can reject the null hypothesis. So note the particular language we're using here. We don't decide whether the alternative hypothesis is true. Instead, we decide whether we can reject the null hypothesis. So when do we reject the null hypothesis? Well, when our p value is less than the value we chose for our significance level. Recall that we chose a significance level of .05. So if we got a p value of .02 in this case, we would reject the null hypothesis. But if we instead got a p value of .1 we would fail to reject the null hypothesis. And that's the basic idea of hypothesis testing. Now, there are some important things to keep in mind about bivariate hypothesis testing. First, concluding that we can reject the null hypothesis does not mean that we've necessarily identified a causal relationship. The bivariate test can tell us if a relationship exists but not whether the relationship is causal. Recall that as researchers we need to consider confounding variables when discussing causality. On their own bivariate tests do not account for confounding variables. So our difference of means test in this example can give us confidence that democracies are more peaceful than non democracies but it doesn't take into account that there could be a third variable causing both the level of conflict in the country and whether the country is democratic. Second, you'll also want to keep in mind that tests like this also depend on having a random sample. So we can't use them in all situations. Finally, note that a difference might be statistically significant, as in we got a small p value, but not actually substantially important in the real world. For example, let's say we conducted a study that looked at the effect of campaign ads on whether people vote. And let's say that we found that TV ads increase the chances that people vote. Meaning we could reject the null hypothesis that TV ads did not increase voting. But it turns out that TV ads only increased the chance that someone would vote by .00001%. Does this difference really matter in the real world? Well, that's a judgment call that the researcher makes but not something the p value can tell us. To conclude, let's review what we've discussed. In this video, we introduced hypothesis testing and discussed a scenario in which a bivariate test could be useful. We had data suggesting that democracies might be more peaceful than non democracies but we wanted to be sure that this difference was unlikely to be a result of chance. We then briefly discussed each of the steps of hypothesis testing. So just to recap, those five steps were, first, state the null and alternative hypothesis. Then choose the significance level and it's related p value. Identify the hypothesis test appropriate for the type of data and your research question. Analyze the data, meaning obtain the test statistic and p value. And interpret the result and decide whether you can reject the null hypothesis. Finally, we noted that there are important limitations to bivariate hypothesis testing. I recommend that you use the quiz questions in this module to check for your understanding.