Week 1

Bestudeer Morling p.193-214 (helemaal achterin) 1 Survey Research

What are surveys and what can they do?

• All surveys are highly structured, meaning they ask respondents prewritten, closed-ended questions with fixed response questions
  • Primary data collection: when social scientists design and carry out own ata collection
  • Secondary data source: a resource collected by someone else
  • Key informant: a person who is usually quite central or popular in the research setting and who shares his or her knowledge with the researcher or a person with professional or special knowledge about the social setting
  • Self-administerd questionnaire (SAQ): a survey completed directly by respondetns thourgh the mail or online
  • Mode of administraion: the way the survey is administered, such as face to face, by phone or mail, or onine
  • Closed-ended question: a focused interview question to which subjects can respond only in preset ways
  • Open-ended question: a broad interview question to which subjects are allowed to respond in their own words rather than in preset ways
• Cross-sectional surveys collect date at a single point in time, whereas longitudinal surveys collect data at multiple points in time
  
  • Cross-sectional survey: a survey in which data are collected at only one time point
  • Longitudinal survey: a survey in which data are collected at multiple time points
  • Repeated cross-sectional survey: a type of longitudinal survey in wich data are collected at multiple time points but from different subjects at each time point
  • Panel survey: a type of longitudinal survey in which data are collected from the same subjects at multiple time points
  • Attrition: the loss of sample members over time, usually to death or dropout
• Advantages of surveys include their ability to collect a wide breadth of information from large groups of people and to asses changes in populations over time, as well as their high level of external validity.
  
  • Poll: a very brief single-topic survey
  • Split-ballot design: a survey in which a randomly selected subset of respondents, typically 50% of those persons selected to participate in the survey, receives one topical module while the other 50% receives a different topical ballot.
• Surveys are subject to multiple sources of error, including nonrespsonse, measurement error, coverage error, and sampling error.
  
  • Measurement error: a type of error that occurs when the approach used to measure a particular variable affects or biases the response provided
  • Coverage error: a type of error that occurs when the sampling frame does not adequately capture all members of the target population, either by systematiclly omitting some or including others more than once
  • Sampling error: the difference betwen the estimates from a sample and the true parameter that arise duet o random chance

Types of surveys: modes of administration

• Face-to-face interviews are often seen as the ‘gold standard’ as the interviewer can ensure the respondent udnerstands the survey an does not skip questions.
  • Interviewer effects: the possibility that the mere presence of an interviewer, or that the interviewer’s personal characteristics, may lead a respondent to answer questions in a particular way, potentially biasing the responses
  • Interview schedule: a prepared list of questions and follow-up prompts that the interviewer asks the respondent
  • Paradata: information about the process by which the survey data were collected
  • Paper-and-pencil interview (PAPI): a survey interview in which the researcher asks questions and records the respondent’s answers in a preprinted copy of the survey booklet
  • Computer-assisted personal interview (CAPI): a face-to-face interveiw in which the researcher uses a laptop or tablet computer that is pre-programmed wit hall of the survey questions and response categories
  • Showcard: a preprinted card that reminds the respondent of all the response options for a particular question or questions
  • Skip pattern: a questions or series of questions associated with a conditional response to a prior question
  • Screener question: a question that serves as a gateway to (or detour around) a follow-up question; also called a filter question
  • Social desirability bias: a type of bias that occurs when study participants report positviely valued behaviors and attitudes rather than giving truthful responses
  • Audio computer-assisted self-interview (ACASI): an interview in which the respondent uses a laptop or tablet to listen to and answer prerecorded questions
• Telephone surveys, like face-to-face surveys, benefit from the guidance of an interviewer. They are also more cost effective and require less advance planning.
  
  • Computer-assisted telephone interview (CATI): a telephon einterveiw in which the respondent uses a laptop or tablet computer that is preprogrammed wit hall of the survey questions an response categories
• Mail surveys are cost effective and generally do not suffer from social desirability biases. Howver, they tend to garner low response rates.
• Onine surveys are becoming increasingly common because they’re cost effective, easy to administer, convenient, and can target large, representative samples. However, particpation may be biased toward younger people and those with most resources.

Bestudeer Morling p.203-230 Bivariate correlational research, herhaling van de stof uit KOM

• Association claims stat that 2 vairables are linked, but they do not state that one causes the other.
• Associaton claims are supprted by correlational reserarch, in which both variables are measured in a se tof participants. (if either of the variabiles is manipulated, the study is an experiment, which coudl potentially support a causal claim)
• Lees de rest op blz 233!
• Interrogating
  
  • Larger effect sizes allow more accurate predictions
  • Larger effect sizes are usually more important
  • The p value is the probability that the sample’s association came from a population in which the association is zero. If the p associated with the result is very small, that is less than 5%, we know the result is very unlikely ot have come from a zero-association population. The correlation is considered statistically significant.
    
    • Usually, the larger the effect size, the more likely i twill be statistically significant
    • The larger the sample, the more likely i twill be statistically significant
    • Restriction of range: if there is not a full range of scores on one of the variables in the association, it can make the correlation appear smaller than it really is.

Lees Field (5th ed.): o HS 4: §4.4-4.5, 4.6.1, 4.6.2, 4.6.5, 4.6.6, 4.6.7, 4.8 o HS 5: §5.3-5.5, 5.8 o HS 8: §8.3, 8.4, 8.4.1, 8.4.2

Geen tt stof

Bestudeer Morling p.214-231 Survey content: what do we ask and how do we ask it?

• Survey content: what do we ask and how do we ask it?
  • Most survey questions are close-ended with fixed response categories. Response categories should be mutually exclusive and exhaustive.
    • Survey instruments: the types of questions asked, the response categories provided, and guidelines that help survey designers organize their questions
    • Stem: the part of the survey question that represents the issue about which the question is asking
    • Dichotomous question: when only 2 options are available to a question, such as ‘yes’ or ‘no’
    • Mutually exclusive: preset response categories that do not overlap with one another, ensuring that respondents select the single category that best captures their views
    • Exhaustive: preset response categorie that give all subjects as least one accurate response
  • Many concepts that social researchers study may be best captured with a composite measure. Indexed and scales are two types of composite measures.
    
    • Rating scale: a series of ordered categories
    • Likert scale: a type of rating scale that captures the respondents level of agreement or disagreement with a particular statement
    • Forced choice: survey questions that do not offer choices like ‘neither agree nor disagree’, therefore forcing the respondent to indicate their general leaning toward either agreement or disagreement
    • Acquiescence bias: the tendency for respondents to asnwer ‘agree’ to close-ended attitudinal questions
    • Ranking items: a type of closed-ended question that asks respondents to rank-order their priorities or preferences
    • Unfolding question: a sequence of questions intended to elicit respondent estimates about topics such as income when respondents are uncertain of the answer; also called unfolding brackets
    • Composite measure: a measure that combines multiple items, whether as a scale or an index, to create a single value that captures a multifaced concept
    • Index: a composite measure that sums responses to survey items capturing key elements of a particular concept being measured
    • Scale: a composite measure that averages responses to a series of related items that capture a single concept or trait, such as depressive symptoms or self-esteem
  • High-quality surveys use clear, neutral language, strive for precision, and avoid double-barred and leading questions. The placement of questions can help researchers establish rapport with respondents , reduce monotony, and limit order effects.
    
    • Double-barred question: a question that asks about two or more ideas or concepts in a single question
    • Codebook: a system of organizing information about a dataset, including the variables it contains, the possible values for each variable, coding schemes, and decision rules.
    • Response set: the tendency to select the same answer to several sequential questions, perhaps out of boredom or a desire to quickly finish the survey
    • Order effects: when the order in which questions appear biases the responses
    • Priming effects: a type of order effects in which exposure to a particular image, word, or feeling shapes how respondents think and feel in the immediate aftermath
• Conducting survey
  
  • Before conducting the servey, researchers often do a pretest with a group of people who are similar tot he target sample.
    
    • Cognitive interview: an interview with survey respondents to understand how they interpret particular questions and terms. Respondents ‘think out loud’ as they answer the questions. Also called a cognitive pretest.
  • To analyze the pretest, researcher scan use both ‘soft’ date, such as commetns from repsondents, as well as ‘hard’ data, such as preliminary statistical analyses.
    
    • Frequency distribution: a presentation of the possible values of a variable along with the number of observations for each value that was observed
• Ethical concerns in survey research
  
  • Survey researchers must take care to protect respondents confidentiality. It is unrealistic and impractical for researchers to promise anonymity tot heir respondents.
    
    • Confidentiality: the participants’s identifying information is only accesible to the research team
    • Anonymity: when no idenityfing information can be linked to respondents and even the researcher cannot identify them
  • Researchers should be sensitive to survey questions that may cause respondents to experience psychological distress.

Bestudeer Morling p.153-164 Construct validity of surveys and polls

• Survey question formats inlcude open-ended, forced-choice, Likert scale, and semantic differential
  • Semantic differential: rating a target object using a numeric scale that is anchored with adjectives
• Sometimes the way a survey question is worded can lead people tob e more likely or less likely to agree wit hit
  
  • Leading question
• Double-barred and negatively worded questions are difficult to answer in a valid way
  
  • Double-barred questions: it asks two questions in one
  • The order in which questions are asked can also affect the responses to a survey
• People sometimes answer survey questions with an acquiescent or fence-sitting response tendency or in a way that makes them look good. Researchers can add items to a survey or change the way questions are written, in order to avoid some of these problems.
  
  • Acquiescence: yeah-saying; this occrus wehn people say ‘yes’ or ‘strongly agree’ to every item instead of thinking carefully about each one
  • Fence sitting: playing it safe by answering in the middle of the scale
  • Socially desirable responding/faking good/faking bad
• Surveys are efficient and accurate ways to assess people’s subjective feelings and opinions; they may be less appropriate for assessing people’s actual behavior, motivations, or memories.
  
  • Self-reports can be inacurate, especially when people are asked to describe why they are thinking, behaving, or feeling the way they do
  • Self-reporting memories of events are not very accurate
  • Consumer’s ratings were correlated with the cost of the product and the prestige of its brand, not its quality

Bestudeer Morling p.499-500 Is that correlation significant?

• Null hypothesis testing
  • Step 1: We assume there is no relationship in the population.
  • Step 2: we collect some data and calculate r.
  • Step 3: we estimate the probability of getting the r we got, or one more extreme, if the null hypothesis is true.
  • Step 4: we decide whether to reject or retain the null hypothesis.
  • When we reject the null hypothesis, we are concluding that the relationship, the r, we observed in our sample is statistically significant – in other words, that r we obtained is unlikely to occur just by chance in a null hypothesis population.
• The correlation coefficient has a sampling distribution
  
  • If the null hypothesis is true, most of the values of would be around 0.
  • The larger our study’s r is (the closer it gets to 1.0 or -1.0), the less likely it becomes that we would get that value, just by chance, if the null hypothesis is true.
• Sample size and r
  
  • A larger r (a stronger r) is more likely to be statistically significant.
  • When a study has a very large sample, it requires a large value or r to be statistically significant. In contrast, when a study has a very large sample, a small r will be statistically significant.

Week 2

Bestudeer Field (5th ed.) HS 18: §18.9-§18.12

-

Bestudeer Field (5th ed.) HS 9: §9.2, 9.7, 9.8, 9.10-9.10.3

-

Bestudeer Morling HS 9 p.244-256 Ruling out third variables with MR analysis

• In a regression design, researchers start with a bivariate correlation and then measure other potentional third variables that might affect it.
  • Multiple regression (or multivariate regression) can help rule out some third variables, thereby addressing some internal validty concerns
• Using multiple-regression analysis, researcher scan see whether the basic relationship is still present, even when they statistically control for one or more third variables. If the beta is still significant for the key variable when the researchers control for the third variables, it means the key relationship is not explained by those third variables.
  
  • In a scatterplot, this becomes visible if you separately consider subgroups and there is no relationship between the two variables.
  • Criterion variable/dependent variable: the variable researchers are most interested in understanding or predicting
  • Predictor variables/independent variables: the rest of the variables
• If the beta becomes significant when the researchers control for a third variable, then the key relationship can be attributed tot hat third variable.
  
  • There will beo ne beta value for each predictor variable. Beta is similar to r, but it reveals more than r does. A positive beta, like a positive r, indicates a positive relationship between that predictor variable and the criterion variable, when the other predictor variables are statistically controlled for. A negative beta, like a negative r, indicates a negative relationship between two variables. A beta that is zero, or not significantly different from zere, represents no relationship. The higher the beta is, the stronger the relationship is between that predictor variable and the criterion variable.
  • The coefficient b represents an unstandardized coefficient. A b is similar to beta in that the sign of b – positive or negative – denotes a positive or negative association (when the other predictors are controlled for). But unlike two betas, we cannot compare two b values within the same tablet o each other, as b values are computed form the original measurements of the predictor variables. Whereas betas are compute from predictor variables that have been changed to standardized units. A predictor variable that shows a large b may not actually denote a stronger relationship tot he criterion variable than a predictor variable with a smaller b.
    
    • Thus: a positive beta, like a positive r, means higher levels of seks on TV go with higher pregnancy risk (and lower levels of seks on TV go with lower pregnancy risk), even when we statistically control fort he other predictor on this table – age. In other words, even when we hol dage constant statistically, the relationship between exposure to TV sex and pregnancy is still there.
  • When p is less than 0.05, the beta (the relationship between the predictor variable and the criterion variable, when the other predictor variables are controlled for) is considerd statisticall significant. When p is greater that 0.05, the beta is considered not significant, meangin we cannot conclude beta is different from zero.
    
    • What if beta is not significant? The relationship goes away when potential third variables are controlled for, for example ‘when you hold parental involvement constant, there is no longer a relationship between frequency of family meals and academic success.
• Even though regression analyses can rule out third variables, they cannot definitively establish causation because they can only control for possible third variables that the researchers happene to measure. An experiment is the only design that definitively established causation.

Bestudeer Morling HS 9 p.264-265 Multivariate designs and the four validities

• Internal validity
  • Longitudinal designs help establish temporal precedence
  • Multiple regression analysis helps rule out third variables
• Construct validity
  
  • How well each variable is measured
  • Reliability
• External validity
  
  • Whether we can generalize from this sample tot he population
    
    • Random sample
• Statistical validity
  
  • Effect size and statistical significance
  • When researchers use multivariate designes, they need to take precautions to look for subgroups, outliers and curvilinear associations

Bestudeer Field (5th ed.) HS 9: §9.3, 9.14

-