2.8.B Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.
Key Concepts
Validity
According to Jack Phillips (1997), there are five ways to determine whether an instrument is valid:
- Content validity is the extent to which the instrument represents the program’s content.
- Construct validity is the degree to which an instrument represents the construct it’s supposed to measure. The construct is the abstract variable that the instrument is intended to measure, such as knowledge or skill.
- Concurrent validity is the extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics.
- Criterion validity is the extent to which the assessment can predict or agree with external constructs. It is determined by looking at the correlation between the instrument and the criterion measure.
- Predictive validity is the extent to which an instrument can predict future behaviors or results.
Reliability
Reliability is the ability of the same measurement to produce consistent results over time. Certain types of data are inherently reliable.
Test Difficulty
Test reliability and validity are maximized if the level of difficulty for answering most test items is more than just a lucky guess and, at the same time, the degree of difficulty is not too high.
- Split-half reliability is a way to test reliability in which one test is split into two shorter ones.
- Test–retest check of reliability is an approach in which the same test is administered twice to the same group of people.
Research Methods (Scientific)
- Observation
- Participation
- Case Study
- Observation
- Survey
- Interview
- Focus Group
Types
- Experimental design includes a controlled factor or group that is given special treatment for purposes of comparison with a constant or controlled group.
- Correlational research is a type of nonexperimental research where the measures of two variables are assessed for the statistical relationship (correlation) between them. Correlation is a measure of the relationship between two or more variables; if one changes, the other is likely to make a corresponding change. If a change moves the variables in the same direction, it is a positive correlation; if a change moves the variables in opposite directions, it is a negative correlation.
- Meta-analytic research is the statistical procedure for combining data from multiple studies. When the influence is consistent from one study to the next, meta-analysis can be used to identify a common effect.
- Longitudinal research involves repeated observations of the same variables, such as people over short or long periods of time.
- Cross-sectional research involves groups of people who differ in the specified variable but share other characteristics, such as socioeconomic status, educational background, or ethnicity.
- Quasi-experimental research designs occur when a treatment is administered to only one of two groups whose members were randomly assigned.
