Contents
Rationale
Research is often described as an active, diligent, and systematic process of inquiry aimed at discovering, interpreting, and revising facts. This intellectual investigation produces a greater knowledge of events, behaviors, theories, and laws and makes practical applications possible. The term research is also used to describe an entire collection of information about a particular subject, and is usually associated with the output of science and the scientific method. The word research derives from the French recherche, from rechercher, to search closely where "chercher" means "to search" (see French language); its literal meaning is 'to investigate thoroughly'. Research is funded by public authorities, by charitable organisations and by private groups, including many companies.
Learning Outcomes
Upon completion of this course, students will be able to
Knowledge
- describe the basics of logical analysis.
- understand the process of science
- appreciate the importance of theory in science
- describes the various ways in which reliability is measured and how SPSS for Windows can be used to do the computations.
- Understand the concept of Degrees of Freedom
- Undedrstand the distinction between applied and basic research.
- describe the logic and basic procedures for both planned comparisons and post hoc tests
- understand the within-subjects designs drawn from the research literature
- understand the concept of meta-analysis
Skill
Upon completion of this course, students will be able to
- utilise the Steven's Scales of Measurement
- undertake statistical analysis
- identify confounding variables
- compute Kappa by hand
- achieve random and stratified random samples
- do the basic computation of a Simple One-Way ANOVA
- use and/or undertake studies using the basic independent-groups design drawn from the research literature
- create a Latin Square of any size
- randomize within blocks
- compute a factorial analysis of variance
- construct and pilot a survey instrument
- select a statistical procedure and links to the instructions for the computational procedures
Today's
Videos
Teaching
and Learning Resources
Click on titles
Curiosity, Creativity, and Commitment. Research Is a Process of Inquiry.
Tutorials
Readings
Logic, from Classical Greek λόγος logos (the word), is the study of patterns found in reasoning. The task of the logician is to set down rules for distinguishing between valid and fallacious inference, between rational and flawed arguments.
Traditionally, logic is studied as a branch of philosophy, one part of the classical trivium, which consisted of grammar, logic, and rhetoric. Since the mid-nineteenth century logic has also been commonly studied in mathematics. More recently logic has been applied to computer science. The parts that make up a computer chip are often called "logic gates."
As a formal science, logic investigates and classifies the structure of statements and arguments, both through the study of formal systems of inference and through the study of arguments in natural language. The scope of logic is therefore large, ranging from core topics such as the study of fallacies and paradoxes, to specialized analyses of reasoning using probability and to arguments involving causality. Logic is also commonly used today in argumentation theory.
Science, in the broadest sense, refers to any system of objective knowledge. In a more restricted sense, science refers to a system of acquiring knowledge based on the scientific method, as well as to the organised body of knowledge humans have gained by the outcome of such research.
Fields of science are commonly classified along two major lines:
These fields are empirical sciences, which means the knowledge must be based on observable phenomena and capable of being tested for its validity by other researchers working under the same conditions.
Mathematics is sometimes classified in a third grouping, called formal science, having both similarities and differences with the natural and social sciences. It is similar to other disciplines in that it involves a careful, systematic study of an area of knowledge; it is different because of its method of verifying its knowledge, using a-priori rather than empirical methods. Mathematics as a whole is vital to the sciences; indeed, major advances in mathematics have often led to critical advances in the physical and biological sciences. Certain mathematical approaches are indispensable for the formation of hypotheses, theories, and laws, both in discovering and describing how things work (natural sciences) and how people think and act (social sciences).
Science as defined above is sometimes termed pure science in order to differentiate it from applied science, the latter being the application of scientific research to specific human needs.
The Starting Point: Asking Questions. Data and the Nature of Measurement
Tutorials
Readings
The Relationship Between the Research Question, Hypotheses, Specific Aims, and Long-Term Goals of the Project
Before you begin writing a grant proposal, take some time to map out your research strategy. A good first step is to formulate a research question.
A Research Question is a statement that identifies the phenomenon to be studied. For example, “What resources are helpful to new and minority drug abuse researchers?”
To develop a strong research question from your ideas, you should ask yourself these things:
- Do I know the field and its literature well?
- What are the important research questions in my field?
- What areas need further exploration?
- Could my study fill a gap? Lead to greater understanding?
- Has a great deal of research already been conducted in this topic area?
- Has this study been done before? If so, is there room for improvement?
- Is the timing right for this question to be answered? Is it a hot topic, or is it becoming obsolete?
- Would funding sources be interested?
- If you are proposing a service program, is the target community interested?
- Most importantly, will my study have a significant impact on the field?
|
 |
External links
A Research Library is a library which contains an in-depth collection of material on one or several subjects. A research library will generally include primary sources as well as secondary sources. Large university libraries are considered research libraries, and often contain many specialized branch research libraries. A representative large research library is the University of Califonia Berkeley Libraries, and a particularly distinguished specialized library that is part of this library system is the Bancroft Library at UC Berkeley.
Research libraries can be either reference libraries, which do not lend their holdings, or lending libraries, which do lend all or some of their holdings. Some extremely large or traditional research libraries are entirely reference in this sense, lending none of their material; most academic research libraries, at least in the U.S., now lend books, but not periodicals or other material.
See also
External links
A Literature Review is a body of text that aims to review the critical points of current knowledge on a particular topic.
Most often associated with science-oriented literature, such as a thesis, the literature review usually precedes a research proposal, methodology and results section. Its ultimate goal is to bring the reader up to date with current literature on a topic and forms the basis for another goal, such as the justification for future research in the area.
A good literature review is characterised by: a logical flow of ideas; current and relevant references with consistent, appropriate referencing style; proper use of terminology; and an unbiased and comprehensive view of the previous research on the topic.
According to Cooper (1988) "a literature review uses as its database reports of primary or original scholarship, and does not report new primary scholarship itself. The primary reports used in the literature may be verbal, but in the vast majority of cases reports are written documents. The types of scholarship may be empirical, theoretical, critical/analytic, or methodological in nature. Second a literature review seeks to describe, summarise, evaluate, clarify and/or integrate the content of primary reports".
Source: Cooper, H.M. (1988): The structure of knowledge synthesis - Knowledge in Society, vol. 1, pp, 104-126.
Retrieved from "http://wiki.mminf.univie.ac.at/wiki/index.php/Englischer_Ausdruck" SOURCES OF LITERATURE REVIEW
See also
The Level of Measurement of a variable in mathematics and statistics is a classification that was proposed in order to describe the nature of information contained within numbers assigned to objects and, therefore, within the variable. The levels were proposed by Stanley Smith Stevens in his 1946 article On the theory of scales of measurement. According to Stevens' theory of scales, different mathematical operations on variables are possible, depending on the level at which a variable is measured.
Statistical Analysis of Data
Tutorials
Readings
Quantitative Research is the systematic scientific investigation of quantitative properties and phenomena and their relationships. Quantitative research is widely used in both the natural and social sciences, from physics and biology to sociology and journalism.
The objective of quantitative research is to develop and employ mathematical models, theories and hypotheses pertaining to natural phenomena. The process of measurement is central to quantitative research because it provides the fundamental connection between empirical observation and mathematical expression of quantitative relationships.
The term quantitative research is most often used in the social sciences in contrast to qualitative research.
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities; it is also used and misused for making informed decisions in all areas of business and government.
Statistical methods can be used to summarize or describe a collection of data; this is called descriptive statistics. In addition, patterns in the data may be modeled in a way that accounts for randomness and uncertainty in the observations, to draw inferences about the process or population being studied; this is called inferential statistics. Both descriptive and inferential statistics can be considered part of applied statistics. There is also a discipline of mathematical statistics, which is concerned with the theoretical basis of the subject.
The word statistics is also the plural of statistic (singular), which refers to the result of applying a statistical algorithm to a set of data, as in employment statistics, accident statistics, etc.
A graph of a bell curve in a normal distribution showing statistics used in educational assessment, comparing various grading methods. Shown are standard deviations, cumulative percentages, percentile equivalents, Z-scores, T-scores, standard nine, and percent in stanine.
|
|
Applied Statistics is the use of statistics and statistical theory in real-life situations.
Anyone who is committed to empirical observation as a means of knowing the universe about us can apply statistics as a research tool. This obviously includes science but includes history and the arts as well. For example, econometrics makes heavy use of applied statistics to study the economy.
In each of these areas, we need to observe, recognize the potential for error in our observations, and plan our research to control the observational error.
See also
External links
|
 |
In statistics, a Meta-analysis combines the results of several studies that address a set of related research hypotheses. The first meta-analysis was performed by Karl Pearson in 1904, in an attempt to overcome the problem of reduced statistical power in studies with small sample sizes; analyzing the results from a group of studies can allow more accurate data analysis.
Although meta-analysis is widely used in epidemiology and evidence-based medicine today, a meta-analysis of a medical treatment was not published until 1955. In the 1970s, more sophisticated analytical techniques were introduced in educational research, starting with the work of Gene V. Glass, Frank L. Schmidt, and John E. Hunter.
The online Oxford English Dictionary lists the first usage of the term in the statistical sense as 1976 by Glass. The statistical theory surrounding meta-analysis was greatly advanced by the work of Nambury S. Raju, Larry V. Hedges, Ingram Olkin, John E. Hunter, and Frank L. Schmidt.
Because the results from different studies investigating different independent variables are measured on different scales, the dependent variable in a meta-analysis is some standardized measure of effect size. To describe the results of comparative experiments the usual effect size indicator is the standardized mean difference (d) which is the standard score equivalent to the difference between means, or an odds ratio if the outcome of the experiments is a dichotomous variable (success versus failure). A meta-analysis can be performed on studies that describe their findings in correlation coefficients, as for example, studies of the correlation between familial relationships and intelligence. In these cases, the correlation itself is the indicator of the effect size.
The method is not restricted to situations in which one or more variables is defined as "dependent." For example, a meta-analysis could be performed on a collection of studies each of which attempts to estimate the incidence of left-handedness in various groups of people.
Researchers should be aware that variations in sampling schemes can introduce heterogeneity to the result, which is the presence of more than one intercept in the solution. For instance, if some studies used 30mg of a drug, and others used 50mg, then we would plausibly expect two clusters to be present in the data, each varying around the mean of one dosage or the other. This can be modelled using a "random effects model."
Results from studies are combined using different approaches. One approach frequently used in meta-analysis in health care research is termed 'inverse variance method'. The average effect size across all studies is computed as a weighted mean, whereby the weights are equal to the inverse variance of each studies' effect estimator. Larger studies and studies with less random variation are given greater weight than smaller studies. A recent approach to studying the influence that weighting schemes can have on results has been proposed through the construct of gravity, which is a special case of combinatorial meta analysis.
Modern meta-analysis does more than just combine the effect sizes of a set of studies. It can test if the studies' outcomes show more variation than the variation that is expected because of sampling different research participants. If that is the case, study characteristics such as measurement instrument used, population sampled, or aspects of the studies' design are coded. These characteristics are then used as predictor variables to analyze the excess variation in the effect sizes. Some methodological weaknesses in studies can be corrected statistically. For example, it is possible to correct effect sizes or correlations for the downward bias due to measurement error or restriction on score ranges.
A weakness of the method is that sources of bias are not controlled by the method. A good meta-analysis of badly designed studies will still result in bad statistics. Robert Slavin has argued that only methodologically sound studies should be included in a meta-analysis, a practice he calls 'best evidence meta-analysis'. Other meta-analysts would include weaker studies, and add a study-level predictor variable that reflects the methodological quality of the studies to examine the effect of study quality on the effect size. Another weakness of the method is the heavy reliance on published studies, which may increase the effect as it is very hard to publish studies that show no significant results. This publication bias or "file-drawer effect" (where non-significant studies end up in the desk drawer instead of in the public domain) should be seriously considered when interpreting the outcomes of a meta-analysis.
Field Research: Naturalistic and Case-Study Research
Tutorials
Readings
A Case Study is a research strategy, sometimes likened to an experiment, a history, or a simulation, though not linked to any particular type of evidence or method of data collection (Yin 2003). It is qualitative research, as isn't often confused by laymen.
Rather than using large samples and following a rigid protocol to examine a limited number of variables, case study methods involve an in-depth, longitudinal examination of a single instance or event: a case. They provide a systematic way of looking at events, collecting data, analyzing information, and reporting the results. As a result the researcher may gain a sharpened understanding of why the instance happened as it did, and what might become important to look at more extensively in future research. Case studies lend themselves to both generating and testing hypotheses (Flyvbjerg, 2006).
Yin, on the other hand, suggests that case study should be defined as a research strategy, an empirical inquiry that investigates a phenomenon within its real-life context. Case study research means single- and multiple case studies, can include quantitative evidence, relies on multiple sources of evidence and benefits from the prior development of theoretical propositions. He notes that case studies should not be confused with qualitative research and points out that they can be based on any mix of quantitative and qualitative evidence (Yin, 2002). This is also supported and well-formulated in (Lamnek, 2005): "The case study is a research approach, situated between concrete data taking technique and methodologic paradigma"
|
|
The Observer-expectancy Effect, in science, is a cognitive bias that occurs when a researcher expects a given result and therefore unconsciously manipulates an experiment in order to find it. Because it can skew the results of experiments (especially on human subjects), double-blind methodology is used to eliminate the effect.
An example of the observer-expectancy effect is demonstrated in music backtracking; some people expect to hear hidden messages when reversing songs, and therefore hear the messages, but to others it sounds like nothing more than random sounds. Often when a song is played backwards, a listener will fail to notice the "hidden" lyrics until they are explicitly pointed out, after which they are obvious. Other prominent examples include facilitated communication, dowsing, and applied kinesiology.
The observer-expectancy effect is also called the experimenter-expectancy effect, observer effect, or experimenter effect.
See also
External links and references
Naturalistic Observation is a method of observation, commonly used by psychologists and social/behavioral scientists, that involves observing subjects in their natural habitats. Researchers take great care in avoiding making interferences with the behaviour they are observing by using unobtrusive methods. When using Naturalistic Observation, the researcher observes the studied behavior in its natural setting without attempting to influence or control it. Therefore, the studies are often conducted in places like streets, homes, and schools. Laboratory studies are rather uncommon for this method because the lab setting itself may "contaminate" the participants, who often act differently when they know they are being studied (see Bias). When a lab study can not be avoided, the researcher participates only as an observer. One advantage of this method is that the information collected is the direct response to a stimulus. However, determining the causes of the observed behavior is sometimes difficult, and special care must be taken to avoid potential Bias.
An Observational Science is a science where it is not possible to construct controlled experiments in the area under study. For example, in astronomy, it is not possible to create or manipulate stars or galaxies in order to observe what happens. Other examples of necessarily observational sciences include geology, paleontology, epidemiology, and much of the social sciences.
Other fields of scientific study can have observational as well as experimental aspects. In high-energy physics, for example, some interactions involving energies higher than can be created in any experiment can be observed indirectly through astronomical observations.
To substitute for the inability to directly construct experiments as part of the scientific method, two main strategies are used. First, multivariate statistical techniques allow the approximation of experimental control with statistical control. Secondly, experimental observations of previously-unobserved phenomena can be used to suggest new hypotheses and test existing ones. This can be seen as making use of pre-existing "natural experiments".
In the social sciences, sociology and economics are generally held to be examples of observational sciences, because of the impracticability (not to mention dubious ethical status) of manipulating whole societies or economies for experimental purposes. However, microeconomics can be regarded as an experimental science, because it is possible to set up experimental micro-economies.
Sometimes fields of study can change from being observational to being experimental: for example, until the early 21st century, the study of comets was entirely observational -- it became experimental when the first man-made cometary collision was engineered in 2005.
See also
External links
Correlational and Differential Methods of Research
Tutorials
Readings
In probability theory and statistics, correlation, also called correlation coefficient, indicates the strength and direction of a linear relationship between two random variables. In general statistical usage, correlation or co-relation refers to the departure of two variables from independence, although correlation does not imply causation. In this broad sense there are several coefficients, measuring the degree of correlation, adapted to the nature of data.
Hypothesis Testing, Validity, and Threats to Validity. Controls to Reduce Threats to Validity
Tutorials
Readings
A Hypothesis (from Greek ὑπόθεσις) consists either of a suggested explanation for a phenomenon or of a reasoned proposal suggesting a possible correlation between multiple phenomena. The term derives from the ancient Greek, hypotithenai meaning "to put under" or "to suppose". The scientific method requires that one can test a scientific hypothesis. Scientists generally base such hypotheses on previous observations or on extensions of scientific theories.
|
 |
One may be faced with the problem of making a definite decision with respect to an uncertain hypothesis which is known only through its observable consequences. A Statistical Hypothesis Test, or more briefly, hypothesis test, is an algorithm to state the alternative (for or against the hypothesis) which minimizes certain risks.
This article describes the commonly used frequentist treatment of hypothesis testing. From the Bayesian point of view, it is appropriate to treat hypothesis testing as a special case of normative decision theory (specifically a model selection problem) and it is possible to accumulate evidence in favor of (or against) a hypothesis using concepts such as likelihood ratios known as Bayes factors.
There are several preparations we make before we observe the data.
- The hypothesis must be stated in mathematical/statistical terms that make it possible to calculate the probability of possible samples assuming the hypothesis is correct. For example: The mean response to treatment being tested is equal to the mean response to the placebo in the control group. Both responses have the normal distribution with this unknown mean and the same known standard deviation ... (value).
- A test statistic must be chosen that will summarize the information in the sample that is relevant to the hypothesis. Such a statistic is known as a sufficient statistic. In the example given above, it might be the numerical difference between the two sample means, m1 − m2.
- The distribution of the test statistic is used to calculate the probability sets of possible values (usually an interval or union of intervals). In this example, the difference between sample means would have a normal distribution with a standard deviation equal to the common standard deviation times the factor
 where n1 and n2 are the sample sizes.
- Among all the sets of possible values, we must choose one that we think represents the most extreme evidence against the hypothesis. That is called the critical region of the test statistic. The probability of the test statistic falling in the critical region when the hypothesis is correct is called the alpha value (or size) of the test.
- The probability that a sample falls in the critical region when the parameter is θ, where θ is for the alternative hypothesis, is called the power of the test at θ. The power function of a critical region is the function that maps θ to the power of θ.
After the data is available, the test statistic is calculated and we determine whether it is inside the critical region.
If the test statistic is inside the critical region, then our conclusion is one of the following:
- The hypothesis is incorrect, therefore reject the null hypothesis. (Therefore the critical region is sometimes called the rejection region, while its complement is the acceptance region.)
- An event of probability less than or equal to alpha has occurred.
The researcher has to choose between these logical alternatives. In the example we would say: the observed response to treatment is statistically significant.
If the test statistic is outside the critical region, the only conclusion is that
- There is not enough evidence to reject the hypothesis.
This is not the same as evidence in favor of the hypothesis. That we cannot obtain using these arguments, since lack of evidence against a hypothesis is not evidence for it. On this basis, statistical research progresses by eliminating error, not by finding the truth.
In statistics a valid measure is one which is measuring what it is supposed to measure. Validity implies reliability (consistency). A valid measure must be reliable, but a reliable measure need not be valid. Validity refers to getting results that accurately reflect the concept being measured.
In psychology, validity is the ability of a test to measure what it was designed to measure, the degree to which the operational definition of a variable accurately reflects the variable it is designed to measure or manipulate.
Validity can be defined a number of ways, though there are no distinct "types" of validity. Validity is, first and foremost, a logical exercise, rather than a computational endeavor. Establishing validity is, essentially, supporting the claim made that the test measures or predicts the construct it purports to predict. At the heart of any validity discussion must be the idea of construct validity, which will be discussed below. The first area of validity that must be considered is the validity of the criterion upon which either groups are distinguished or predictions made. One of the most popular means of determining criterion validity, is to correlate measures with a criterion measure known to be valid, such as a measure of job performance that actually assesses performance on a given job or a specific task. When the criterion measure is collected at the same time as the measure being validated the goal is to establish concurrent validity; when the criterion is collected later the goal is to establish predictive validity. Similar to criterion validity is construct validity, where an investigator examines whether a measure is related to other variables as required by theory. Content validity, or face validity, is simply a demonstration that the items of a test are drawn from the domain being measured; it does not guarantee that the test actually measures phenomena in that domain.
According to classical test theory, predictive or concurrent validity (correlation between the predictor and the predicted) cannot exceed the square root of the correlation between two versions of the same measure -- that is, reliability limits validity.
Single-Variable, Independent-groups Designs
Tutorials
Readings
In probability theory and statistics, the Variance of a random variable (or somewhat more precisely, of a probability distribution) is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. Where the expected value shows the location of the distribution, the variance indicates the scale of the values. A more understandable measure is the square root of the variance, called the standard deviation. As its name implies it gives in a standard form an indication of the possible deviations from the mean.
The variance of a real-valued random variable is its second central moment, and it also happens to be its second cumulant.
In computer science and mathematics, a Variable (IPA pronunciation: [ˈvɛɹiəbl]) (sometimes called a pronumeral) is a symbolic representation denoting a quantity or expression. In mathematics, a variable often represents an unknown quantity that has the potential to change; in computer science, it represents a place where a quantity can be stored. Variables are often contrasted with constants, which are known and unchanging.
 |
The term has a similar meaning in the physical sciences and engineering: a variable is a quantity whose value may vary over the course of an experiment (including simulations), across samples, during the operation of a system. Variables are generally distinct from parameters, although what is a variable in one context may be a parameter in another. For more on this distinction, see the article on "parameter".
In applied statistics, a variable is a measurable factor, characteristic, or attribute of an individual or a system – in other words, something that might be expected to vary over time or between individuals. |
In mathematical statistics, 'variable' has a technical meaning - random variables or random sampling are defined in the mathematical context of measure theory as measurable functions from a probability space to a measurable space.
Correlated-Groups and Single-Subject Designs
Tutorials
How do correlated groups designs differ from independent groups designs?
Why are they called correlated groups designs?
Correlated-Groups Designs
Introduces a correlation between groups in the way groups are formed
Within-subjects design: Same participants in each group
Matched-groups design: Groups formed by matched random assignment
More sensitive than independent-groups designs
Controlling for individual differences makes it easier to detect small effects
The existence of a correlation between conditions has important implications for design and analysis
Factorial Designs
Tutorials
Readings
In statistics, a Factorial Experiment is an experiment whose design consists of two or more factors, each with discrete possible values or "levels", and whose experimental units take on all possible combinations of these levels across all such factors. Such an experiment allows studying the effect of each factor on the response variable, as well as the effects of interactions between factors on the response variable.
 |
For the vast majority of factorial experiments, each factor has only two levels.
|
In statistics, Analysis of Variance (ANOVA) is a collection of statistical models and their associated procedures which compare means by splitting the overall observed variance into different parts. The initial techniques of the analysis of variance were pioneered by the statistician and geneticist R. A. Fisher in the 1920s and 1930s, and is sometimes known as Fisher's ANOVA or Fisher's analysis of variance, due to the use of Fisher's F-distribution as part of the test of statistical significance.
A Second Look at Field Research: Field Experiments, Program Evaluation, and Survey Research
Tutorials
Readings
A Field Experiment applies the scientific method to experimentally examine an intervention in the real world (or as many experimental economists like to say, naturally-occurring environments) rather than in the laboratory. Field experiments, like lab experiments, generally randomize subjects (or other sampling units) into treatment and control groups and compare outcomes between these groups. Clinical trials of pharmaceuticals are one example of field experiments. Economists have used field experiments to analyze discrimination, health care programs, charitable fundraising, education, information aggregation in markets, and microfinance programs.
Program Evaluation is essentially a set of philosophies and techniques to determine if a program 'works'. It is a practice field that has emerged, particularly in the USA, as a disciplined way of assessing the merit, value, and worth of projects and programs. Evaluation became particularly relevant in the 1960s during the period of the Great Society social programs associated with the Kennedy and Johnson administrations. Extraordinary sums were invested in social programs, but the means of knowing what happened, and why were not available.
 |
Behind the seemingly simple question of whether the program works are a host of other more complex questions. For example, the first question is, what is a program supposed to do? It is often difficult to define what a program is supposed to do, so indirect indicators may be used instead. For example schools are supposed to 'educate' people. But what does 'educate' mean? Give knowledge? Teach how to think? Give specific skills? If the exact goal cannot be defined well, it is difficult to indicate whether the program 'works'.
Another question about programs is, what else do they do? There may be unintended or unforeseen consequences of a program. Some consequences may be positive and some may be negative. These unintended consequences may be as important as the intended consequences. So evaluations should measure not just whether the program does what it should be doing, but what else it may be doing.
Perhaps the most difficult part of evaluation is determining whether it is the program itself that is doing something. There may be other events or processes that are really causing the outcome, or preventing the hoped for outcome. However, due to the nature of the program, many evaluations cannot determine whether it is the program itself, or something else, is the 'cause'. |
One main reason that evaluations cannot determine causation involves self selection. That is, people select themselves to participate in a program. For example, in a jobs training program, some people decide to participate, and others, for whatever reason, do not participate. It may be that those who do participate are those who are most determined to find a job, or who have the best support resources, thus allowing them to participate and allowing them to find a job. The people who participate are somehow different from those who don't participate, and it may be the difference, not the program, that leads to a successful outcome for the participants, that is, finding a job.
If programs could, somehow, use random assignment, then they could determine causation. That is, if a program could randomly assign people to participate or to not participate in the program, then, theoretically, the group of people who participate would be the same as the group who did not participate, and an evaluation could 'rule out' other causes.
However, since most programs cannot use random assignment, causation cannot be determined. Evaluations can still provide useful information. For example, the outcomes of the program can be described. Thus the evaluation can say something like, "People who participate in program xyz were more likely to find a job, while people who did not participate were less likely to find a job."
If the program is fairly large, and there are many participants, and there is enough data, statistical analysis can be used sometimes to make a 'reasonable' case for the program by showing, for example, that other causes are unlikely.
Another approach is to use the evaluation to analyze the program process. So instead of focusing on the outcome (for example, did people in a jobs training program get jobs), the evaluation would focus on what the program was doing. For example, did people seem to learn the skills being taught? Did people stay in the program or did they drop out part way through? Were the teachers teaching appropriate skills? And so forth. This information could help how the program was operating.
People who do program evaluation can come from many different backgrounds, such as sociology, psychology, economics, social work or many other areas. Some graduate schools also have specific training programs for program evaluation.
Program evaluations can involve quantitative methods of social research or qualitative methods or both.
Statistical Surveys are used to collect quantitative information about items in a population. Surveys of human populations and institutions are common in political polling and government, health, social science and marketing research. A survey may focus on opinions or factual information depending on its purpose, and many surveys involve administering questions to individuals. When the questions are administered by a researcher, the survey is called a structured interview or a researcher-administered survey. When the questions are administered by the respondent, the survey is referred to as a questionnaire or a self-administered survey.
Final Preparations before Data Collection
Tutorials
Readings
Research Process. Generally, research is understood to follow a certain structural process. Though step order may vary depending on the subject matter and researcher, the following steps are usually part of most formal research, both basic and applied:
A common misunderstanding is that by this method a hypothesis can be proven. Instead, by these methods no hypothesis can be proven, rather a hypothesis may only be disproven. A hypothesis can survive several rounds of scientific testing and be widely thought of as true (or better, predictive), but this is not the same as it having been proven. It would be better to say that the hypothesis has yet to be disproven.
A useful hypothesis allows prediction and within the accuracy of observation of the time, the prediction will be verified. As the accuracy of observation improves with time, the hypothesis may no longer provide an accurate prediction. In this case a new hypothesis will arise to challenge the old, and to the extent that the new hypothesis makes more accurate predictions than the old, the new will supplant it.
Research Methodology: An Evolving Discipline
Tutorials
Readings
The Centre’s research programme is led by priorities identified in the Centre strategy, but also aims to stimulate imaginative new developments in both quantitative and qualitative research methods and to be responsive to new needs and opportunities that arise. The research programme is in general based upon the following kinds of projects:
- Longer-term projects: These are undertaken primarily by the Nodes and focus on innovative methodological development within the context of substantive research problems.
- Networks for methodological innovation: These are commissioned and co-ordinated by the Hub with the aim of stimulating debate on new methodological challenges and reviewing developments within specific methodological fields.
- Methodological reviews: These are commissioned externally or carried out by Hub staff.
- Affiliated projects: These are co-ordinated by the Hub.
- Hub based research projects: These are undertaken primarily by Hub staff.
See also
Recommended Texts
Resources
Research Methods Knowledge Base
Science Portal
Writing a Research Report: APA Publication Style
Conducting Library Research
Statistical Resources
Statistical Computation Procedures
Using SPSS for Windows
Examples of Using the Decision-Tree Flowcharts
Random Numbers
Sampling is that part of statistical practice concerned with the selection of individual observations intended to yield some knowledge about a population of concern, especially for the purposes of statistical inference. In particular, results from probability theory and statistical theory are employed to guide practice.
The sampling process consists of 5 stages:
- Definition of population of concern
- Specification of a sampling frame, a set of items or events that it is possible to measure
- Specification of sampling method for selecting items or events from the frame
- Determine the sample size
- Implement the sampling plan
- Sampling and data collecting
- Review of sampling process
Making Observations
Observation is an activity of a sapient or sentient living being, which senses and assimiliates the knowledge of a phenomenon in its framework of previous knowledge and ideas.
However, personal observations gathered without the aid of instruments are often unreliable¹ and not always reproducible. Therefore they are not of much use in exact sciences like physics. It is therefore often necessary to use various instruments like: spectrometers, oscilloscopes, cameras, telescopes, interferometers, taperecorders, thermometers etc. and tools like clocks, scale that help in improving the accuracy, quality and utility of the information obtained from an observation. Observation invariably requires logical thinking as logic is necessary for assimiliation of the knowledge that is presented by an observation.
The accuracy and tremendous success of science is primarily attributed to the accuracy and objectivity of observation of the reality that science explores.
It is uncertain that reality can be observed at all.
Research Examples
We have identified dozens of research examples that typify the range of studies in each of the research approaches listed below. Please note that typical research studies are multifaceted--often representing several different types of studies in a single research program. We have provided references and brief descriptions of studies as illustrations.
|
