It is a fact that I have no liberal friends who buy into the notions of conspiracy theories. All of the liberal friends that I have know how to read research. They know what reliability and validity mean. They know that isolated studies do not substantiate anything, and that an empirical body of evidence must exist in support of a hypothesis before one can consider it to be true. I’m not saying that ALL liberals do….but the ones that I know do, which is probably why we’re friends.
I do have Republican friends and a lot of Libertarian friends who do believe in conspiracy theories, however. They love to spout off about the “science” they heard someone else talk about at some point in time……with no substantiation, whatsoever. While I love some of these people dearly, they’re the ones who frequently quote Einstein without a clue about how to interpret his work, the context it was intended for, nor bother to even check up to see if the little Facebook memes they post and attribute to Einstein actually came from the man.
Take, for instance, the conspiracy theory about chem trails. The theory is that Bill Gates or, at times, someone else, is spraying the atmosphere with harmful chemicals in order to attain mind control. Tell me something. When was the last time you experienced mind control? I mean…if you’re broke, it’s probably because you can’t manage or earn money…..Not because your mind is being controlled. If something doesn’t turn out the way you want it to, it probably has to do with the decisions that you make rather than because your mind is being controlled by the government. Right? What I don’t particularly understand about the chem trails conspiracy is this. Chem trails are made of aluminum, strontium and barium. Look it up. That is what they contain. These things….. Aluminium, Strontium and Barium aren’t even particularly harmful, so my reasoning is that you are going to imagine a global conspiracy to dose people with stuff you should pick something more dangerous than some random metals. Right? You know, go with drugs or something?
Then, there are the conspiracy theorists who DO go with the drugs theory….saying that the government is putting drugs in our drinking water in the form of fluoride. Yes, research supports the fact that exposure to excessive consumption of fluoride over a lifetime may lead to increased likelihood of bone fractures in adults, and may result in effects on bone leading to pain and tenderness. Children aged 8 years and younger exposed to excessive amounts of fluoride have an increased chance of developing pits in the tooth enamel, along with a range of cosmetic effects to teeth. BUT HOW MUCH FLUORIDE DOES IT TAKE TO DO THIS? They don’t ever stop to do the research about THIS aspect of the argument.
The fact is, that the amounts of fluoride used in public drinking water are so small that one would have to consume virtually an entire lake before experiencing the negative results. The benefits are FAR more prominent, in fact. Listen to this. Fluoride exists naturally in virtually all water supplies and even in various brands of bottled water. IT IS ALREADY THERE. What’s at issue is the amount of fluoride in water. There are proven benefits for public health that come from having the optimal level of fluoride in the water — just enough to protect our teeth. In 2011, federal health officials offered a new recommended optimal level for water fluoridation: 0.7 parts per million. 0.7 parts per MILLION!!! Do you realize how tiny that amount is?
In the 1940s, more than 15 percent of World War II recruits were denied the ability to enlist in the Army because they lacked six pairs of opposing teeth. The adult human mouth contains 32 teeth, and yet just 70 years ago a large number of 21- to 35-year-olds did not have even 12 good teeth. Much of the credit for the nation’s better oral health can be attributed to the decision in the 1940s to begin adding fluoride to public drinking water systems. According to the American Dental Association, fluoridation reduces tooth decay in all age groups by 20 percent to 40 percent “even in an era with widespread availability of fluoride from other sources, such as fluoride toothpaste.” People aren’t being “dumbed down” by the Federal government due to the addition of fluoride in the water. I have a friend who keeps saying this, yet when I ask for evidence to support it, all I am told is to look up the website of another conspiracy theorist where there is absolutely ZERO evidence to support this bizarre claim.
Then, there is the whole thing about gun control. When people quote Adolf Hitler about disarming citizens in order to conquer a nation, they seem to forget that no one has ever turned up a source documenting that Hitler literally said this (or something very similar). There is absolutely NO evidence ANYWHERE that the man made this statement. If you don’t believe me, I challenge you to find the original quote….via a RELIABLE source other than that by some conspiracy theorist.
The book Hitler’s Table Talk, 1941-1944: Secret Conversations records Hitler as having said the following sometime between February and September 1942:
If the term “conquering a nation” in the original quote is interpreted to mean that establishing and maintaining oneself as the autocratic head of a country (as Hitler did in Germany) requires disarming the civilian population, then the Table Talk passage cited above doesn’t really fit — Hitler was speaking of the need to disarm non-Aryans in the parts of Russia that had been occupied by German forces in the midst of a war, not of stripping all Germans of their guns. (And it’s unlikely that Hitler would have expressed such a concept in this context, as the 1938 German Weapons Act passed during Hitler’s rule actually loosened gun ownership rules for non-Jewish Germans.) If the term “conquering a nation” is interpreted to mean that the forcible overthrow and takeover of another country’s government and territory necessitates the disbanding and disarming of even local security forces, then the passage cited above might be considered as expressing the same idea as the original quote.
Gun control is not about taking guns away from American citizens. It is about using common sense and making it difficult for mentally ill people who are most likely to commit murder, to actually procure weapons. It is about reducing sales of high powered, automatic weapons that were CREATED to DESTROY HUMAN LIVES and that have absolutely nothing to do with what the founding fathers termed “a right to bear arms” when all they had were muskets that took 5 minutes or so to load with a single bullet. People who twist the words of the Constitution to include all of these new, unnecessary super weapons are negating the point of what the original framers meant. I call it cherry picking.
Many of my conservative/libertarian friends still eat red processed meats, drink copious amounts of alcohol and suck on cancer sticks all day long, but will raise issues about how the Federal Government is conspiring against them with this or that “right” being violated. If they are so concerned about these alleged claims about the government, whey are they doing things to themselves that will, no doubt, end their lives, because valid and reliable scientific evidence supports the harm that these activities cause? It makes no sense, whatsoever.
The point of all this is that when a conspiracy theorist is cornered and asked to provide substantiation (other than hearsay from other conspiracy theorists), or to present an isolated study that falls apart when tested for reliability and validity, they will try to dodge the question. Some get mad. Some claim that they’re not here to do my research for me. (I love that one….) but, across the board, NONE of them EVER produce scientific substantiation with validity and reliability, that supports a word they say. Not one. SO many ignorant people quote science to try to make points when they are not even aware of what the different types of validity and reliability are! Many didn’t even make it through college and have no idea how to actually read and interpret research. For instance….reliability. Here are the different kinds:
- Test-retest reliability is a measure of reliability obtained by administering the same test twice over a period of time to a group of individuals. The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for stability over time.
Example: A test designed to assess student learning in psychology could be given to a group of students twice, with the second administration perhaps coming a week after the first. The obtained correlation coefficient would indicate the stability of the scores.
- Parallel forms reliability is a measure of reliability obtained by administering different versions of an assessment tool (both versions must contain items that probe the same construct, skill, knowledge base, etc.) to the same group of individuals. The scores from the two versions can then be correlated in order to evaluate the consistency of results across alternate versions.
Example: If you wanted to evaluate the reliability of a critical thinking assessment, you might create a large set of items that all pertain to critical thinking and then randomly split the questions up into two sets, which would represent the parallel forms.
- Inter-rater reliability is a measure of reliability used to assess the degree to which different judges or raters agree in their assessment decisions. Inter-rater reliability is useful because human observers will not necessarily interpret answers the same way; raters may disagree as to how well certain responses or material demonstrate knowledge of the construct or skill being assessed.
Example: Inter-rater reliability might be employed when different judges are evaluating the degree to which art portfolios meet certain standards. Inter-rater reliability is especially useful when judgments can be considered relatively subjective. Thus, the use of this type of reliability would probably be more likely when evaluating artwork as opposed to math problems.
- Internal consistency reliability is a measure of reliability used to evaluate the degree to which different test items that probe the same construct produce similar results.
- Average inter-item correlation is a subtype of internal consistency reliability. It is obtained by taking all of the items on a test that probe the same construct (e.g., reading comprehension), determining the correlation coefficient for each pair of items, and finally taking the average of all of these correlation coefficients. This final step yields the average inter-item correlation.
- Split-half reliability is another subtype of internal consistency reliability. The process of obtaining split-half reliability is begun by “splitting in half” all items of a test that are intended to probe the same area of knowledge (e.g., World War II) in order to form two “sets” of items. The entire test is administered to a group of individuals, the total score for each “set” is computed, and finally the split-half reliability is obtained by determining the correlation between the two total “set” scores.
Then, there is the issue of validity. Validity refers to how well a test measures what it is purported to measure. You have to have validity in research or the research cannot be supported as true. While reliability, as discussed above, is necessary, it alone is not sufficient. For a test to be reliable, it also needs to be valid. For example, if your scale is off by 5 lbs, it reads your weight every day with an excess of 5lbs. The scale is reliable because it consistently reports the same weight every day, but it is not valid because it adds 5lbs to your true weight. It is not a valid measure of your weight.
Here are the different types of validity:
1. Face Validity ascertains that the measure appears to be assessing the intended construct under study. The stakeholders can easily assess face validity. Although this is not a very “scientific” type of validity, it may be an essential component in enlisting motivation of stakeholders. If the stakeholders do not believe the measure is an accurate assessment of the ability, they may become disengaged with the task.
Example: If a measure of art appreciation is created all of the items should be related to the different components and types of art. If the questions are regarding historical time periods, with no reference to any artistic movement, stakeholders may not be motivated to give their best effort or invest in this measure because they do not believe it is a true assessment of art appreciation.
- Construct Validity is used to ensure that the measure is actually measure what it is intended to measure (i.e. the construct), and not other variables. Using a panel of “experts” familiar with the construct is a way in which this type of validity can be assessed. The experts can examine the items and decide what that specific item is intended to measure. Students can be involved in this process to obtain their feedback.
Example: A women’s studies program may design a cumulative assessment of learning throughout the major. The questions are written with complicated wording and phrasing. This can cause the test inadvertently becoming a test of reading comprehension, rather than a test of women’s studies. It is important that the measure is actually assessing the intended construct, rather than an extraneous factor.
- Criterion-Related Validity is used to predict future or current performance – it correlates test results with another criterion of interest.
Example: If a physics program designed a measure to assess cumulative student learning throughout the major. The new measure could be correlated with a standardized measure of ability in this discipline, such as an ETS field test or the GRE subject test. The higher the correlation between the established measure and new measure, the more faith stakeholders can have in the new assessment tool.
4. Formative Validity when applied to outcomes assessment it is used to assess how well a measure is able to provide information to help improve the program under study.
Example: When designing a rubric for history one could assess student’s knowledge across the discipline. If the measure can provide information that students are lacking knowledge in a certain area, for instance the Civil Rights Movement, then that assessment tool is providing meaningful information that can be used to improve the course or program requirements.
- Sampling Validity (similar to content validity) ensures that the measure covers the broad range of areas within the concept under study. Not everything can be covered, so items need to be sampled from all of the domains. This may need to be completed using a panel of “experts” to ensure that the content area is adequately sampled. Additionally, a panel can help limit “expert” bias (i.e. a test reflecting what an individual personally feels are the most important or relevant areas).
Example: When designing an assessment of learning in the theatre department, it would not be sufficient to only cover issues related to acting. Other areas of theatre such as lighting, sound, functions of stage managers should all be included. The assessment should reflect the content area in its entirety.
What are some ways to improve validity?
- Make sure your goals and objectives are clearly defined and operationalized. Expectations of students should be written down.
- Match your assessment measure to your goals and objectives. Additionally, have the test reviewed by faculty at other schools to obtain feedback from an outside party who is less invested in the instrument.
- Get students involved; have the students look over the assessment for troublesome wording, or other difficulties.
- If possible, compare your measure with other measures, or data that may be available.
American Educational Research Association, American Psychological Association, &
National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: Authors.
Cozby, P.C. (2001). Measurement Concepts. Methods in Behavioral Research (7th ed.).
California: Mayfield Publishing Company.
I guess the crux of all this is that if you want to be taken seriously, be able to back up what you say in a way that can be supported by studies that actually measure the thing that is under question (validity) and that have repeatable results whenever a study is repeated more than once. THEN, I’ll listen to you.