Glossary (EDS 113)
Basic Terminologies & Concepts
Reliability – Validity
Reliability
Reliability refers to the consistency of an assessment. An assessment is considered to be reliable if the measure of students’ achievement yields the same result regardless of time, place or even if assessment is done repeatedly.
Example: A group of kindergarten students are assessed to name parts of a human body. A reliable assessment yields a consistent outcome even if test items require the students to label, match or draw the parts of a human body. Regardless of time it is taken, place it is taken or despite repeated trials too, variation of result based from a given standard gives a measure of its reliability.
Validity
Validity refers to accuracy of an assessment by which it measures what it is designed to measure. An assessment is considered to be valid if its content is aligned with the instructional objectives. Validity can also be measured through its criterion and construct by which the assessment is concurrent with external criterion; and outcome can be useful to predict future performances.
Example: Taking a final exam in math – content of assessment should reflect coverage of items requiring mathematical ability and not reading comprehension. And if criterion of test highly correlates with national standardized test or other external (related) test. Likewise, if test results accurately estimates a student’s current and future ability related to the criterion of the test.
Reliability and validity are two important characteristics of a strong assessment. Both reliability and validity require distinct evidences. Reliability is determined by consistency of rate or test-retest reliability. Meanwhile, validity evidence is determined by content, construct and criterion.
Formative Assessment-Summative Assessment
Formative Assessment
Formative assessment is an ongoing assessment used to provide information during instructional process. It is utilized for feedback while learning is taking place. This is important to students because it measures their learning progress throughout the course. This is important for teachers too because it gives them feedback on areas that need improvement in terms of effectiveness of instruction and activities. Formative assessment can be in the form of homework, formal/informal question & answer sessions, journals and observation during class activities.
Summative assessment
Summative assessment can refer to testing the overall learning a student has achieved in a period time. Its result reflects teaching effectiveness and sums up learning outcome. Summative assessments can range from quizzes, unit-test, minor and major exams.
Summative assessment measures general learning outcome and performance while formative assessment drives “learning in the making”.
Criterion-referenced Test – Norm-referenced Test
Criterion-reference test
“Criterion-reference test is a test that provides a basis for determining level of knowledge and skills in relation to a well-defined domain of content” (Hambleton, 2006). Criterion-referenced test is designed to primarily measure an examinee’s content knowledge and skills regardless if items are to be used to discriminate outcome among other students.
For instance, taking a Licensure Exam for Teachers (LET) where content of test determines the examinee’s ability; whether or not the examinee has the skills to qualify to be a licensed teacher or not (pass or fail).
Norm-referenced test
“Norm-referenced test compare an examinee’s performance to that of other examinees” (Danielle, 2008). It is designed to determine measure of individual relative performance with respect to the performance of others. Here, student A’s test score is interpreted against test scores of others.
An example could be, when new students are to take admissions test. Upon enrollment, they will be assigned to certain sections depending on their average score. Students who got exceptionally high scores can enjoy the benefit to be in a science class.
A norm-referenced test focuses primarily with test outcomes, whether or not an examinee answered more questions correctly compared to other examinees. Meanwhile, a criterion-referenced test requires an examinee to demonstrate his/her ability level. Although it uses a cut score, it serves as a determinant whether or not the student has absorbed the content knowledge or has mastered the skill being taught.
Formal– Informal Assessments
Formal Assessment
Formal Assessment is a conventional way of testing. It is generally intended to assess overall achievement, compare a student’s performance to their peers, or find a student’s strengths and weaknesses. (Weaver, 2006). Formal assessments are normally used to evaluate how well the students learned the concepts and skills. Examples of formal assessments include quizzes, major exams, and standardized tests.
Informal Assessment
Informal assessment is like criterion-referenced test by which assessments are not data driven but rather content and performance driven (Weaver, 2006). This particular assessment monitors students’ progress casually. Examples of informal assessments can be question & answer session, discussion, observation, participation and portfolio assessment.
Formal assessments are standardized measures; it contains data which can be equated as a grade. Meanwhile an informal assessment casually monitors students’ progress. Using both formal and informal assessments is ideal to be used in a class. I think using both is balanced and it measures overall learning achievement of students.
Traditional – Alternative Assessments
Traditional Assessment
The most widely used traditional assessment tools are multiple-choice tests, true/false tests, short answers, and essays (Dikli, 2003). These types of assessments are objective; it requires one answer and is easy to mark.
Alternative Assessments
On the contrary, alternative assessments are performance based. It requires the students to demonstrate what they have learned. It can be measured by open-ended questions, portfolios, simulations, journals and projects (Dikli, 2003).
If instructional method largely involves lectures, teacher-centered discussion which requires students to listen, read and memorize, then traditional assessment could be appropriate for this class. However, if teaching strategy involves active participation by the students, then alternative form of assessment may be appropriate for this class. However, I think using both tools is appropriate – to objectively assess students’ learning.
Figure 1:
- Formal-Summative Assessments such as examination, thesis, assignment, research report, oral presentation and portfolio.
Common characteristics of these types of assessments are as follows:
- assigns grade equivalence for student performance and ability
- uses a systematic way to measure students’ progress
- uses to measure overall achievement of the student
- outcome can be used to compare a student with other groups
2. Formal-Formative Assessments such as web posting, assignment framework, research plan, tutorial tasks, test, laboratory task.
Common characteristics of these types of assessments are as follows:
- uses data for scoring
- can be norm-referenced and criterion referenced
- done during the instructional process of the course
3. Summative-Informal Assessments such as oral presentation, interview, web posting, on-line task and field trip.
Common characteristics of these types of assessments are as follows:
- measures core concept the student has achieved
- requires active participation among students
- content and performance driven
4. Informal-Formative assessments such as interview, email exchange, research plan development, tutorial discussion and moodle posting.
Common characteristics of these types of assessments are as follows:
- monitors student progress during the course
- can casually be done
- engages students to actively input in the class
- can be used for feedback regarding teachers’ instructional method
Figure 2: Reliability-Validity
1. Reliable, not valid (upper-left target) – In this case, the content of assessment might be consistent but it does not assess what it is suppose to assess. Example: Hiring an IT specialist. If assessment is based on the applicants’ ability to surf the net, use Facebook and yahoo messenger, it does not accurately predict his computer proficiency which is suited to become an IT specialist.
2. Not reliable, valid (lower-right target) – Here, content of assessment may have required a student to demonstrate acquired skill; however, the determinants of assessment may not be focused on core concepts. Example: Research. This assessment tool can be valid but it does not guarantee reliability. Even if the objective of research is directly related to the learning goals, however, research outcome may not be consistent of student’s acquired knowledge/ learning. Variables of truthfulness about the research too may affect consistency of outcome.
3. Not valid, not reliable (lower left target) – This case may be the worse of all assessment outcome. If content of assessment are not focused to the learning goals and if assessment tool is inconsistent, then, there is nowhere to go. An example could be, if a teacher wants to assess a student’s reading comprehension. If assessment requires a student to demonstrate mathematical ability such as subtraction, addition, multiplication, division, then the assessment is out of context and unreliable.
4. Reliable, valid (upper right target) – This is the ultimate goal a teacher would want to attain in constructing an assessment. It should be both reliable and valid – bulls eye! Here, the content of assessment is consistent and it strongly aligns with the instructional goals of the course. Standardized test are the most common examples we can use for this figure.
References:
Atherton J S (2011) Teaching and Learning; Assessment [On-line: UK] Retrieved January 15, 2013 from http://www.learningandteaching.info/teaching/assessment.htm
Hambleton, R. (2006). Criterion-referenced tests. Retrieved January 16, 2013 from http://www.education.com/reference/article/criterion-referenced-tests1/
Danielle. (2008, May 22). norm-referenced vs. criterion-referenced testing. Retrieved January 16, 2013 from http://www.altalang.com/beyond-words/2008/05/22/norm-referenced-vs-criterion-referenced-language-tests/
Weaver, B (2006). Formal versus Informal Assessments. Retrieved January 16, 2013 from Scholastic Web site: http://content.scholastic.com/browse/article.jsp?id=4452
Dikli, S. (2003). Assessment at a distance: Traditional vs. alternative assessments. Retrieved January 16, 2013 from http://www.tojet.net/articles/v2i3/232.
maezarsanchez 