The Illusion of Knowing in College: A Field Study of Students with a Teacher-Centered Educational Past

Rationale of the Present Field Study

The present study is based on the metacognitive model described above, according to which self-regulation of thought and action for the purpose of optimizing learning or testing is dependent on students’ ability to gather realistic information about the knowledge and skills they possess. Thus, to induce metacognitive awareness (Flavell, 1979) of what is known and what is not known, students are asked to predict their test and class performance (functional confidence) and report their degree of trust in the predictions made (subjective confidence) at four different times during the semester. They are also required to examine their predictions against the grades they actually receive and reflect on any discrepancies.

Students with a teacher-centered educational past offer an uncharted ground for testing the power of recurrent and explicit self-assessment on metacognitive awareness and performance as they habitually rely on the instructor for self-assessment. It is hypothesized that in such students self-assessment practice, relative to a condition without practice, will improve metacognitive awareness and thus estimates of final test performance made at the end of the semester. Based on the findings of Miller and Geraci (2011b), students with less than desirable performance are expected to benefit the most from self-assessment practice in terms of improved metacognitive awareness rather than actual performance. However, such students in KSA may also be those that most firmly rely on verbatim learning and on the assessment provided by the instructor (i.e., the expert). Thus, students who routinely treat the instructor as the principal source of appraisal may discount the feedback provided by explicit self-assessment, rendering self-assessment practice moot. Namely, a pedagogical tool, which has been proven to be effective in the Western world, may not be equally effective in the Middle East (Mahrous & Ahmed, 2010; Smith, 2006). Alternatively, its novelty may grab students’ attention, leading them to recognize the usefulness of the information it provides. As a new gadget whose utility is suddenly recognized due to compulsory use, self-assessment practice may induce a change not only in metacognitive awareness, but also in performance through new habit formation. Additionally, students may be more or less sensitive to an intervention intended to enhance metacognitive awareness depending on the adverse experiences they encountered in college. Consider that time constraints, poor academic background, and limited course-relevant experiences and skills have been reported to be key sources of withdrawal from courses (Hayward, 2003; Lee & Choi, 2011; Poling, 1979). Thus, the number of college classes students completed compared to those they attempted (i.e., class completion ratio) can be treated as an index of experiences in which performance did not match expectations. Studies have shown that, in addition to interventions that encourage or require students to use support services (Gordanier, Hauk, & Sankaran, 2018), simply reminding students of their standing in a course can improve performance and thus reduce course withdrawal rates (Chen & Okediji, 2014; Smith, White, Kuzyk, & Tierney, 2018). It is reasonable to expect responses to a technique that is intended to promote self-understanding, such as self-assessment practice, to vary with students’ class completion rate. Namely, we predict that smaller class completion ratios may be associated with students’ increased sensitivity to overt self-assessment practice, as such practice in class both encourages and normalizes the task of understanding one’s current situation.

If functional and subjective confidence ratings are considered local judgments, it is reasonable to ask whether they are informed by broader traits, such as general self-efficacy. General self-efficacy refers to students’ underlying confidence to perform well across diverse tasks and situations. It is considered a motivational trait (Chen, Gully, Whiteman, & Kilcullen, 2000) positively correlated with task completion rates (Eden, 1984, 1988; Pajares, 1996), persistence (Bandura, 1977; Pajares, 1997), motivation and engagement (Bandura, 1989; Bandura & Schunk, 1981). In the Western world, some evidence exists that it can predict functional confidence (Avhustiuk et al., 2018), but evidence of its ability to predict academic achievement is mixed (Pintrich, Smith, Garcia, & McKeachie, 1993; Wilhite, 1990; Zeegers, 2004). In students with a teacher-centered educational past, general self-efficacy may predict functional and subjective confidence ratings if such local judgments are the reflection of a motivation to change established learning habits. It may even predict performance if motivation has led to action. In addition, general self-efficacy may specifically modulate students’ responses to self-assessment practice so that students with varying confidence in their abilities are more or less sensitive to the practice of estimating grades.

Participants

The participants were 685 female college students at a University in the Eastern Province of KSA. Participants reported Arabic as their first language and English as their second language. For university admission, students had demonstrated English language competence through standardized English proficiency tests. Students completed primary and secondary schooling in KSA. They were all commuters from nearby towns or cities who were enrolled in a Core Curriculum class, which offered practice in basic academic skills (e.g., writing, speaking, reasoning, etc.) across a variety of topics or focused on a specific domain of knowledge, such as history, Arabic culture and religion, and psychology. If a student was enrolled in more than one of the selected Core Curriculum classes, withdrew, or missed the midterm or the final test, her data were not included in the current sample (12.51%). Weimer’s dimensions differentiating educational approaches (Weimer, 2002) were used to classify students as possessing a teacher-centered educational past and habits of information acquisition reliant on verbatim learning. Due to gender-segregation rules, a corresponding male sample was unattainable.

Materials and Procedure

Convenience sampling (i.e., consent of the instructor, and minimal or absent overlap of students) was used to select 32 classes. For each instructor, sections of the same class were randomly assigned to either a self-assessment practice condition (n = 337 students) or a control condition (n = 348 students) to ensure that instructors would have half of the classes assigned to practice and the other half to control. Instructors were either local hires or foreigners, but all possessed graduate degrees obtained outside KSA.

In Table 1, the practice condition involved 4 phases in which students estimated their class grade (phase 1 and phase 3) as well as their grade on the midterm (phase 2) and final exams (phase 4), before and after each test. As per institutional guidelines, all classes required a midterm test and a final test, which 1) included a fair mixture of short-answer and multiple-choice questions, and 2) assessed not only retention and understanding of key information, but also use of knowledge, such as application, analysis, and evaluation (see Bloom’s taxonomy; Bloom, 1976).

Phase 1 was initiated during the first 2 weeks of the semester. Students assigned to the practice condition were given a sheet on which they were to predict their class grade on a scale from 0 to 100, as well as to express their confidence in the prediction made on a scale from 0 (not at all confident) to 4 (extremely confident). All students, irrespective of the condition to which they were assigned, completed the New General Self-Efficacy (NGSE; Chen, Gully, & Eden, 2001) inventory. The NGSE required students to report the extent to which they agreed with each of eight statements of general confidence in one’s abilities on a scale from strongly disagree (1) to strongly agree (5).

Phase 2 of the practice condition was implemented during the administration of midterm tests. Before starting the midterm test and after having completed it, each student predicted her performance. For both prospective and retrospective estimates, the student expressed her confidence in the prediction made. Instructors discussed the content of the midterm test upon returning it to students to ensure adequate performance feedback. Phase 3 of the practice condition was administered during the last 2 weeks of the semester. Students were again asked to predict their class grade as well as express their subjective confidence in the prediction made. Estimates and confidence ratings of phases 1–3, which were made on separate sheets of paper, were returned to students within a week so that information was available for review before additional predictions were required. Students were encouraged to reflect on discrepancies between predictions and actual performance, and, where needed, to consider remedies either by themselves or with the aid of the instructor. They were informed that explicit and periodic assessment of their performance followed by self-reflection would offer them the opportunity to understand what they acquired and how they acquired it in reference to the demands posed either by a specific activity or by the collection of activities that a class placed on them. As predictions of class grades could be verified only at the end of the semester, students were encouraged to check the running average of all completed activities on their course management system to evaluate the extent to which their predictions matched their actual performance. Phase 4 was implemented during the administration of the final tests. Each student predicted her performance on the final test, as well as expressed her confidence in the prediction made before and after the test. To minimize the temptation of aspirational predictions, in all phases, students were reminded to report what they reasonably believed to be their grade at the time of the inquiry. They were explicitly discouraged to report what they wished their grade to be.

As the semester moved closer to its end, all students were expected to have increased their knowledge of the demands of the class in which they were enrolled (i.e., as assessed by differences between phase 1 and phase 3), and of the more specific demands of its tests (i.e., as assessed by differences between phase 2 and phase 4). Thus, the control condition, where students did not predict their grades in phases 1–3, served to offer a “business-as-usual” setting, against which the effects of self-assessment practice could be measured. As in the practice condition, however, instructors discussed the content of the midterm test upon returning it to students to ensure adequate performance feedback (see Mahrous & Ahmed, 2010). Furthermore, students assigned to the control condition predicted their grades and reported their confidence in their predictions during phase 4 so that functional and subjective confidence estimates expressed without any practice could be compared with those made in the condition where estimates were a recursive exercise.

Predictions of Final Test Grades: Students’ Accuracy of Estimation and Confidence

At the end of the semester, predictions made by students who practiced self-assessment were expected to be more accurate and confident than those of students assigned to the control condition. Performance might follow suit if self-assessment practice affected students’ study activities. However, because knowledge of a test increased after having taken it, all students were expected to be more accurate and confident in their predictions after having completed the final test. To test these predictions, a 2 timing of prediction (prospective and retrospective) X 2 condition (control and practice) mixed factorial ANOVA was conducted on functional confidence (i.e., accuracy) and subjective confidence of estimation of final test grades. Descriptive statistics are displayed in Table 1. Retrospective estimations were more accurate than prospective estimations, F(1, 683) = 33.64, MSE = 16.36, p < .001, ${\mathrm{\eta }}_{\mathrm{p}}^2$ = .047. As illustrated in Figure 1, estimations in the practice condition were superior to those of the control condition, F(1, 683) = 7.04, MSE = 304.45, p = .008, ${\mathrm{\eta }}_{\mathrm{p}}^2$ = .010 (interaction: F < 1, ns). Subjective confidence in estimated values, however, was not sensitive to either condition or timing of prediction (Fs ≤ 2.34, p ≥ .127).

A 2 condition (control and practice) X 2 test (midterm and final exam) mixed factorial ANOVA on students’ grades indicated that not only prediction accuracy, but also overall performance was higher in the practice than in the control condition, F(1, 683) = 6.65, MSE = 348.96, p = .001, ${\mathrm{\eta }}_{\mathrm{p}}^2$ = .010. No significant differences between tests were observed, F(1, 683) = 3.01, p = .069. Yet, a significant interaction suggested a more complex pattern, F(1, 683) = 8.81, MSE = 98.24, p = .003, η_p² = .013 (see Figure 2). Tests of simple effects indicated that students’ final test performance was higher in the practice than in the control condition, t(683) = 3.87, p < .001. If predictions of final grades reflected the impact of self-assessment practice on students’ test performance, the absence of a difference between conditions in midterm test grades (t < 1, ns) indicated that explicit self-assessment required time before it could be beneficial to performance.

Predictions of Class Grades: Students’ Accuracy of Estimation and Confidence

Predictions of class grades (see Table 1) were considered more complex than those involving test grades as they would require students to combine estimates of the outcomes of an array of activities carried out during an entire semester. Nevertheless, as students’ knowledge of class demands improved from the beginning (phase 1) to the end of the semester (phase 3), predictions in the practice condition were expected to be more accurate and confident. A one-way ANOVA indicated that accuracy did not change, F(1, 336) = 2.43, p = .120, from phase 1 to phase 3. However, subjective confidence declined, F(1, 336) = 42.15, MSE = .63, p < .001, ${\mathrm{\eta }}_{\mathrm{p}}^2$ = .111, suggesting that increasing knowledge of class demands made students less likely to trust their estimates. Consistent with the null finding regarding prediction accuracy, a one-way ANOVA on class grades indicated that performance in the practice and control condition did not differ, F(1, 683) = 1.49, p = .223.

In our study, class grades not only combined performance evaluations from multiple sources (e.g., tests and assignments), but also received less clearly targeted feedback (Mahrous & Ahmed, 2010), thereby being more difficult to estimate than final test grades. According to students, predictions of final test grades could benefit from the objective feedback obtained from the midterm, whereas predictions of class grades on phase 3 had to rely in part on the running average of all completed activities on the course management system. Running averages were considered by students less informative than the feedback gathered from estimating midterm performance and completing the midterm. The nebulous feedback of class grades appeared only to have made students more cautious. Indeed, as self-assessment practice increased from phase 1 to phase 3, it had no visible impact on the accuracy of estimation of class grades or on actual grades, but it promoted less confidence in the estimates made.

Prediction Accuracy and Subjective Confidence as a Function of Performance Level and Condition

Regression analyses were conducted to assess whether high and low performing students’ functional and subjective confidence would be differentially sensitive to self-assessment practice (see Table 2). In these analyses, condition (control and practice), performance (final test grades) and the interaction term served as predictors. Performance, which was treated as a continuous variable, was centered on its mean value (Field, 2009). Retrospective ratings of final test grades and related confidence served as the outcome variables because they provided a stringent test of the impact of self-assessment practice above and beyond the knowledge of class and test demands that would be normally acquired by students at the end of the semester.

Performance contributed to the accuracy of retrospective predictions (see top half of Table 2), indicating that predictions changed from liberal (i.e., overestimation) to conservative (i.e., underestimation) as performance increased. The significant interaction, however, revealed that estimation accuracy benefited from practice selectively, depending on the students’ performance level. The same regression analysis performed on students’ subjective confidence regarding their retrospective predictions uncovered a significant independent contribution of condition and performance (see bottom half of Table 2), indicating that self-assessment practice made students less confident in their predictions and that confidence decreased as performance declined. Yet, the significant interaction revealed that confidence declines as a function of assessment practice were dependent upon students’ performance level. Regression analyses pertaining to prospective predictions and related subjective confidence are reported in the Appendix 1. They showed a similar pattern for prospective estimates, and an attenuated pattern for subjective confidence, whereby confidence and performance merely decreased together.

The contribution of performance to both outcome variables illustrated a phenomenon also reported by Miller and Geraci (2011). Namely, as performance declined, retrospective estimates became more inflated, but less confident. To better understand the patterns yielded by the interaction of condition and performance, we organized students into four performance levels based on their final test grades: (highest: 100–90%; high: 89–80%; low: 79–67%; and lowest: below 67%, which corresponded to failing the test). Table 3, which reports descriptive statistics, uncovers several patterns involving the selective impact of self-assessment practice. The practice exercise benefited the most the accuracy of retrospective estimates of students with barely passing (i.e., low) performance, whereas it reduced the subjective confidence in the estimates made as performance increased.

Prediction Accuracy, Subjective Confidence, and Performance as a Function of Self-Efficacy, Completion Rates, and Condition

Self-assessment of test performance, including functional and subjective confidence, might be the expression of students’ broader sense of trust in their abilities (as indexed by general self-efficacy), and failed expectations (as indexed by class completion rates). Regression analyses were conducted to determine the contribution of these factors, in addition to self-assessment practice, to functional and subjective confidence and final test grades. As discussed earlier, retrospective ratings of final test grades and related confidence served as the outcome variables because they provided a stringent test of the impact of self-assessment practice above and beyond the knowledge of class and test demands that students would normally acquire in a semester. In these analyses, condition (control and practice), general self-efficacy, class completion rates and two key interaction terms (condition X general self-efficacy, and condition X class-completion rates) served as predictors. Interaction terms were selected in recognition of the fact that the impact of self-assessment practice might vary depending on students’ class completion rates or self-efficacy. Class completion rates and general self-efficacy, which were treated as continuous variables, were centered on their respective mean values (Field, 2009).

The results of these regression analyses are displayed in Table 4. The only significant contributor to functional and subjective confidence ratings was self-assessment practice. However, both self-assessment practice and general self-efficacy (NGSE) contributed to students’ final test performance. Specifically, students’ self-efficacy increased with their test performance, suggesting that a broader sense of confidence in one’s ability may promote a more efficient execution of test requirements. Yet, it did not predict local judgments, such as functional and subjective confidence of final test grades, suggesting that such judgments may be considered the expression of a motivational state rather than of a motivational trait.

Qualitative Evidence

Informal focus groups (as part of debriefing sessions) and observations were conducted at the end of the semester with students and instructors regarding the impact of self-assessment practice. Given the hectic nature of end-of-the-semester activities, convenience sampling of students and instructors was utilized. To ensure candor and respect anonymity, individual comments were recorded without a link to the self-assessment data collected in class. These inquiries led to the following additional qualitative findings: 1) Practice helped students keep attention focused on discrepancies between predicted grades and actual grades. It also made them acutely aware of the distinction between desired grades and grades that could be predicted if exerted effort, adopted study strategies, and demands of tests or class are carefully and objectively considered. 2) Attention to self-assessment helped some students feel more in control of their learning and led them to seek a better understanding of how their study efforts and strategies to prepare for tests could be related to their performance. (3) Noticeably diminished were instructors’ reports of grade groveling inside and outside the classroom, grade appeals, as well as reports of adversarial interactions between instructors and students regarding performance. Of course, given our reliance on convenience sampling, and the absence of a link between individual comments and the data collected in class, these opinions and observations are informative, but of limited impact and in need of additional testing.