Hosts of reports over the past few decades have pointed out the need for quantitative skills and conceptual mathematical foundations for undergraduates studying life science (American Association for the Advancement of Medicine, 2009; American Association for the Advancement of Science [AAAS], 2011; National Research Council [NRC], 2003; Steen, 2005). With the continuing growth of computational and data science approaches across the life sciences, these reports broadly agree that 21st-century biologists will be well-served through enhanced comprehension of the core quantitative concepts used throughout biology. Calculus provides one of the most fundamental mathematical frameworks that underlie science and is universally included as a core course for science, technology, engineering, and mathematics (STEM) students around the world. Calculus is a major component of quantitative training for biology undergraduates (Bressoud et al., 2013, 2015).
For decades calculus has been a required quantitative course for biology undergraduates, and biology students make up nearly 30% of all students taking Calculus 1 across all types of U.S. undergraduate institutions (Bressoud et al., 2013, 2015). The standard mechanism for teaching calculus in the United States has been through formal course sequences designed for a broad collection of science and engineering students. Historically, some institutions have either included life science students in these courses or have developed specialized courses for these students separate from and with somewhat different topic coverage than the standard science and engineering courses. Such specialized courses have sometimes been broadly inclusive of social science students as well, but some have focused explicitly on life science students, because they often make up a significant fraction of all STEM students at an institution. Over recent decades several biocalculus texts were developed that focus on standard calculus topics (Neuhauser, 2011; Adler, 2012; Schreiber et al., 2014) or take a somewhat broader perspective of quantitative topics to include linear algebra, probability, and discrete-time modeling (Bodine et al., 2014; Stewart and Day, 2015).
Our purpose is to explore the development and initial validity assessment of the BioCalculus Assessment (BCA), which aims to evaluate, in a comparative approach, undergraduate student understanding of calculus concepts embedded in the context of life science examples. Our objective is to develop a tool that can be effective in comparing alternative formats for student comprehension of concepts from calculus, particularly the alternative courses available to life science students at many U.S. institutions. Thus, the BCA has been developed explicitly to provide a means to assess the impact on calculus concept comprehension of different modalities of calculus instruction arising from different emphases and inclusion of concrete biological contexts. Options for students in this study include a standard science and engineering calculus sequence as well as a sequence designed specifically for life science students that emphasizes biology applications to enhance comprehension of calculus concepts. Three calculus concepts formed the basis of the BCA: rates of change, modeling, and interpretation of data and graphs.
It is our expectation that instruments such as the BCA and SRBCI can be applied to develop guidance regarding the impact of inclusion of life science disciplinary examples in calculus and statistical reasoning courses. Given the importance of quantitative methods across the life sciences, biology faculty may use the results of more expansive applications of the BCA to encourage their faculty colleagues who teach calculus to do so in a manner that is most effective for their students. This should also contribute to broader educational research questions regarding the impact of learning methods on student conceptual comprehension (Koedinger et al., 2013).
The validity of the test, that is, the degree to which evidence supports the interpretation of test scores (Nunnally, 1978; Crocker and Algina, 1986; Pedhazur and Schmelkin, 1991; American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, 2014), is an important aspect to examine when developing a test. Test validity is assessed through an accumulation of evidence that reinforces a test is measuring what it is intended to measure. The Standards for Educational and Psychological Testing indicate evidence such as content validity, response processes, and the internal structure of the test can be collectively used to support test validity (American Educational Research Association et al., 2014) and were used to frame our validation process for the BCA.
Content validity is an aspect of validity evidence that refers to the relevance, representativeness, and technical quality of items included on a test (Messick, 1995; American Educational Research Association et al., 2014). Evidence of content validity can be collected through systematic reviews by subject matter experts who give feedback on the adequacy of test items and the representation and relevance of the items to the domain (Reeves and Marbach-Ad, 2016).
We computed a content validity index (CVI) for each item from the review panel responses by counting the number of experts who rated the item a three or four on each rating criteria and dividing that number by the total number of experts reviewing the item. This, along with overall mean ratings for all three criteria were used to determine the most defensible items to include in the first iteration of the assessment. Items with a mean CVI less than 0.80 and an overall mean rating less than 3.3 were removed (Davis, 1992). This resulted in 35 test questions to consider for inclusion on the instrument. These 35 questions were then reevaluated by our research team, using comments and suggestions made by the expert reviewers to ensure representativeness of the concept, clarity of the item, and overall quality. After reevaluation, 23 of the highest-rated items were included in a first draft of the instrument.
We conducted two focus groups with students to ensure that the students interpreted test items as intended, ensure that the language and notation used on the test were familiar to students, and obtain feedback from students about question wording and distractor choices. Criteria for student participation in the focus groups included undergraduates who had declared a biological science major and who had either taken 1) the AP Calculus exam but who had not taken calculus at the university, 2) two semesters of calculus at the university level, or 3) two semesters of Mathematics for the Life Sciences (a calculus course for life science majors that teaches calculus concepts in biological context). These criteria ensured that the focus group students would have the relevant educational background in mathematics to understand the concepts represented in the assessment. Email invitations to participate were sent to 463 prospective students, and a total of 19 students participated in one of two focus groups in Spring 2016 (10 and 9 students, respectively). All students had declared a biological sciences major, except one student who was from an environmental and soil science major. Nine of the students met the criteria of having AP Calculus exam credit, eight had taken two semesters of calculus at university, and two had taken two semesters of Mathematics for the Life Sciences. Ten of the focus group participants were female.
Owing to test fatigue, students gave minimal feedback regarding questions that appeared at the end of the instrument. A new version of the instrument with the edited questions in reverse order was piloted with 14 students. Using the same student criteria and recruitment strategies from the focus group stage, we recruited 14 students (of whom four had also participated in the focus group portion of the project) to participate in one of two pilot study groups (groups of six and eight students). All students in these pilot groups had declared a biological science major. Nine of the students met the criteria of having AP Math exam credit, two had taken two semesters of university calculus, and three had taken two semesters of Mathematics for the Life Sciences. Eight of the participants were female.
We provided paper copies of the instrument to the students, who were given the opportunity to provide written or oral comments about each question if they felt a question was confusing in any way. Members of the research team were available to answer any questions the students had. After these pilot tests, minor revisions were made to questions for clarity based on the feedback from the pilot study students. We used the resulting 22-question preliminary instrument from this phase of development in the evaluation phase of the study.
Faculty in Calculus 1 (C1), Calculus 2 (C2), and Mathematics for the Life Sciences (BioCalc) administered the BCA in class over three semesters to students enrolled in their classes. C1 covers topics in differential calculus with no integral calculus and with emphasis on rates of change, and C2 expands on this to cover topics in integral calculus and series. These courses meet for 4 hours each week. C1 and C2 use a classical science and engineering calculus approach, emphasizing symbolics, graphing, and hand calculation, having limited applications mostly to physics, and allowing the use of graphing calculators. BioCalc is the second course of a two-course sequence, the first of which provides an overview of discrete mathematical topics including linear algebra; descriptive statistics; and discrete probability with applications to population modeling, allometry, and population genetics. The second course in the sequence, BioCalc, covers both differential and integral calculus, using biological examples particularly drawn from population biology, including exponential and logistic growth, and some physiological examples, including photosynthesis and blood flow. A focus throughout this two-course sequence is interpretation of data, simple modeling, and the use of a computational software package, MATLAB, to expose students to applications and numerical illustration of the key concepts in a biological context. BioCalc meets for 3 hours a week, and the focus on modeling, data, and computational software are emphases that do not appear in C1 and C2.We chose these courses because students within each course should be exposed to most, if not all, topics included on the assessment, and the courses are representative of how calculus topics are often split across calculus sequences. Essentially every biology major at the University of Tennessee, Knoxville, takes either the C1 and C2 sequence or BioCalc and its predecessor course covering topics in discrete mathematics. The coverage of the BCA content within C1, C2, and BioCalc is indicated in Table 1 by major concepts and subcontent with corresponding BCA question numbers. Many questions incorporated more than one of the three major concepts.
795a8134c1