Engagement of introductory biostatistics students in a novel hybrid course format

Course format

The master's level introductory biostatistics course was offered in a 12-week semester with classes, two exams, and a final project poster presentation, and is a required course for students pursuing a master of public health (MPH). Each class involved one hour of lecture and two hours of laboratory instruction. In-person attendance was mandatory for the first class, exams, and final project poster presentation; for all other sessions students were given the choice of attendance in-person, online during class (synchronous), or online after class ended (asynchronous). The synchronous offering was provided using Google’s “Hangouts on Air,” with a live screen share and audio broadcast viewable through YouTube®. The “Q&A Module” allowed viewers to type questions that appeared on the instructor’s screen during lecture. These questions were answered verbally as were questions raised by in-person attendees, and became a clickable index of the lecture. Recordings became available immediately afterwards on the course’s YouTube channel (https://www.youtube.com/user/ph750spring2014).

Technology required

This format required that the computer used to present material in class be connected to internet and have a working microphone. In the absence of a dual-screen setup, the “Q&A” module was shown side-by-side with a Microsoft PowerPoint® window used to show lecture slides. A “Lavalier” microphone was used to improve sound quality, but an ordinary webcam microphone would suffice.

Course survey

A total of 47 introductory biostatistics students were given the opportunity to participate in a survey at the end of the semester. An additional 58 introductory epidemiology students whose course followed a more typical hybrid format of alternating in-person and online sessions were also surveyed for comparison. This study was reviewed by the Hunter College (City University of New York) Institutional Review Board (590445-1) and determined exempt according to federal regulations, under 45 CFR 46.101(b). Informed electronic consent was obtained as part of an anonymous online questionnaire. Study participants were given the option and incentive of entering a draw for a $50 Amazon gift certificate upon completion of the questionnaire.

The end of course survey included demographic characteristics such as age, gender, race/ethnicity and travel time to campus. Participants were asked to choose which format was their preferred way to attend lectures. They were subsequently asked an open-ended question, “Why did you prefer this format?” Responses that pertained directly to the question asked were analyzed and assigned primary and secondary topics based on recurring and prevalent themes within the total set of responses. The keyword topics used to categorize responses were as follows: 1) interactivity, 2) convenience, 3) avoiding commute and 4) learning preference. The complete survey and summarized responses are provided in the Supplementary material and Data availability sections, respectively.

Data were collected via the Google Forms® encrypted website and is noted under the Data availability section.

YouTube data collection

Viewership among other metrics are automatically documented by the YouTube® website and can be accessed using the analytics dashboard. The dashboard can be found under “analytics” in the “creator studio” section of the Hunter College PH750 Spring 2014 YouTube® account. Data within the time frame of February 1, 2014 to March 4, 2014 were extracted from this dashboard and saved as a comma separated values file. See the Data availability section.

Statistical power

Statistical power was estimated using the pwr.2p2n.test function from the R pwr package. For a sample size of N=26 and a two-proportion test, we estimated 90% power to detect an effect size of between 1.3 and 1.6 for balanced groups and for groups with a 4:1 prevalence ratio respectively.

Data analysis

The survey data were downloaded from the Google Forms® encrypted website and is noted under the Data availability section. Bivariable relationships were only investigated among biostatistics students due to a low response rate from the introductory epidemiology course (19% or 11/58 of these students responded to the survey). Two respondents who were registered for both courses were considered as biostatistics students.

Associations between continuous variables like age and travel time to class and course format preference were analyzed by non-parametric Kruskal-Wallis analysis of variance. Associations between categorical variables and course format preference were assessed by Fisher’s Exact Test. YouTube longitudinal viewing data were visualized as stacked line plots.

Data cleaning and all analyses and plotting were performed using R version 3.1.1 (R Core Team, 2014). Code and data to reproduce all results in this manuscript are provided at https://github.com/LiNk-NY/H-HybridCE.

Demographics

A total of 26 introductory biostatistics students out of 47 or 55% participated in the anonymous survey. Demographic characteristics of the biostatistics students are shown in Table 1. The majority of these respondents were female (72%), with an average age of 33 years (SD = 8). The median age of the sample was 30 years with a range between 22 and 55 years of age. Of those who answered the survey, 46% of students identified as Non-Hispanic white, 27% as Non-Hispanic black and 15% as Hispanic. The average travel time to the Hunter College Silberman School of Social Work building was 97 minutes (SD = 63 minutes). The median commute time was 90 minutes with a range from 2 to 240 minutes.

Course format preferences

Table 1 also shows self-reported course format preferences and preference reasons. Forty-six percent of respondents preferred the in-person format; thirty-five percent preferred the synchronous course format, and nineteen percent preferred the asynchronous online format. Students were asked, “If you were to take this class again next semester, which format would you choose?” The majority of students (69%) would prefer having the option of both for each class, in-person or online, as was done in this class. “Learning preference” was the most endorsed reason with 10 mentions (making up 38% of respondents), followed by “interactivity” with 9 mentions (35%), “convenience” with 7 (27%), and “avoiding commute” with 6 or 23% of biostatistics students.

Analysis of qualitative survey questions

A qualitative analysis of the long response items in the survey extracted primary and secondary reasons for course preferences. Primary and secondary reasons were determined according to order of appearance. Secondary reasons were only documented for those with more than one reason in the same long response item. These preferences were categorized into four salient topics: 1) interactivity, 2) convenience, 3) avoiding commute, and 4) learning preference. Interactivity was defined as the advantage to be able to ask questions and interact with either the professor or classmates during in-person or online lectures. For example, responses similar to “I prefer class in-person because it allows for interaction with classmates and for me to ask questions in real time” were coded as “interactivity”. The “convenience” topic was determined in any comments where students indicated learning with less external effort or difficulty. Comments similar to, “I liked to be able to pause the lectures and rewind/repeat statements to better understand difficult topics,” were categorized as “convenience” reasons. The “avoiding commute” topic was frequent enough to warrant its own category. Responses that indicated saving time, for example, “I did not have to commute to Harlem,” were coded as “avoiding commute”. Learning preference is indicated when students’ comments explain cognitive advantages to their course preferences.

Self-reported reasons for course format preference

Table 2 shows relationships between reported course format preference and a number of demographic characteristics along with primary preference reasons. Only the given qualitative reasons for preference were found to be significantly related to course format preference (p <.001, Fisher’s Exact Test). Choice of traditional in-person course format was strongly associated with preference for “interactivity” (58% of those who prefer the in-person format) and with ingrained “learning preference” (50% of those who prefer the in-person format). Conversely, among those who favored the synchronous online course format, “avoiding the commute” was the more frequent reason for such choice. Those who preferred the asynchronous online course format most commonly reported “convenience” as the reason for their choice followed by “learning preference”.

Figure 1 shows both primary and secondary reasons provided in qualitative survey responses, by course format preference. Those who preferred the in-person course format endorsed “interactivity” and “learning preference” more often than those who preferred either the synchronous or the asynchronous formats. Those who preferred the asynchronous format tended to mention “convenience” and “learning preference” as reasons for such choice. Among those who preferred the synchronous format, avoiding the commute was the dominant reason for format preference.

Figure 1. Course format preference reasons by preferred format.

Combined primary and secondary self-reported reasons for course format preference were extracted from qualitative topic analysis of an open-ended survey item. Among students who preferred the in-person course format, interactivity was mentioned 7 times in response to the open-ended question, “Why did you prefer this format?” Those who preferred the asynchronous online format mentioned its convenience 5 times in response to the open-ended question. Students who preferred the synchronous online format were likely to mention avoiding the commute as a reason for such preference.

Other factors associated with course format preference

Association between life circumstances and course format preference was nearly significant, with students responsible for the care of children and students working full-time more likely than others to prefer asynchronous viewing (p=.09 and p=.08 respectively, Fisher’s Exact Test). Age, travel time to campus, gender, race-ethnicity, having a STEM undergraduate degree (Science, Technology, Engineering, Mathematics), having previously taken a hybrid-format course, and self-reported confidence level with statistics were not associated with course format preference.

Synchronous/asynchronous viewing patterns of the lectures

YouTube view frequency patterns of lecture videos were recorded and visualized as stacked line plots. Figure 2 shows the number of views per online video lecture up until the first exam, where the reference line indicates the date of the first exam. The viewing patterns show a high number of views on the date the lecture video is uploaded and an increased number of views for all lecture videos right before the occurrence of the exam. The small increase in viewing between two holidays, Lincoln’s birthday (February 12) and President’s day (February 17) relative to other non-class days shows that some students used those days off to watch lecture videos. This viewing pattern was consistent during the rest of the course and before the second exam.

Figure 2. YouTube lecture viewing patterns.

Shaded peaks reflect increased live-online lecture viewership. Stacked areas show cumulative views per lecture video. An increase in online views for all lectures is observed just before the first exam on March 3rd (indicated by a dashed reference line). Smaller stacked shaded areas reflect asynchronous online viewership of lecture videos with a marked increase in views just before the exam. This pattern of most online attendance occurring during the lecture, followed by low but steady rates of asynchronous viewing and a viewing peak immediately before exam was observed among the remaining video lectures and before the second exam.