Background

iCollege is a Georgia State University digital learning management system widely used and relied upon by students and faculty. During this project, I was one of the UX researchers working on several iCollege-related projects to collect data that could provide essential insights to inform and guide decisions.

Goal

This research project sought to examine the student iCollege experience across crucial digital platforms to generate insightful and actionable data that will inform design decisions leading to improvements in the usability and functionality of iCollege.

Challenge

The primary challenge for this project involved defining the project scope, which we determined would focus on students’ experiences. Another significant challenge was utilizing appropriate research methods and strategies to facilitate open and honest communications and feedback throughout the research process to deliver critical and actionable insights.

Roles & Responsibilities

I worked on this project as a User Experience Researcher under the supervision of Victoria Patterson (Learning Experience Designer & UX Research Coordinator at CETLOE).
My major responsibilities on this project included:

• Defining the purpose, scope, and duration of the project.

• Developing a project proposal.

• Conducting/participating in strategy meetings.

• Developing/implementing research methods, surveys, and interviews.

• Developing consent forms for research participants.

• Conducting interviews with research participants.

• Transcribing and analyzing research data.

• Synthesizing research results and presentations, and reports.

Process

Brainstorm

We developed a project task flow to guide our actions as we brainstormed the best methods applicable to the research goals.

Recruitment

We recruited research participants from the CETLOE student advisory board (a group of undergraduate students maintained by the department for research purposes to provide critical insight and explore new opportunities).

Methodology

Primary research involved a survey and an interview session. The survey was administered through Qualtrics, after which an interview would be scheduled at the participant's convenience. The goal was to be able to map out and analyze the user's experiences with iCollege across the mobile app, website, and other related avenues.

Research Factors:

Additionally, we utilized some principal research factors to outline the research objectives, develop questions to guide the research methods, and narrow the focus on the project goals in awareness of the limited time frame and participant pool.

Survey:

We developed a list of survey questions, including major demographic questions and questions to elicit answers that provide an overview of the user’s experiences with iCollege in qualitative and quantitative terms.

Interviews:

After completing the survey, participants would be prompted to schedule an interview through Microsoft bookings.
The interview sessions were uniformly scheduled for 45 minutes each and were conducted and recorded virtually using Webex.
I developed and utilized a set of guiding questions for each interview (keeping in mind that each participant's response may lead to different follow-up questions and discussions). I conducted five interviews during this specific research phase.
The guiding interview questions are listed below:

Discovery

The survey provided critical but general insights, providing a broad overview of participants' experiences. In addition, the follow-up interviews I conducted provided a great opportunity to explore individual participants' experiences, frustrations, and feedback related to using iCollege as a digital learning management system.

Affinity Mapping

I could pull the insights generated from the primary research data into an affinity map to help synthesize and organize important parts of participants' responses and feedback about using iCollege.

User Personas

I created hypothetical user personas aggregated from the survey and user interviews to help us orient the research process within a human-centered framework that enables us to empathize with and understand participants through their own words and experiences.

Research Themes

Participant Characteristics

• All full-time students.

• 67% of participants are international students.

• 33% of participants are classified as in-state students.

iCollege Engagement

• All participants have used iCollege for a minimum of one year.

• The majority of participants use the Pulse mobile app.

• The majority of participants frequently access iCollege through a Personal Computer.

• A significant portion of Participants often uses their mobile browsers for iCollege.

Overall Experience

60% of survey participants say their experiences using iCollege have been satisfactory, but there is room for improvement.

Findings & Insights

• Participants use icollege frequently, multiple times, and for hours.

• Participants use various devices (phones, tablets, PC) for College.

• Device use varies according to task and expectations.

• Participants report that they usually access iCollege and complete their tasks effectively with very few difficulties.

• Personal computers and tablets (when applicable) are generally used for more intensive/extensive tasks such as assignments, submissions, quizzes, etc.

• Long-term experience with iCollege is a crucial factor in how participants evaluated & discussed their experiences.

User Frustrations

• Disappearing notifications when using the mobile Pulse app.

• Unclear website informational structure, directions, and links.

• Lack of an integrated calendar and notification system.

• iCollege’s incompatibility with some third-party content and websites.
  Complicated and confusing email feature.

Recommendations & Next Steps

• Improvements in design and structure for better user experience, thereby reducing frustrations.

• An increased degree of autonomy and access to customization & personalization features to facilitate student interaction and engagement with courses and site content.

• Improve third-party compatibility issues to reduce frustrations.

• Improve access to iCollege informational training & guide content to combat errors, confusion, and frustrations.

• Improvements to iCollege email feature for better usability.

Reflections

The iCollege Student User Experience project was carried out from September to November 2022, and it involved conducting several user interviews and collecting and analyzing survey responses. Reflecting on the project with the knowledge that I have gained from this research, there are a few areas that, if investigated further, could provide critical information about the student user experience with iCollege.
A project like this would benefit from more research focus on first-year or first-time student iCollege users as this will provide insights into pain points, areas of frustration, or satisfaction during the adjustment process to using iCollege.
Conducting unmoderated usability tests in addition to contextual inquiries would be extremely helpful in uncovering fresh areas of research data on users' experiences, feelings, and motivations.
Comprehensive, in-depth market research into similar products and competing technologies would have provided valuable insights into the functions and capabilities of digital learning management systems.
Lastly, I believe this project could benefit monumentally from a collaborative and participatory research approach involving other user and learning experience researchers, designers, and engineers working cross-functionally on essential features and components of iCollege, including design, information architecture, interaction, and accessibility features.