top of page

Ed-Tech Services Design

Product: Online bootcamps to help users learn programming skills through online synchronized learning.

Project: This is a 2-months project working as a Customer Experience (CX) designer to study customers needs and design customer experience.

Problem

Online education has increased access to educational resources by overcoming geographic and financial barriers. However, the average completion rate of Bittier online learning products is as low as 20%. It is not even a surprise since learning is painful as learning is somehow anti-natural that customers have to take the initiative consistently to complete the class. Therefore, supporting services are important. We want to design the supporting service by understanding the interface between customers and the service providers (e.g., teachers/class managers/TAs).

Background

PRODUCT NAME        Data Science Class

PRODUCT TYPE           A beta-testing class

TIME                                    11/11/2017 - 2/28/2018

USERS                                 11 full-time employees in internet companies

PRODUCT DESIGN   Students will complete 2 projects in 8 weeksIn each week, students should finish 3 video classes and 1 live stream class (practice class) to learn.

Research

The goal of the research is to understand the obstacles that hinder the mastery of knowledge and skills. The ultimate goal is to find out solutions of learning system design to ensure the learning outcome for the formal class.

 

* Research went through all 8 weeks and completed in the end of the class.

User Segmentation

user segmentation+score.png

(click to open it)

  • Goal: understand users' background to design the product

  • Method: surveyed all 11 learners with a questionnaire before the class

  • Result: Most learners are expertise in programming-related fields but still are beginners in data science.

  • Conclusion: Probability and statistics foundation should be taught slowly and patiently, while programming languages could be taught at a faster pace.

Interview

Group.png

(click to open it)

  • Goal: understand typical learners

  • Method: interviewed all 11 learners about their experience and expectation in the beginning, middle and the end of the class.

  • Result:

    • Learners are at beginner level.

    • Learners goal is to understand basic knowledge and skills.

  • Conclusion: We can make the product mastery-based and friendly to beginners.

Participation Analysis

Group 5.png

(click to open it)

  • Goal: since participation correlates with learning outcome, we investigated factors that influence the participation of practice class.

  • Method: analyze the participation of live practice class every week and interviewed learners about their participation.

  • Result: 

    • 1st turning point at week 3: The participation rate is going up after we redesign the product, making the class master-based (click to see method).

    • 2nd turning point at week 6: The participation rate dropped because of the holiday season.

  • Conclusion: The mastery-based redesign promotes participation.

User Satisfaction Analysis

Bitmap.png

(click to open it)

  • Goal: analyze the quality of each part of the product

  • Method: surveyed learners after each week about class teaching quality, smoothness of videos, usability of LMS, and the responsiveness of class manager.

  • Result:

    • the overall trend was going up.

    • the quality of the video class is the least satisfied.

  • Conclusion: we need to improve the quality of the video class teaching - making it clearer, direct and short-cut.

Challenges

1. How to integrate services with contents?

I generated this framework that summarized a big picture of online classes design. Then I will explain how mastery-based learning falls into this framework.

Group 5.png

The first goal, which is the foundation of the pyramid, is all about project management. For example, synchronized class involves a lot of detailed work such as onboarding learners, coordinating with teachers on schedule, uploading videos&learning materials, granting certificates, etc. Although most of them could be automized by programming, we still need to make sure everything runs on the right track. After achieving this goal, we will get an MVP.

 

Secondly, based on the MVP, the 2nd thing is to look at the content quality to make sure the class is delivered coherent, logical, and vividly, from a perspective of pedagogy. Usually learning theories which try to explain how learning happens such as Constructivism and Cognitivism helps instructors to design classes.

 

Lastly, the most important thing is to ensure students eventually master certain skills and understand concepts, which is often simply equal to learning assessment such as grading HW and exam. However, learning assessment can simply tell us how good their performance is rather than promote the performance. To promote the performance, a good way is to look at the motivation. Although Behaviorism is traditionally used to motivate learners, there are many psychological factors that have been proved important in enhancing motivation, such as self-regulation, growth mindset, goals, and social belonging.

 

 

2. What is user satisfaction in the context of online learning services?

User satisfaction is a user's response to the above all 3 things in the model. When designing a satisfaction survey or interviewing, I usually refer the model to make sure I mention everything - logistics quality, content quality, and learning supports quality.

 

 

3. How to improve the user experience by providing a mastery-based learning environment?

Mastery is hard, especially when learning happens remotely. Mastery-based learning is different from traditional content-rich learning where teachers provide contents regardless of how much learners can absorb.

Based on the 2nd and 3rd level of the model, we can make efforts on both cognitive and non-cognitive part to achieve the goal. Here is a real example.

  • Goal: learners master logistics regression in R programming

  • Design:

    • Cognitive part: The key is to repeat. Usually teachers will teach one time and leave learners to understand by themselves. However, in a mastery-based class, we would like to provide a chance as much as possible for learners to practice until they master. For example, in this class teacher provided 2 similar datasets and ask them to code and generate plots for logistic regression in the class. There will be a HW to enhance the coding memory. Although it is easy to do, the challenging part is to keep it short and simple. 

    • Motivational part: Different with eating or playing games, learning is somehow anti-natural that users need to make effort to get a good outcome. The process is always painful. Therefore, we need to carefully help them participating into the class and discussion. For example, we increased the forum participation by awarding students who introduced themselves, who shared books or articles, and who proposed questions and answers. To make sure the discussion is seen by teachers, we require that TA provide answers in 24 hours and teachers reply students as much as they can. By doing these, their participation is enhanced. We also interviewed to every students 3 times in a semester and to know about their goal setting, struggle and how we can help, which strengthened the interpersonal connection between learners and us.

gif-bounce-arrow.gif

Previous

LMS

mastery-based
bottom of page