The xAPI Challenges App

The xAPI Challenges App provides practice opportunities for people learning how to collect specific user data from their Articulate Storyline eLearning offerings.

• Responsibilities: Instructional Design, eLearning Development, App Development, xAPI Implementation, Evaluation 
• Audience: Instructional designers and eLearning developers that use Articulate Storyline
• Tools Used: xAPI, Glide Apps, Storyline 360, Veracity LRS, Watershed LRS, Zapier, Drip
• Year: 2020

Overview

I designed and developed the xAPI Challenges App to help instructional designers and eLearning developers learn more about the Experience API (xAPI).

My xAPI + Articulate Storyline tutorials were already helping many people, but it was difficult gauging how effective they were. People reached out semi-regularly with technical questions and gratitude. However, I wondered what the experience was like for everyone who did not reach out.

So, I decided to create the xAPI Challenges App as a way to provide practice opportunities for each of the tutorials. The app would serve two purposes:

1. It would give the learners a chance to test their understanding, receive feedback, and learn from their mistakes, and
2. It would give me data about where people struggle, thereby enabling me to improve the tutorials to make them more effective.

I developed the app to include the following content and features:

• Practice quizzes for each of the ten tutorials in the xAPI + Storyline series
• Five “statement sending challenges” that require people to send xAPI statements to one of my Learning Record Stores (LRSs). The app evaluates these statements for accuracy automatically.
• A leaderboard that’s sorted by the number of challenges that people have completed.
• Automated emails that provide resources when people fail a challenge, as well as help guide people back to the app if they become inactive.
• A Slack community where people can ask questions and help others.
• Dashboards that show me how people are performing on each quiz and question.

If you would like to try the challenges for yourself, then you can sign up for the app. Otherwise, read on to learn more about how I designed and developed the app.

Process

To ensure that this app would work as desired, I needed to incorporate many different tools and technologies. I cover each of them briefly below, but if you have any specific questions, feel free to ask them in my eLearning Slack channel.

Creating the Glide App

I created the xAPI Challenges App with a tool called Glide, which lets you build web apps from Google Sheets. The app is made up of several components:

• A sign-up form that allows you to enter your name, website, and a brief bio. People have the option to make their profile public, which helps them get recognition and leads from the leaderboard.
• A profile sheet that keeps track of your information, as well as which challenges you’ve completed. 
• A home screen that shows a progress bar and suggests the next challenge that you should complete.
• A challenges screen that shows the list of all available xAPI challenges (as well as which ones you have completed).
• An individual screen for each challenge that links to the tutorial and quiz for that challenge.
• A leaderboard screen that ranks people by the number of challenges they’ve completed. You can select these profiles to see people’s bios, websites, and contact information.

Glide made it simple to build the user interface for the app. The challenges came with using the Glide features in creative ways to achieve the functionality I was looking for, as well as using complex Google Sheets formulas to evaluate the statements submitted by the users for the “statement sending” challenges. 

Developing the Quizzes

To design and develop the quizzes, I wrote a series of questions for each xAPI + Storyline tutorial. I used the objectives for each tutorial to guide this process.

From there, I developed the quizzes in Articulate Storyline. I asked my illustrator to create robot characters that would fit the theme of the app, and then I used a different character depending on the difficulty of the tutorial and associated quiz.

Since the Storyline quizzes are hosted outside of the Glide App, I needed to identify the user when they accessed a quiz. I made a simple form that asks the user for their email address when they first launch one of the quizzes.

From there, I cache the email address in the user’s browser. When they return to any of the Storyline quizzes, it prompts them to confirm that the cached email address is indeed their email address. This makes the UX smoother and leaves less room for error.

I also added detailed custom xAPI tracking to all of the quizzes. This allowed me to see exactly what users answer for each question, as well as how long they are spending on individual questions and quizzes as a whole.

As you will see later, this data helps me improve and optimize the tutorials.

Automated Emails

Furthermore, when a user passes or fails a quiz, I send an xAPI statement to Veracity LRS. These statements are automatically forwarded to Watershed LRS, which has a Zapier integration. I use Zapier to bring the Watershed LRS data into the Google Sheet that powers the Glide App (to measure completions).

I added to this Zap so that it tags users in my email automation tool, Drip, whenever people pass or fail a quiz. 

I used Drip to create email automations that ask users if they need additional help whenever they fail a quiz.

Whenever someone passes a quiz successfully, they receive a congratulations email that guides them into the next challenge.

I also set up sequences so that if the user is inactive for a set number of days, they receive nudge emails that guide them back into the app (and ask them if they need additional help). 

Many people respond to these emails with the issues that they’re facing, so this has helped me get a much better pulse on where people struggle when it comes to sending custom xAPI statements.

Creating the “Statement Sending” Challenges

The statement sending challenges were one of the driving forces that led me to create this app. I wanted users to be able to send xAPI statements to one of my LRSs, and, if it met the given criteria, they would receive credit for the challenge.

To unlock the statement sending challenges, I required users to first pass the quiz for that associated challenge. Upon passing, they receive an automated email that includes the LRS credentials and criteria that the xAPI statement must meet.

When they send the xAPI statement successfully, it goes to a Veracity LRS that I created just for this purpose. From there, it gets forwarded to Watershed LRS. It triggers an automation with Google Sheets, and it brings the user’s statement into the Google Sheet that powers the Glide App.

If the statement meets all of the criteria (as evaluated by Google Sheets formulas that I created), then it changes a TRUE / FALSE variable on the Profile sheet, designating that the user has now successfully completed the statement sending challenge.

If they do not pass the challenge within a set number of days, they receive email follow up that provides resources and asks if they need additional help.

When people explain the issues that they are having, I am able to look at my Veracity LRS to see the components of their xAPI statement and determine why they’re having trouble.

This not only allows me to help people more efficiently, but it also helps me identify elements that I should draw more attention to in my tutorials.

Building the Slack Community

I created a Slack channel to facilitate pilot reviews with the xAPI Challenges App before bringing it live. Originally intended for feedback, I noticed that people began asking questions about how to complete some of the challenges.

From there, people began asking more general instructional design and eLearning questions. I decided to open up the channel to any interested instructional designers and eLearning developers, and we now have over 300 members.

This community serves as an excellent resource for people completing the challenges. It is easy to get same-day help for any xAPI challenges that people are facing, and others in the community are able to provide their expertise to help.

The Slack community is open to all of my mailing list subscribers, so if you’re interested, then consider signing up.

Evaluation

As mentioned previously, I conducted two pilot reviews with a group of enthusiastic participants that had a range of previous experience with xAPI.

This enabled me to iron out any technical issues within the app, as well as make minor user experience improvements.

After bringing the app live, I began creating dashboards and visualizations based on the xAPI data that I received. 

Some of the most useful have been the pie charts that show how users respond to each question in each quiz. This helps me see which questions people struggle with the most, as well as identify which distractors are most common.

These dashboards help me identify ways to improve my tutorials by making certain concepts more clear. When I make the change and see the users perform better on the associated question, I know that the change has improved the content. This is a key component of data-driven eLearning design.

Takeaways

This project was a significant amount of work, but I am glad that I created it. I enjoyed having a project of my own where I could implement xAPI and use data to improve my product. 

I also found the automated emails to be an effective way to provide guidance, and I’ve learned that many eLearning projects would benefit from this functionality.

As an app, the xAPI Challenges App functions more as a prototype. Due to Glide Apps functionality, it usually takes around five minutes for someone to see their completion registered in the app. 

I also think that this app would be better if it was focused on xAPI more generally, and not xAPI + Storyline specifically. However, for the audience that needs to use xAPI with Articulate Storyline, this app serves its purpose.

In all, building this app was a great learning experience. It allowed me to experiment with different technologies and techniques, and it helped me improve my xAPI tutorials in the process.

If you need a hand building a learning app, adding automated emails to your eLearning offerings, or implementing xAPI, then you should contact me for a free quote.