Devlin Peck
In this tutorial, you'll learn how to query your Learning Record Store (LRS) to work with the xAPI data that you've collected. You can use this xAPI data to draw powerful conclusions about the effectiveness of your learning programs.
This skillset will allow you to create adaptive learning experiences, in-course leaderboards and dashboards, and so much more.
Before continuing, I highly recommend that you complete the 3-part Getting Started with xAPI Tutorial Series. You may also choose to complete some of the intermediate tutorials so that you have a better handle on xAPI before diving into this advanced tutorial.
Once you're comfortable with the structure of xAPI statements and how to send them, feel free to move forward with this tutorial.
Feel free to ask for help in the eLearning Development space in the ID community (free for mailing list subscribers).
When you query the LRS, you're essentially copying specific xAPI statements from the LRS into a JavaScript variable so that you can work with the data in the browser (using JavaScript).
When you have access to xAPI statements like this, you can use that data as needed to adjust your course and display data based on the statements already in the LRS.
With the current version of the xAPI specification (1.0.X), you can only query based off certain elements of the xAPI statement. This puts pressure on whoever is implementing xAPI to ensure that they do so in a way that makes sense for your queries.
For example, you cannot query based off of anything in the xAPI statement's result object. If you wanted to query the LRS for every statement where the user scored 100%, you wouldn't be able to do so.
Instead, you would have to query statements based off of one or more query-able elements (listed below), then cycle through them and show only those where the user scored 100%. (If this sounds confusing, don't worry too much now...we'll dive into some examples later.)
Here are the elements that you can query:
We will dive into the technical details and how to query based off these elements in the next section.
For now, it's important to know that we can query based off one, multiple, or all of these elements; for example, we can query the LRS to pull all of the statements from a specific actor performing a specific verb since a specific time, or we can just query the LRS to pull all of the statements with a specific object ID.
The statement ID is unique to each statement, so we can expect to get one statement back if we query by statement ID.
Querying the LRS to use the data in a browser makes sense when we want to incorporate that data back into our learning experience. For example, we may want to check whether a user passed a quiz in Course A to determine if we should show additional supporting content in Course B.
However, we must always stay cognizant of how many statements we're getting back in our query. Many LRSs restrict the number of statements that we can get from a single query (usually between 30 and 50), and this is to save the browser and network from getting overloaded with your request.
Imporant note: It's possible to pull more than 50 statements in a query, and I will explain how to do this later in the tutorial.
This is why it's important to be as specific as possible with your queries. Rather than pulling all statements from a given actor and then cycling through them to find specific verbs, include the verb ID with your query.
If you expect to receive more than 50 statements from your query, you should probably use an alternative approach to working with the data.
For example, most LRSs have powerful analytics tools that you can use on their platform, and you can use business intelligence tools to store and work with your data.
Some of these tools may even have APIs that you can use to pull the specific piece of data that you need; for example, say that you want to know what the average score is across every user that completed your quiz.
If 5,000+ users have completed your quiz, it will crash your browser trying to query so many statements and working with them client-side. Instead, you would want to use a tool that calculates the average on their servers and then reports it directly to the browser via a single query.
In short, keep your queries as specific as you can so that they return as few statements as possible.
To set up and send the query to our LRS, we're going to need to use JavaScript. I recommend creating a new folder titled "xAPI Query Test" and a new JavaScript file within that folder titled "query.js".
Imporant note: Add your xapiwrapper.min.js file to the folder so that you can use its features when it comes time to send the query. (Link to official download.)
The first thing we'll do in the new JavaScript file is set up our "conf" object, just as we had to do when we were sending statements.
I'm going to set the "conf" object for a test data set that was included with my Veracity LRS signup. Feel free to use these credentials if you don't already have a data set that you'd like to work with. If you do have a data set that you'd like to work with, then change your endpoint and credentials accordingly.
{% c-block language="js" %}
const conf = {
"endpoint": "https://sample-lrs-lekofoa.lrs.io/xapi/",
"auth": "Basic " + toBase64("username:password")
};
ADL.XAPIWrapper.changeConfig(conf);
{% c-block-end %}
Now that we're authenticated with our LRS, we can create the query that we will send to the LRS.
First, we'll create a parameters variable that can communicate with the LRS through the xAPI wrapper, like so:
{% c-block language="js" %}
const parameters = ADL.XAPIWrapper.searchParams();
{% c-block-end %}
We'll use this variable later in the code to execute the query. For now, though, we need to add properties to this variable to define the query that we want to send. The general structure for this is as follows:
{% c-block language="js" %}
parameters["element to query"] = "element value that we're looking for";
{% c-block-end %}
Let's consider an example for each element that we can query.
Important note: This next section is meant to serve just as much as a resource as it is part of the tutorial. Feel free to bookmark this page to assist with your future LRS querying needs.
You can query the LRS to grab a specific statement by its ID:
{% c-block language="js" %}
parameters["id"] = "7e573c2b-47bb-447b-be49-fbe4e7827132";
{% c-block-end %}
Note that you will need to know the statement's ID beforehand to do this. You will also have to swap the ID in this example with your statement's ID.
Important note: Despite this code looking accurate per all of the research that I've done, I have not had success querying by the statement ID. Feel free to try it, or if you see an issue and know the resolution, please let me know.
When you query the statements by a given agent, you must pass the object that you used to identify the agent. For example, if you identify your agents with an email address, you can query the agent like this:
{% c-block language="js" %}
parameters["agent"] = '{"mbox": "mailto:devlin@peck.consulting"}';
{% c-block-end %}
If you identify your agent with an account, you can query the agent like this:
{% c-block language="js" %}
parameters["agent"] = '{"account": {"homePage": "http://lms.veracity.it","name": "661081051161041013269989897"}}';
{% c-block-end %}
You can query the LRS by verb ID, too:
{% c-block language="js" %}
parameters["verb"] = "http://adlnet.gov/expapi/verbs/initialized";
{% c-block-end %}
You can use the following code to query the LRS by activity or object ID:
{% c-block language="js" %}
parameters["activity"] = "https://www.coursera.org/specializations/python/3";
{% c-block-end %}
Querying by timestamp gets slightly trickier because the timestamp must be in ISO 8601 format (as discussed in the duration tutorial).
The easiest way to navigate this is to create a new Date object using "new Date('insert date here')" and converting it to ISO 8601 with the toISOString() function. You can see this executed in the code below:
{% c-block language="js" %}
const collectSinceDate = new Date("January 1, 2020 00:00:00");
const collectBeforeDate = new Date("February 1, 2020 00:00:00");
parameters["since"] = collectSinceDate.toISOString(); /* Returns statements since Jan 1, 2020 */
parameters["until"] = collectBeforeDate.toISOString(); /* Returns statements from before Feb 1, 2020 */
{% c-block-end %}
As you can see, you use the "since" and "until" keywords to query the statements by timestamp (you can also use only one or the other). The above code would return all statements with timestamps in January, 2020.
Once you've set up the parameters that you'd like to filter by, you're ready to send the query to the LRS. You can do so with the following line of code:
{% c-block language="js" %}
const queryData = ADL.XAPIWrapper.getStatements(parameters)
{% c-block-end %}
For this example, let's imagine that we want to pull all statements where a user "initialized" the machine learning lesson. Our full JavaScript code look like this:
{% c-block language="js" %}
const conf = {
"endpoint": "https://sample-lrs-lekofoa.lrs.io/xapi/",
"auth": "Basic " + toBase64("username:password")
};
ADL.XAPIWrapper.changeConfig(conf);
const parameters = ADL.XAPIWrapper.searchParams();
parameters["verb"] = "http://adlnet.gov/expapi/verbs/initialized";
parameters["activity"] = "https://www.coursera.org/learn/machine-learning/1";
const queryData = ADL.XAPIWrapper.getStatements(parameters)
{% c-block-end %}
Once this code is executed, the queryData variable will hold the response from the LRS. This response includes all of the statements that matched the given parameters, which we can access using "queryData.statements" in JavaScript.
However, there is one exception to this...
Many LRSs will limit the number of statements that can be returned from a given query. Yet Analytics LRS, for example, limits the number of statements to 50, whereas Veracity LRS has a much higher limit of 500.
If you find yourself in a position where you need to return statements beyond this limit, then it's important to know that you do have an option available to you (despite it degrading the user experience due to the bandwith it requires).
When the LRS cannot return every statement that you requested due to the query limit, it will return a "more" property on your returned object with a URL that you can use to access the rest of the statements. We can access this URL using "queryData.more", and we can pull the next set of statements (up to the query limit) using the following code:
{% c-block language="js" %}
if (queryData.more && queryData.more !== ""){
const moreStatements = ADL.XAPIWrapper.getStatements(null, queryData.more);
console.log(more.statements);
}
{% c-block-end %}
This code only executes if the "more" property exists, and if it does, then it requests the additional statements from the provided URL. You would be able to access this set of statements using "moreStatements.statements".
Likewise, if there were still statements remaining after this second query, you would be able to access the next URL using "moreStatements.more". Continuing to set up if statements like this can get cumbersome and inelegant, so it's best practice to keep your queries within the query limit of the LRS.
If you're specific enough with your queries to keep the number of returned statements within the query limit, then you don't need to worry about the code discussed in this section.
Now that our JavaScript is set up to issue the query, we can set up a web page to display our results.
Important note: If you only plan to use your xAPI data within a Storyline course, then you will not need to use the code in this section. However, having a simple display of your query data on a web page is helpful for troubleshooting and testing.
To display the results from our query, let's set up a simple HTML page in our "xAPI Query Test" folder titled "results.html". Feel free to copy and paste the following code into your new HTML file (this will only work if you've named your files exactly as I suggested):
{% c-block language="markdown" %}
<!DOCTYPE HTML>
<html lang="en"></html>
<head></head>
<title>Query Results</title>
<meta charset="UTF-8">
<body></body>
<h1>Query Results</h1>
<ul id="statement-list"></ul>
<script type="text/javascript" src="xapiwrapper.min.js"></script>
<script type="text/javascript" src="query.js"></script>
{% c-block-end %}
This code adds the structure for the HTML page, creates an unordered list with the "statement-list" ID, and links our HTML file to the JavaScript files that we've included in our folder.
Now let's return to our "query.js" file. We need to write some code that will populate the unordered list with all of the statements returned by our query.
We'll populate our unordered list with the following approach:
You can add the following code at the bottom of your "query.js" file to accomplish this:
{% c-block language="js" %}
queryData.statements.forEach(populateStatements)
function populateStatements(statement) {
const newItem = document.createElement("li")
const newStatement = document.createTextNode(statement.actor.name + " " + statement.verb.display["en-US"] + " " + statement.object.definition.name["en-US"])
newItem.appendChild(newStatement)
document.getElementById("statement-list").appendChild(newItem)
}
{% c-block-end %}
In the first line of code, we apply the populateStatements function to each item in the queryData.statements array (which holds each of our returned statements).
The populateStatements function takes an argument, "statement," which represents one item in the queryData.statements array. It then converts the elements of the statement that we tell it to into a list item before appending it to the HTML page.
Let's take a closer look at the newStatement variable that we created; this is where we tell the code which elements of our statement to add to the list:
{% c-block language="js" %}
const newStatement = document.createTextNode(statement.actor.name + " " + statement.verb.display["en-US"] + " " + statement.object.definition.name["en-US"])
{% c-block-end %}
This code creates a text node with the actor's name, the verb that they performed, and the object that they acted on.
As you may notice, we use dot notation to access these different elements of the statement.
We start with the statement as a whole and work our way down the "branches" of the statement to get more and more specific. (The most common analogy for this is thinking about the branches of a tree.)
For example, let's consider the structure of an xAPI statement:
{% c-block language="js" %}
const statement = {
"actor": {
"name": "Devlin",
"mbox": "devlin@peck.consulting"
},
"verb": {
"id": "http:/www.example.com/ran"
"display": { "en-US": "ran" }
},
"object": {
"id": "http://www.example.com/marathon",
"definition": {
"name": { "en-US": "marathon"},
"description": { "en-US": "a super long marathon"}
},
"objectType": "Activity"
}
};
{% c-block-end %}
Now we can use dot notation to access different elements of the statement:
However, what if we want to return the values that have a language code before them ("en-US")? If we tried to use dot notation, our JavaScript would throw an error because it would try interpreting the dash as a subtraction operator.
Instead, we need to use bracket notation, like so:
If you're confused by the code overall, don't worry. The code to add the statements to the HTML5 page is not important. However, you should understand the explanation above about how to access the different elements of an xAPI statement, as this will help you work with the statements returned by your query.
Now that your code is in place, save your files and open the "results.html" file in your web browser. You should see all of the statements returned by the query.
If you'd like to change which elements of the statement are displayed, modify the "newStatement" variable within the "populateStatements" function that you created.
We dove into some technical details in this tutorial, but at this point, you should know how to query the LRS to return the xAPI statement data that you want.
All we did with this data so far is display it on a webpage, but in the next advanced xAPI tutorial, we'll look at how to build an xAPI-enabled leaderboard within an Articulate Storyline course.
