JavaScript in a NoSQL database

The latest release of MarkLogic (the popular Enterprise NoSQL database) brings server-side JavaScript support. You can now run JavaScript code directly against documents in the database, which opens up some powerful possibilities. I’d recommend reading the documentation and checking out the examples on GitHub.

In this article I’ll cover two features: loading content into the database using server-side JavaScript, and updating that content.

I’m assuming you’ve got MarkLogic installed and have created a REST application server. If you need help with those steps, read this document.

To quickly spin up a REST application server, grab this example configuration file (make sure the port number is available) and call the webservice via curl, Postman, or your preferred REST client.

Here’s config.json:

{
  "rest-api": {
    "name": "tamas-blog-example",
    "group": "Default",
    "database": "tamas-blog-example-content",
    "modules-database": "tamas-blog-example-modules",
    "port": "7000"
  }
}

And here’s a curl statement (update your username/password, the path to config.json, and the hostname of your MarkLogic instance if needed):

curl --digest --user myMarkLogicUserName:MyMarkLogicPassword -X POST -d@"path/to/config.json" -i -H "Content-type:application/json" http://localhost:8002/v1/rest-apis

As of curl v7.4+, digest-based authentication works differently. You’ll also need to specify the realm: curl --digest --user "myRealm\myMarkLogicUserName:MyMarkLogicPassword" -X POST -d@"path/to/config.json" -i -H "Content-type:application/json" http://localhost:8002/v1/rest-apis

Navigate to your Query Console (http://localhost:8000) and check the ‘Content Source’ dropdown. You should see a database called ‘tamas-blog-example-content’ (assuming you used the exact config.json above).

Time to load data. There are several ways to do this, but since this article is about server-side JavaScript, we’ll use built-in functions. The dataset comes from https://github.com/mledoze/countries, except I’ve broken it into individual JSON documents per country. You can grab the dataset from my GitHub page.

Make sure you’ve selected the right database from the Content Source dropdown first. That tells the system where to load the data.

To load the data, we need a few built-in MarkLogic functions. The first is xdmp.filesystemDirectory(), which takes a path as its argument and lists all documents in a folder:

We can take that information and insert the individual documents into the database using xdmp.documentLoad(). As part of this function, we insert the documents and also specify their URIs (unique identifiers) and a collection.

The original documents didn’t contain a primary field for the country name, but I needed that piece of information to create an index. Using server-side JavaScript, I can bolt on an extra JSON property to every document in the database.

declareUpdate();

for (var country of fn.collection('countries')) {
  var obj = country.toObject();
  obj.id = obj.name.common;
  var uri = xdmp.nodeUri(country);
  var collections = xdmp.documentGetCollections(uri);
  xdmp.documentInsert(uri, obj, xdmp.defaultPermissions(), collections);
}

The code above iterates through all documents in the countries collection. (fn.collection() returns a ValueIterator, which means we can use the ES6 for ... of loop.) Documents in the database are stored as nodes, and nodes are immutable. To get a mutable object, we call toObject(). Once we’ve done that, we can add a new property (in our case, id). It’s worth noting that we need to store the collection information in a variable because re-inserting the document strips all collection data.

The dataset I’ve provided via GitHub already has the id field.

Hit the explore button next to the Content Source dropdown. You should see 250 JSON documents inside your database. That’s it.

If you’d like to know how to use this data from a Node.js application with the MarkLogic Node.js Client API, read my other article.