JavaScript in a NoSQL database

The latest release of the popular Enterprise NoSQL database, MarkLogic, brings server side JavaScript support to the server - in other words, it is possible to run JavaScript code against the documents in the database which gives us some very powerful options. I do recommend that you read the documentation as well as have a look at the examples on GitHub.

In this article I'd like to discuss two features - loading content into the databas using server side JavaScript and updating the content.

I am also going to assume that you have MarkLogic installed and have created a REST application server. If you need help with these steps read the this document.

To quickly create a REST application server you can take this example configuration file (make sure that the port number is available) and call the webservice either via curl or Postman or your preferred REST client.)

Here's config.json:

{
  "rest-api": {
    "name": "tamas-blog-example",
    "group": "Default",
    "database": "tamas-blog-example-content",
    "modules-database": "tamas-blog-example-modules",
    "port": "7000"
  }
}

And here's a curl statement (make sure you update your username/password combination, the path to config.json and if required the hostname of your MarkLogic instance as well.)

curl --digest --user myMarkLogicUserName:MyMarkLogicPassword -X POST -d@"path/to/config.json" -i -H "Content-type:application/json" http://localhost:8002/v1/rest-apis

Please be aware that as of curl v7.4+ the way digest based authentication works has changed. In light of this change you also need to specify the realm as part of the curl statement: curl --digest --user "myRealm\myMarkLogicUserName:MyMarkLogicPassword" -X POST -d@"path/to/config.json" -i -H "Content-type:application/json" http://localhost:8002/v1/rest-apis

Navigating to your Query Console (http://localhost:8000) and examining the items in the 'Content Source' dropdown should allow you to see a database called 'tamas-blog-example-content' providing the fact that you have used exactly the same config.json provided above.

It's time to load data into the database. This can be done via quite a few ways but as this article is about server side JavaScript we are going to use some built-in functions. The database has been taken from https://github.com/mledoze/countries, except I have broken it up into manageable pieces and created an individual json document per country - this dataset can be access from my GitHub page.

Please make sure to select the right database from the Content Source dropdown field first - this will tell the system which database to load the data into.

In order to load the data we need to make use of a few built-in MarkLogic functions - the first one being xdmp.filesystemDirectory() which accepts a path as its argument and lists all the documents that in a particular folder:

We can take this information and insert the individual documents into our database using the xdmp.documentLoad() function. As part of this function we not only insert the documents but also specify their URIs (unique identifiers) and a collection.

The original documents didn't contain a primary field that would display the name of the country but I needed this piece of information in order to create an index. Using server-side JavaScript I can add an additional JSON property to all the documents in the database.

declareUpdate();

for (var country of fn.collection('countries')) {
  var obj = country.toObject();
  obj.id = obj.name.common;
  var uri = xdmp.nodeUri(country);
  var collections = xdmp.documentGetCollections(uri);
  xdmp.documentInsert(uri, obj, xdmp.defaultPermissions(), collections);
}

The code above iterates through all the documents that belong to the countries collection. (For your information fn.collection() returns a ValueIterator which means that we can use the new ES6 for ... of loop). The documents in the database are stored as nodes and nodes are immutable. In order to create mutable objects we need to call the toObject() method. Once we do that we can add an extra property to the object (in our case, we are adding a property called id). It is also important to note that we have to store the collection information in a variable as re-inserting the document removes all collection information.

Please note that the dataset I have provided to you via GitHub has the id field already.

Go ahead and press the explore button next to the Content Source dropdown you should see 250 JSON documents inside your database. Success!

If you'd like to know how to utilise this data from a Node.js application using the MarkLogic node.js client API please read my other article.