Build an autocomplete search tool for PoolParty

For one of my projects I wanted to emulate the situation where a content author or editor needs to search a PoolParty project for matching concepts in order to tag their content. In an earlier article I showed how to create a SPARQL query to do this. However, the application was a bit primitive (hey, I'm not a developer, remember).

I've always wanted to work out how to do autocomplete search boxes (those that update results as you type), and it seemed to me that I could revisit my old primitive code and develop it into something a bit more slick. Now that I've done this, I thought I'd share it, because from everything I've seen on the web while researching this, clear and commented code is the exception rather than the rule.

So, here goes. As usual, I'll give a walk-through of the application and the code, and then you can find the complete source at the end of this article.

Note that to run this application you will need to plug in your own PoolParty SPARQL endpoint; the code here just has a placeholder: "[project]". It should be fairly easy to implement for your server, but if you need some help in doing this then please use the contact form to get in touch, and I'll be pleased to help you out.

A quick run-through of the application

On loading the application (via index.html) the user sees a simple search box.

Autocomplete search walk-through 1

The user then starts to type in the search text. In this case she will be searching the Medical Subject Headings taxonomy (MeSH 2016), and so the search term is medically-related; she's looking for concepts about coagulation of blood. Typing in coag results in the immediate appearance of a dropdown list of matching concepts, with the search string highlighted in the retrieved concepts.

Autocomplete search walk-through 2

The user can then click on the best match (Blood Coagulation Factors). In this demo example the skos:prefLabel and the URI are pasted into the page below.

Autocomplete search walk-through 3

If the user wants to choose further matches she can clear the search box and start typing again.

In a real life case that label and URI would probably be written as a tag into a database (or preferably as a triple into a graph database).

Notice that the page never re-loads. The search is carried out using the autocomplete functions in jquery-ui. Notice also that although this is using a search form, there is no need for a submit button. Each keystroke in the search box sends a search, retrieves the results and reworks the results display.

Working through the code

The application has two code files; index.html and search.php. The first builds the interface, dispatches the search request and formats the response. The second carries out the search and sends back the response.

On loading index.html the script loads the usual libraries for bootstrap css, jquery and jquery-ui. There is also a specific local style that sets the width for the dropdown list box.

<!doctype html>
<html>
<head>
<title>PoolParty auto-complete search form</title>
<link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.0.3/css/bootstrap.min.css"><!-- load bootstrap via CDN -->
<link rel="stylesheet"href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script></head>
<style type="text/css">
.ui-autocomplete{max-width:500px}
</style>
</head>

The remainder of the html is in a div element that contains a form and a div that will display the search results.

<div class="col-sm-6 col-sm-offset-3">
  <h1>Search PoolParty for matching concepts</h1>
  <form action="#" method="get">
    Find concepts containing:
    <input type="text" id="autocomplete" name="query_content"><br/>
  </form>
  <hr>
  <!-- The result div is where the returned results will appear -->
  <div id="result">

  </div>
</div>

Now to the jQuery code that initiates the search and processes the display of results. There is a lot going on here, so let's take it a bit at a time. The overall function runs the jquery-ui autocomplete function on the form element above (which has an id of "autocomplete"). This points to the php script search.php, which is where the remote search will be conducted. Note that this function waits until the set number of characters has been typed, and then invokes the search function in search.php for each character typed. I will show search.php shortly; for now, all that matters is that it takes an incoming text string (see query_content in the listing above), runs a query and returns the response.

The results contain two pieces of information for each concept found in PoolParty; the search label (defined in the SPARQL query, covered below) which is either a skos:prefLabel or the skos:altLabel, and the concept URI. These are packaged up as json and the individual values can be extracted using ui.item.value (for the human-readable label) and ui.item.uri (for the URI). These values are then used to build up the collection of results displayed in the list box, but in the process they are also highlighted (the _renderItem function). This replaces the label string with a new string which has a highlight around the text matched from the searchbox. This in turn is using the JavaScript RegExp object, and the "gi" means global (find all matches) and case-insensitive searching. The results are assembled as list items in an unordered list. This is displayed as a dropdown list. When an item in the list is clicked (the select event, triggered when the search completes; that is, every time search.php sends back data) the program adds a piece of html to the results div, showing the ui.item.value and the ui.item.uri (together with its anchor tag).

<script>
$("#autocomplete").autocomplete({
  source: "search.php",
  minLength: 2,//search after three characters
  select: function(event,ui){
    //format the response values into html
    //ui.value contains the searchLabel which is the found label  (prefLabel or altLabel) for the concept
    var html = '<p>' + ui.item.value + '</p>';

    // and ui.item.uri has the URI of the concept
    html += '<p><a href="'+ui.item.uri+'">'+ui.item.uri+'</a></p>'

    // $('#result').html(html);//insert the formed html into the result div
    // or, if you want to put several results on the page, use append
    $('#result').append(html);//append the formed html to the result div
  }
}).data("ui-autocomplete")._renderItem = function (ul, item) {
  // highlight the search string in the returned text
var newText = String(item.value).replace(
new RegExp(this.term, "gi"),
  "<span class='ui-state-highlight'>$&</span>");
  return $("<li></li>")
    .data("item.autocomplete", item)
    .append("<div>" + newText + "</div>")
    .appendTo(ul);
};
</script>
</body>
</html>

Let's now turn to search.php. This is a small script that takes in the data passed from the form in index.html (note that the jquery autocomplete tools do the form dispatch as a GET, and the payload is called "term").

The program uses the EasyRDF library to manage the request sent to the SPARQL endpoint and the corresponding response. We'll get to that shortly.

When you dispatch a form in html you send off an array of form data. Each item in the array is identified by its form id. In this case, the value that is dispatched is the text typed into the form, and this is sent from index.html to search.php as a GET, and the payload is called "term" (this is defined in jquery ui autocomplete).

In search.php the _GET array is picked up and the array member called "term" is sent to the runPPQuery function. The response from that function is an array. Each item of that array is an object which contains a large number of properties. We're only interested in two of those properties; the searchLabel (which comes from either skos:prefLabel or skos:altLabel) and the concept (which is the URI). These are then added to a new associative array called $return_array, and that is converted to json using json_encode. The final echo is just to provide a handle for the index.html program to pick up the json.

<?php
// include the EasyRDF libraries
require_once 'vendor/autoload.php';
// Check whether the post array is empty or contains a $query_content variable
if(!empty($_GET)) {
  // The text in the search form is passed as a GET called term
  $searchstring = $_GET["term"];
  $results = runPPQuery($searchstring);

  $return_rows = array();
  // Loop through the results to form the return values
  foreach($results as $result){
    //strval strips the text out of the object
    $row['value'] = strval($result->searchLabel);
    $row['uri'] = strval($result->concept);
    // add this result to the return_rows array
    $return_rows[] = $row;
  }

  // Format the response as json and output
  // This will be picked up in index.html
  echo json_encode($return_rows);
}

The real final piece of code is the runPPQuery function. This is where the script formulates a SPARQL query to send to PoolParty.

Every taxonomy project in a PoolParty instance has a SPARQL endpoint. This is basically a web location to which you can send a query. The endpoint responds with a package of results.

Here, you can see the entire function. First we create a new SPARQL endpoint object, based on a project in a PoolParty server.  Next comes the SPARQL query. I'm not covering how to build SPARQL queries, because there are plenty of resources on the web and anyway it's not a core part of this article. Suffice to say that the query takes the search string and sends off a query asking for any concepts that contain that string in either the skos:prefLabel or the skos:altLabel. Since a PoolParty concept must have one prefLabel and may have any number of altLabels, this maximises the chances of getting back a reasonable match.

Notice, too that we are limiting the results to 20, though you can change this or leave it off (though prepare for many results and a performance hit). Like the regular expression above, this one is case-insensitive ("i").

The results come back in $qresult, which is returned to the code that called the function (see above).

function runPPQuery($the_searchstring) {
  $sparql = new \EasyRdf\Sparql\Client('https://tellura.poolparty.biz/PoolParty/sparql/[project]');

  // Define the SPARQL query - we want any concept in the project that contains the
  // string defined in the query within its prefLabel or altLabel properties
  $qresult = $sparql->query(

  'PREFIX skos:<http://www.w3.org/2004/02/skos/core#> SELECT DISTINCT ?concept ?searchLabel WHERE { { ?concept skos:prefLabel ?searchLabel. } UNION { ?concept skos:altLabel ?searchLabel. } FILTER (regex(str(?searchLabel), "'. $the_searchstring .'", "i"))} LIMIT 20'

  );
  return $qresult;
} // end of runPPQuery function

That brings me to the end of this technical article. I hope you find it useful in building your own PoolParty search solutions. Below you will find the complete code for both the index.html and search.php scripts.

index.html

<!doctype html>
<html>
<head>
<title>PoolParty auto-complete search form</title>
<link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.0.3/css/bootstrap.min.css"><!-- load bootstrap via CDN -->
<link rel="stylesheet"href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script></head>
<style type="text/css">
.ui-autocomplete{max-width:500px}
</style>
</head>

<body>

<div class="col-sm-6 col-sm-offset-3">
  <h1>Search PoolParty for matching concepts</h1>
  <form action="#" method="get">
    Find concepts containing:
    <input type="text" id="autocomplete" name="query_content"><br/>
  </form>
  <hr>
  <!-- The result div is where the returned results will appear -->
  <div id="result">

  </div>
</div>

<script>
$("#autocomplete").autocomplete({
  source: "search.php",
  minLength: 2,//search after three characters
  select: function(event,ui){
    //format the response values into html
    //ui.value contains the searchLabel which is the found label  (prefLabel or altLabel) for the concept
    var html = '<p>' + ui.item.value + '</p>';

    // and ui.item.uri has the URI of the concept
    html += '<p><a href="'+ui.item.uri+'">'+ui.item.uri+'</a></p>'

    // $('#result').html(html);//insert the formed html into the result div
    // or, if you want to put several results on the page, use append
    $('#result').append(html);//append the formed html to the result div
  }
}).data("ui-autocomplete")._renderItem = function (ul, item) {
  // highlight the search string in the returned text
var newText = String(item.value).replace(
new RegExp(this.term, "gi"),
  "<span class='ui-state-highlight'>$&</span>");
  return $("<li></li>")
    .data("item.autocomplete", item)
    .append("<div>" + newText + "</div>")
    .appendTo(ul);
};
</script>
</body>
</html>

search.php

<?php
// include the EasyRDF libraries
require_once 'vendor/autoload.php';
// Check whether the post array is empty or contains a $query_content variable
if(!empty($_GET)) {
// The text in the search form is passed as a GET called term
$searchstring = $_GET["term"];
$results = runPPQuery($searchstring);

$return_rows = array();
// Loop through the results to form the return values
foreach($results as $result){
//strval strips the text out of the object
$row['value'] = strval($result->searchLabel);
$row['uri'] = strval($result->concept);
// add this result to the return_rows array
$return_rows[] = $row;
}

// Format the response as json and output
// This will be picked up in index.html
echo json_encode($return_rows);
}

function runPPQuery($the_searchstring) {
$sparql = new \EasyRdf\Sparql\Client('https://tellura.poolparty.biz/PoolParty/sparql/[project]');

// Define the SPARQL query - we want any concept in the project that contains the
// string defined in the query within its prefLabel or altLabel properties
$qresult = $sparql->query(

'PREFIX skos:<http://www.w3.org/2004/02/skos/core#> SELECT DISTINCT ?concept ?searchLabel WHERE { { ?concept skos:prefLabel ?searchLabel. } UNION { ?concept skos:altLabel ?searchLabel. } FILTER (regex(str(?searchLabel), "'. $the_searchstring .'", "i"))} LIMIT 20'

);
return $qresult;
} // end of runPPQuery function