Build your own PoolParty backup utility

As a part of your governance process, if you are a PoolParty taxonomy manager you will know that you have a number of options for backing up your precious taxonomy data. If you have an on-premises server you can back up the data folder of your PoolParty installation. On any server, cloud or on-premises, you can create system or project snapshots. You can export any level of a taxonomy in RDF or Excel format. But you can also do a quick backup by choosing the PROJECT > Download Project menu when you have a project loaded.

This approach gives you a downloaded PoolParty Archive (.ppar) file. This type of file backup is a complete version of your project in a form that you can store in your own file archives, and it can also easily be used to restore your project to another PoolParty server.

This is fine for doing a backup of just a few projects, but it gets tedious when you get into double figures. So I have written a small php utility that makes the whole process a lot easier. You need to do some configuration in the file to specify your PoolParty server and the user, but from then on it's easy. Just put this program in a web server (localhost is fine, and so is xAMP if you don't want the hassle of setting up a local web server), load it into your browser, and it will export a PoolParty archive file for every project in your system. For each project it will give a (configurable) filename with a timestamp. Here is a screenshot of the program running (I have blocked out some of the filenames for confidentiality reasons).

The report gives you a list of all of your projects, with the filename (which is timestamped with the year, month, day, hour, minute and second) and the file size.

I'm not claiming this to be a particularly well-written application, but it works in my hands and I've found it to be a real time-saver.

As usual, I'll go through the code step by step, and I've included the full source at the end of the article.

Walking through the code

The first thing to mention is that if you have lots of projects you will need to increase the php memory limit. I had dozens of projects to back up, so I allowed 2048 MB of memory. This is a huge memory allowance, and I'd normally use far lower levels. For this reason I didn't want to modify php.ini (the normal way to change memory limits) but instead put this command at the top of the program. This way the increased memory is only used when the program runs. There is probably a way to refactor the code so that it doesn't need so much memory, but that's beyond my meagre skills. I'm also bringing in the Httpful library (which is the RESTful API that I use for all of these applications).

// We need loads of memory, and this is an alternative to putting it into php.ini
ini_set("memory_limit","2048M");

include('./httpful.phar');

Next I set up the server and the first of two method URIs that I need for this application.

$ppserver = 'https://tellura.poolparty.biz';
$uri = $ppserver . '/PoolParty/api/projects';

This URI is used to retrieve a list of all projects to which the user has access.

Next, I sent this uri off to PoolParty; the results came back in the $response variable. You will need to replace [user] and [password] with your own values.

$response = \Httpful\Request::get($uri)
->authenticateWith('[user]', '[password]')
->send();

When you use php for this kind of application, the response comes back in the form of an array of php objects, which I've unpacked here in a couple of steps (you could package this up into a single command, but I wanted to show the steps).

$projects_array = (array)$response;
// Get the "body" member out of the array
$projects = $projects_array["body"];

So I cast the $response object to an array, and then extracted the body member of that array. The body member contains the stream of data that makes up the list of projects.

Next, I had to work through that list of projects, and for each project carry out the necessary processing. I'm using the php foreach command here.

foreach ($projects as $project) {
$project_data = (array)$project;
$processedProject = processProject($project_data['title'], $project_data['id']);
echo "Successfully processed and saved project " . $project_data['title'] . " to " . $processedProject . "</strong><br />";
}

For each of the projects in the list, I cast the object to an array just as before, then extracted the title and id members from that array. Then I sent those off as parameters to the processProject function (see below). The results from that function came back as a message confirming the export. This could be improved in the hands of a competent programmer, but it works.

Since the program runs this function and gets back a message for each project in turn, I ended up with a page of responses like the one at the top of the page.

Let's go through the processProject function now. The function takes two incoming parameters; the project title and the project UUID. We need the first to define the filename and the second to retrieve the data from PoolParty.

First, I set up a timestamp - this will be added to the filename (obviously, this is optional but I like to have timestamps). Then I modified the project title, making it lowercase and removing spaces and special characters. Then I defined the URI for the REST command (we're using pparExport here - see here for the documentation for this method).

function processProject($project_title, $project_uuid) {
// create a new timestamp
$x = date_default_timezone_set ( 'GB' );
$timestamp = date('YmdHis');

// Make the filename lowercase and remove spaces
$mod_project_title = preg_replace("/[^A-Za-z0-9]/", "", strtolower($project_title));

// define the uri to retrieve the project data
$uri = "https://tellura.poolparty.biz/PoolParty/api/projects/" . $project_uuid . "/pparExport";

Next I sent off the request; note that I am specifying that I expect a zip file as the response. This is because the pparExport method returns data with a MIME type of application/zip. As above I cast the response object to an array, and then extracted the body member of that array.

$response = \Httpful\Request::get($uri)
->expects('application/zip')
->authenticateWith('[user]', '[password]')
->send();
$response_array = (array)$response;
$file_contents = $response_array["body"];

In this case the body member contains the stream of data that makes up the .ppar file. So I needed to put this data into a file and write it out.

$outputpath = "/tmp/";
$outputfilename = "PoolParty_project_" . $mod_project_title . "_" . $timestamp . ".ppar";
$retvalue = file_put_contents($outputpath . $outputfilename, $file_contents);
$processedProjectInformation = $outputpath . $outputfilename . ": file size = " . $retvalue . " bytes.";
return $processedProjectInformation;

First I defined the location on disk for the file. This will depend on the OS that you're using, of course. As I work on a Mac I chose to use /tmp as my location, as any application can write to this. I found that I had permissions problems when I tried to write it elsewhere; once again, a competent programmer could improve this, and on Windows I think the restrictions on where you can write data are fewer. 

Next, the file_put_contents command writes out the file to disk. The $retval response is either the file size in bytes of the resulting file, or an error message if the file write fails. Finally, I created a return message and returned that to the code that called processProject.

OK, that's it for this short article. I hope you find this utility useful for your PoolParty server.

Application source code

This listing has all of the code needed to run the application, with comments. I called this program backupprojects.php, but you can call it what you like (you will need the .php).

<?php
/*
 #################################
  This program retrieves a list of projects from a PoolParty repository
  and then retrieves the project archive (.ppar) file for each.
  Tested with the RESTful APIs of PoolParty v7.2
  Ian Piper
  Tellura Information Services
  20210106
 #################################
*/

  // We need loads of memory, and this is an alternative to putting it into php.ini
  ini_set("memory_limit","2048M");
  // include the restful client library. If the httpful.phar file isn't in the folder with this script, download it
 include('./httpful.phar');  
  // This application does two calls to the PoolParty API. The first retrieves a list of all projects to which the user
  // has access, and the second loops over all of the projects by UUID and exports the project in ppar (=zip) format.
  // Finally it saves each file to a defined filename and location.
  // Note that it can take a long time. I have processed 90 projects with this application and it takes around 20 min.
 // Set the PoolParty repository - insert the server host where indicated below
 $ppserver = '[server]';
 $uri = $ppserver . '/PoolParty/api/projects';
 // Send off the request; the program will only return projects to which the user has access.
 // It may be best to use a special api user with no roles but membership of all necessary groups
 // See the developer docs: https://help.poolparty.biz/doc/administrator-guide/poolparty-administration/poolparty-user-administration/setup-an-api-user-for-poolparty
 $response = \Httpful\Request::get($uri)
   ->authenticateWith('[user]', '[password]')
     ->send();
 // We have a response object as a json package. Now need to unpack it to 
 // nice php arrays and variables that we can do something useful with!
 // Cast the response object to an array
 $projects_array = (array)$response;
 // Get the "body" member out of the array
 $projects = $projects_array["body"];
  // Now process the data, outputting the filename for each project
  // and obviously writing out the file
  foreach ($projects as $project) {
    // The project data is in a PHP object, so cast it to an array
    $project_data = (array)$project;
    // process the project data
    $processedProject = processProject($project_data['title'], $project_data['id']);
    // report the results
    echo "Successfully processed and saved project " . $project_data['title'] . " to " . $processedProject . "</strong><br />";
  }
  /*
 ########################################################
  processProject 
  This function uses the pparExport API function
  to retrieve the ppar file for the project.
  It uses the project_data['id'] array member to provide the identifier for that project
  It creates a filename from the project_data['title'] array member
  Then it saves the file.
 ########################################################
*/
  function processProject($project_title, $project_uuid) {
    // create a new timestamp
    $x = date_default_timezone_set ( 'GB' ); 
    $timestamp = date('YmdHis');
    // Make the filename lowercase and remove spaces
    $mod_project_title = preg_replace("/[^A-Za-z0-9]/", "", strtolower($project_title));
    // define the uri to retrieve the project data
    $uri = "https://[server]/PoolParty/api/projects/" . $project_uuid . "/pparExport";
    // Run the request and pick up the response
    $response = \Httpful\Request::get($uri)
        ->expects('application/zip')
        ->authenticateWith('[user]', '[password]')
        ->send();
    // We now have a response object. Cast it to an array
    $response_array = (array)$response; 
    // Extract the body element of the array (which should be the raw zip file content)
    $file_contents = $response_array["body"]; 
    // Write the project ppar file out to disk
    // Note that the outputpath location is OS specific. On a Mac I use /tmp as an output location because
    // I had problems with permissions when writing anywhere else. Any user can write to /tmp
    $outputpath = "/tmp/";
    $outputfilename = "PoolParty_project_" . $mod_project_title . "_" . $timestamp . ".ppar";
    $retvalue = file_put_contents($outputpath . $outputfilename, $file_contents);
    $processedProjectInformation = $outputpath . $outputfilename . ": file size = " . $retvalue . " bytes.";
    return $processedProjectInformation;
  } // end function processProject

  ?>