How to Recover Lost ServiceNow Data

A regular backup of your ServiceNow data is critical. In the previous post we have described why it was necessary to perform independent backups of individual tables using tools such as SnowMirror Backup. While knowing you have regular snapshots of your ServiceNow data, the real challenge can be restoring that data and relationships in the event of a data loss. Now, let’s look at the options that we have to restore tables from backups.

If you want to restore a table from backups then you start with the latest (or target) full backup and then you restore the latest differential backup. If the incremental backups are being used then you have to restore all increments since the last full backup.

SnowMirror is able to backup ServiceNow tables into two formats: CSV and native ServiceNow XML. These formats are the same or very similar to those created by ServiceNow export features. Depending on a backup format the restore strategies are different.

Example Data Model

In this example we have a very simple data model consisting of two tables Employee and Department. They both have the Name field and each employee has a reference to a department.

Native ServiceNow XML

Using the native XML format for backups and then the restore process is the most straightforward way and it has many benefits. On the other hand, it is a low level ServiceNow feature which does not allow any mistakes. Incautious data upload might create even more damage to the ServiceNow data.

The benefits are clear:

1. The restore procedure is extremely simple.
2. Existing records are updated and new records stay untouched. The missing are restored of course.
3. The sys_ids remain the same which is very good for referential integrity.
4. References can be restored in any order.

This is how the table looked like before the restoration started. Please note the record Radek which was not in the backup file, missing record Jana and an updated record Pavel:

After executing the Import XML command the table is restored including all references. Please note that Radek stayed in the table, Pavel got updated and Jana is restored:

CSV

Restoring data from a CSV file is a bit more complicated, on the other hand, the whole process is under your full control and it is easier to adjust the data to your needs. The ServiceNow Import Sets have many features and they are well described in the documentation.

Here is a list of steps and hints to make the restore easier.

1. Create a new Data Source from the CSV file.
2. Create a Transform Map for the table to restore.
3. Use the Auto Map Matching Fields feature to map all fields from CSV to the target table
4. Select a coalesce field to make the updates work. Either a unique field (e.g. incident number) or a sys_id. See below.

Updating existing records during the recovery is vital otherwise the import set creates new records instead, Using a unique field to coalesce the records is not always possible or it is not the best way. If you want use sys_id for this purpose then it is necessary to add this mapping to the field map manually by mapping the source u_sys_id to the target sys_id and making them as coalescing values.

This technique ensures that updates will be executed correctly, however, the inserts will be created with generated sys_ids and not with the sys_ids from the backup file, which might break relationships with other tables. To ensure the inserted records get a sys_id from the file create an onBefore transform script like this:

Restoring relationships (references to other records) with CSV is much more complicated than with the XML format. In the ideal world the correct restore procedure is to start from the tables that are being referenced and then continue with the rest. But not all of the data model are trees.

By default, CSVs contain display values of referenced records and the transform process finds the references using display values. However, what if the referred record is not there (yet). ServiceNow would either create it or cancel (ignore) the relationship. This is configurable. None of these options are ideal for the recovery process.

SnowMirror Backup allows you to include sys_ids of the references into the CSV backups which is not available in the standard ServiceNow exports. It helps with record identification but not with the missing references. This has to be resolved by custom scripts inserting reference sys_ids even for missing references. In this case the order of restored tables does not matter.

Download a Free Trial

Webinar: How to Backup ServiceNow Data