A regular backup of your ServiceNow data is critical. In a previous post, we described why it is necessary to perform independent backups of individual tables using tools such as SnowMirror. Once you know that you have regular snapshots of your ServiceNow data, the real challenge can be to restore data and relationships in the event of a data loss. Now let’s take a look at the options that we have to restore tables from backups.
If you want to restore a table from backups, you should start with the latest (or target) full backup and then restore the latest differential backup. If the incremental backups are being used, then you have to restore all increments since the last full backup.
SnowMirror is able to back up ServiceNow tables into two formats: CSV and native ServiceNow XML. These formats are the same or very similar to those created in ServiceNow’s export features. Depending on the backup format, the restore strategies are different.
Example Data Model
In this example we have a very simple data model consisting of two tables, Employee and Department. They both have the Name field and each employee has a reference to a department.
Native ServiceNow XML
Using ServiceNow’s native XML format for backups and then the restore process is the most straightforward solution, and it has many benefits. On the other hand, it is a low level ServiceNow feature which does not allow for any mistakes. If you’re not careful when uploading the data, you might cause even more damage to your ServiceNow data.
The benefits are clear:
- The restore procedure is extremely simple.
- Existing records are updated and new records stay untouched. Of course the missing records are restored.
- The sys_ids remain the same, which is very good for referential integrity.
- References can be restored in any order.
Here’s what the table looked like before the restoration started. Please note the record Radek which was not in the backup file, missing record Jana and an updated record Pavel:
After executing the Import XML command, the table is restored including all references. Please note that Radek stayed in the table, Pavel was updated and Jana was restored:
CSV
Restoring data from a CSV file is a bit more complicated, but on the other hand, the whole process is under your full control and it’s easier to adjust the data to your needs. The ServiceNow Import Sets have several features and they are well described in the documentation.
Here’s a list of steps and hints to make the restore easier.
- Create a new Data Source from the CSV file.
- Create a Transform Map for the table to restore.
- Use the Auto Map Matching Fields feature to map all fields from CSV to the target table
- Select a coalesce field to make the updates work. Either a unique field (e.g. incident number) or a sys_id. See below.
Updating existing records during the recovery is vital. If you don’t, the import set will instead create new records. Using a unique field to coalesce the records is not always possible or it is not the best way. If you want to use sys_id for this purpose then it is necessary to add this mapping to the field map manually by mapping the source u_sys_id to the target sys_id and making them into coalescing values.
This technique ensures that updates will be executed correctly, but the inserts will be created with generated sys_ids and not with the sys_ids from the backup file, which might break relationships with other tables. To ensure the inserted records get a sys_id from the file create an onBefore transform script like this:
Restoring relationships (references to other records) with CSV is much more complicated than with the XML format. In an ideal world, the right restore procedure would be to start from the tables that are being referenced and then continue with the rest. But not all data models are trees.
By default, CSVs contain display values of referenced records and the transform process finds the references using display values. But what if the referred record is not there (yet)? ServiceNow would either create it or cancel (ignore) the relationship. This is configurable. None of these options are ideal for the recovery process.
SnowMirror allows you to include sys_ids of the references into the CSV backups which is not available in the standard ServiceNow exports. It helps with record identification but not with the missing references. This has to be resolved by custom scripts inserting reference sys_ids even for missing references. In this case, the order of restored tables does not matter.
If you’d like to to talk with us about how SnowMirror can help your business get the most out of ServiceNow.
