The new Consistency Check feature is very useful and very popular functionality in SnowMirror 3.2. It compares the number of records on the ServiceNow side with the number of rows in the mirror database after each replication run. If the numbers are different then the synchronization raises a warning which might trigger an email notification. There are several frequently asked questions and most of them about what to do when SnowMirror claims that there is an inconsistency between ServiceNow and the mirror database. This article covers most of the common cases.
Q: Why is the Consistency Check disabled?
The new Consistency Check feature is disabled by default for all upgrading users to prevent unwanted warnings in the existing production environments. Backwards compatibility reasons. For all new installations starting with SnowMirror 3.2 the new feature is enabled by default. To enable the feature manually go to Settings -> General Settings and configure it in the Consistency Check section. This new feature requires the Aggregate Web Service Plug-In to be active on the ServiceNow side.
Q: What if there are records created or deleted during the replication run?
SnowMirror algorithm is ready for such situations. It does not compare only two numbers but it tests the actual mirror table row count against an interval which includes updated and deleted records modified since the run started.
Q: I enabled Consistency Check and now I have many warnings.
The best approach for existing deployments is the following:
- Enable the consistency check
- Wait until all the synchronizations perform their scheduled runs or run Synchronize manually
- For all synchronizations with warnings run Differential Synchronization or Clean&Sync to fix all issues from previous runs
- Handle all the remaining and emerging warnings one by one (see following FAQs)
Q: How can I get more details about the warning?
Go to the activity log of the replication run. The consistency check procedure writes the last lines of the log. One of the last lines looks like the following:
INFO: 2016-04-15 02:18:10 - Check: ServiceNow: 845,851, Mirror DB: 1,016,669. Upserted: 11,098, Deleted: 0, Allowed interval: <1016669 ; 1027767>
By reading this log line and the lines before, you can figure out if there are records missing on the mirror side or if you have some orphans in it.
Q: Some records are missing what shall I do?
In this case, the mirror table does not contain all the records from ServiceNow. There are two cases with different solution approaches:
- The warning happens right after a full load. Then it means SnowMirror is not able to download all the records from ServiceNow even if replicating the table from scratch. In this case, the usual suspect is ACLs (permissions). The user account being used for SnowMirror does not have access to all the records in the table. Please note that even the admin account might be limited by some ACLs or business rules. Please review the settings on the ServiceNow side.
- The warning had appeared after an incremental load. This case requires some investigation to find records that are missing. One of the ways is to create a second synchronization for the same table (e.g. incident2), perform a full load and then compare the tables. For example by using a union of left joins. Then analyze the missing records, especially by reading the sys_updated_on field. A typical cause is that a system administrator inserted the record using the Import XML feature in ServiceNow and the sys_updated_on timestamp was not updated and thus the record is being missed by the incremental load. Fix by running the Differential Sync or update the timestamps to the current time.
Q: The mirror table contains more records than the ServiceNow table.
As in the previous case. There can be more causes for it.
- The table is not audited but the synchronization uses Audit Delete Strategy. So the deleted records are not tracked by SnowMirror.
- It is some weird system table. Especially tables starting with v_ are virtual tables and behave in strange ways.
- The table is being rotated by ServiceNow – transactions table, history lines, tables with 0001 number in the name, etc.
- Archiving plug-in is active in the instance
