Part 2: SharePoint Replication

June 06, 2014

In this post we will continue our discussion (Part I: SharePoint Replication) on a solution we developed for replicating SharePoint over satellite communications. This post will cover the replication process and how we detect conflicts using some logic borrowed from the Microsoft Sync Framework.

Replication, Conflicts, and how we borrowed from the MS Sync Framework

For our replication to work, we needed to create a few site columns in SharePoint which would hold metadata associated to the replication process. Below is a list of the site columns we used for the replication process and a brief description of the columns functionality.

Name

Type

Use

ReplicaItemID

GUID

Provides a unique identifier for the item within the replication

ReplicaCreateSiteID

GUID

Provides a unique identifier for the SharePoint site that created the item

ReplicaCreateVersion

Integer

Provides an incrementing version number for this item (n+1)

ReplicaUpdateSiteID

GUID

Provides a unique identifier for the SharePoint site that updated the item

ReplicaUpdateVersion

Integer

Provides an incrementing version number for this item (n+1)

ReplicaTombstoned

Boolean

Provides a flag indicating the item should be deleted

ReplicaConflictID

GUID

Provides a unique identifier for items that have a replication conflict

ReplicaConflictPath

String

Provides a path to the SharePoint List or Library for items that have a replication conflict

Using these site columns, we extended some of the base SharePoint Content Types into “Replicated” Content Types. We also added XML elements to the Content Type definitions which wired up our Replication Event Receiver. With the Content Type inheritance model in SharePoint, this meant that any Content Type derived from our “Replicated” Content Types will trigger calls to methods in our Replication Event Receiver.

We needed the Replication Event Receiver to fire on 3 types of SharePoint events, ItemAdded, ItemUpdated, and ItemDeleting. By default, Event Receivers tied to ItemAdded and ItemUpdated are asynchronous. We needed these events to fire synchronously so we could give the users feedback if the replication failed. To do that, we set the synchronization value on the event to be synchronous.     

<Synchronization>Synchronous</Synchronization>

Our next step is to create a generic way to send the columns associated to the “Replicated” Content Type. To do this, we created a serializable class which we could convert to and from an XML file using System.XML.Serialization. We package the XML file with any attachments associated to the item into a .zip file in the DFS-R share named using the GUID from the ReplicaItemID column. When items are updated or deleted, we open the matching .zip in the DFS-R share and change the contents. This allows DFS-R to use Remote Differential Compression (RDC) to send only the differences on a change.

Our replication process will use the site columns to detect replication conflicts and inform the system administrators so a correction can be made. We have a few scenarios which our process will handle, below is a list of them and an explanation of how the site columns will be used to detect the conflicts.

​Example of item create


Replica ItemID

Replica CreateSite ID

ReplicaCreateVersion

ReplicaUpdate Site ID

Replica UpdateVersion

ReplicaTombstoned

Item 1

Server 1

1

Server 1

1

False

Item 2

Server 2

1

Server 2

1

False

Item 3

Server 1

1

Server 1

1

False

  • Items 1 and 3 were created on Server 1
  • Item 2 was created on Server 2

Example of item update

Replica ItemID

Replica CreateSite ID

ReplicaCreateVersion

Replica UpdateSite ID

ReplicaUpdateVersion

ReplicaTombstoned

Item 1

Server 1

1

Server 2

2

False

Item 2

Server 2

1

Server 2

2

False

Item 3

Server 1

1

Server 1

1

False

  • Item 1 and 2 were updated on Server 2

Example of item delete

Replica ItemID

Replica CreateSite ID

ReplicaCreateVersion

Replica UpdateSite ID

ReplicaUpdateVersion

ReplicaTombstoned

Item 1

Server 1

1

Server 2

2

False

Item 2

Server 2

1

Server 2

2

False

Item 3

Server 1

1

Server 1

2

True

  • Item 3 was deleted on Server 1

Example of item conflict

DFS-R XML Files

Replica ItemID

Replica CreateSite ID

ReplicaCreateVersion

Replica UpdateSite ID

ReplicaUpdateVersion

ReplicaTombstoned

Item 1

Server 1

1

Server 2

2

False

Item 2

Server 2

1

Server 1

3

False

Item 3

Server 1

1

Server 1

2

True

SharePoint Items

Replica ItemID

Replica CreateSite ID

ReplicaCreateVersion

Replica UpdateSite ID

ReplicaUpdateVersion

ReplicaTombstoned

Item 1

Server 1

1

Server 2

2

False

Item 2

Server 2

1

Server 2

3

False

Item 3

Server 1

1

Server 2

2

False

  • Item 2 was updated on Server 1 without Server 2’s knowledge and is in conflict
  • Item 3 was deleted on Server 1 without Server 2’s knowledge and is in conflict

When a “Replicated” item is added or updated in SharePoint, we first look at the DFS-R share for a .zip file with the same ReplicaItemID. If we don’t find one, we can assume the item is new to the replication so we can skip the conflict detection. If we do find a .zip file, we open the XML file containing all the site column data and begin to compare the ReplicaCreateSiteID, ReplicaCreateVersion, ReplicaUpdateSiteID, and ReplicaUpdateVersion values. If the SharePoint item ReplicaUpdateVersion is less than the ReplicateUpdateVersion of the XML file, we have a conflict. The same is true if the ReplicaUpdateVersions are the same but the ReplicaUpdateSiteIDs are different. Our conflict detection logic is similar to the Microsoft Sync Framework.

Along with the Replication Event Receiver, we created a Windows Service to import the items from the DFS-R share into SharePoint. We could have used a custom timer job in SharePoint but it isn’t really intended for long running processes. In the Windows Service, we attach a FileSystemWatcher to monitor the DFS-R share. As files are created or changed, the FileSystemWatcher will fire events which our replication process will subscribe to and import the item into SharePoint.

Wrapping it up

So that’s it, a simple way to replicate SharePoint content using DFS-R. Our client has been extremely happy with the result. They currently have 40+ drilling platforms around the world with up-to-date policies and procedures.


Information and material in our blog posts are provided "as is" with no warranties either expressed or implied. Each post is an individual expression of our Sparkies. Should you identify any such content that is harmful, malicious, sensitive or unnecessary, please contact marketing@sparkhound.com.

Meet Sparkhound

Review our capabilities and services, meet the leadership team, see our valued partnerships, and read about the hardware we've earned.

Learn How We Work

See how our Plan/Build/Run methodology drives real client success, and gain our team's perspectives on timely tech topics.

Engage With Us

Get in touch any of our offices, or checkout our open career positions and consider joining Sparkhound's dynamic team.