Repository navigation
Do not request deleted records during the initial offline pull #576
Description
Is your feature request related to a problem?
Yes. During the initial pull of an offline table, the client currently sends the __includedeleted=true query parameter even though the local table is expected to contain no previously synchronized records.
This can significantly slow down the initial synchronization when the remote table contains a large number of soft-deleted records. Those records are downloaded even though they cannot result in any local deletion during a normal initial pull.
The behavior originates in PullOperationManager.PrepareQueryDescription(), which currently adds __includedeleted=true regardless of whether a delta token, represented by lastSynchronization, exists.
Describe the solution you'd like
Only request deleted records when the pull is an incremental synchronization and a previous synchronization timestamp exists.
For an initial pull, where lastSynchronization is zero, the query should not include __includedeleted=true.
Deleted records are required during incremental pulls so that records deleted remotely can also be removed from the local database. During a normal initial pull, however, there are no previously synchronized local records to remove.
A possible implementation would be:
/// <summary>
/// Prepares the query description for use as a pull request.
/// </summary>
/// <param name="source">The query description to modify.</param>
/// <param name="lastSynchronization">The last synchronization date/time.</param>
/// <returns>A modified query description for the actual query.</returns>
internal static QueryDescription PrepareQueryDescription(
QueryDescription source,
DateTimeOffset lastSynchronization)
{
QueryDescription query = new(source);
if (lastSynchronization.ToUnixTimeMilliseconds() > 0L)
{
BinaryOperatorNode deltaTokenFilter = new(BinaryOperatorKind.GreaterThan)
{
LeftOperand = new MemberAccessNode(null, "updatedAt"),
RightOperand = new ConstantNode(lastSynchronization)
};
query.Filter = query.Filter is null
? deltaTokenFilter
: new BinaryOperatorNode(
BinaryOperatorKind.And,
query.Filter,
deltaTokenFilter);
// Delta pulls need deleted records so that local copies can be removed.
query.QueryParameters[ODataQueryParameters.IncludeDeleted] = "true";
}
else
{
// Initial pull: no previously synchronized local records need to be removed.
_ = query.QueryParameters.Remove(
ODataQueryParameters.IncludeDeleted);
}
query.RequestTotalCount = true;
query.Top = null;
query.Skip = 0;
query.Ordering.Clear();
query.Ordering.Add(
new OrderByNode(
new MemberAccessNode(null, "updatedAt"),
true));
return query;
}
Activity