Improving the Team Synchronize support
This document outlines how other plug-ins can participate in the Synchronize
View and how the Synchronize View is being refactored for Eclipse 3.0.
Interested parties should review this document and verify that their
use-cases are addressed and that the solution satisfies their needs.
The work on the Sychronize View is being done to support the following
3.0 plan items:
- 37705 Improve
In addition to supporting a generic headless Team API numerous plug-ins
are also interested in using the Synchronize technology. Currently
the CVS plugin uses this support but we haven't officialy made it
available for others. We feel that this is a complicated piece that
could be re-used by many other Team plug-ins to provide rich integration
of their tool into Eclipse.
- 36957 Support
The Synchronize operation is an excellent candidate for running in
the background. Some changes are required to the sync model to accomodate
concurrent calculation of resource synchronization.
This document doesn't cover JSR-147 or CVS concurrent operations.
Last Modified: February 24th, 2004
As stated above, there are two motivating reasons for the 3.0 work
relating to the Team Synchronize APIs: (1) to enhance workflows, (2)
to provide an API for other plug-ins to use. Note that synchronization
support doesn't exclusively imply that the Synchronize View is used
to display and manipulate the synchronization states. Instead, it is
our intention to provide a framework for creating, managing, and then
finally displaying this synchronization state. The presentation of the
synchronize states can be flexible, and could be then shown in either
the Synchronize View, or in other more appropriate places (e.g. editors,
This is a key for Team plug-ins for rich integration into Eclipse.
The motivations for the proposed workflow changes changes are described
Dynamic syncrhonization model support
One of the most important changes we are proposing is to migrate from
a static sync model to a dynamic one. Previously, a user would synchronize
a set of resources with their remotes and subsequent changes made in
the workspace would only be reflected in the synchronization state shown
in the view if the user explicitly re-synchronized. This also meant
that you could not browse local changes without running a potentialy
long operation. This static model also caused problems if we wanted
to refresh the remote state in the background and incrementally update
the sync model as remote changes were found.
Synchronize View re-use
The view should support showing multiple types of synchronization targets.
The old sync view was re-used by clients but the synchronization state
of the previous type was lost. For example, synchronizing with a Dav
server then again with CVS. We are currently evaluating ways of providing
a view that shows different contexts. Some of the options are to have
a dropdown to select the different subscribers or to have a tabbed view.
I like the drop down because it takes less screen real estate and you
don't have the tab problem of truncating the titles.
Flexible Presentation of Synchronization state
It should be possible to display synchronization states of local resources
in logical models that don't necessarily mirror the physical hierarchical
file system layout. In addition, it should be possible to display this
model to the user in other contexts than in the Synchronize View itself
(e.g. editor, embeded compare editor, wizard, dialog).
The synchronization UI should be responsive and support incremental updating
that won't interrupt the users workflows. For example, when changes are
made either to local workspace resources or to the sync model the changes
should be processed asynchronously.
Background fetching of remote state
Currently when a project is synchronized the complete remote tree is
fetched and a progress dialog is shown. The user can't perform any other
operation while the remote tree is fetched. There is no reason why this
couldn't be fetched in the backgroup without blocking the UI and would
allow an awareness of what other people in the team are working on and
help reduce the time needed to find out what others are doing.
The terminology and look and feel of the CVS merge editor, compare
editors, and sync view is different. Basically a merge is the same as
synchronizing except for the fact that you can't commit changes. Merging
is already a tricky operation and having a conflicting usage model between
the sync and merge makes it even harder. (35577)
The comparison criteria that is used to calculate the sync state is
hidden from the user and it isn't obvious for them when the criteria
should be changed and what advantage it will have. This is also seen
when a user synchronizes a project against an existing project on the
server and notices that most files are conflicting. There is no easy
steps for calculating the 'real' changes. This may be solved by individual
Team plug-ins, but it is worth noting that it is a problem encountered
in the Synchronize View. (24887, 21612, 28143)
Splitting the Synchronize View from the Compare components
We would like to split the Synchronize view from the compare components
and have a dedicated view that shows synchronization state in different
forms and then allow the user to browse the changes using the compare
editors. Here is a list of motivating reasons for the split:
- Would like to use real editors for merging instead of the
neutered ones currently used. For example, the java editor
shown in the sync view looks real (e.g. syntax coloring) but is a
scaled down java editor. This causes integration and context problems
because it is very useful to try and write code in these editors without
resorting to opening the file with the real java editor. Then going
back to the sync view only to find that the file hasn't refreshed.
- We wanted more flexibility in showing resource synchronization in
different ways: table, tree, tabletree, split view with statistics....
Currently the compare DiffTreeViewer subclasses must be a tree and
this limits our implementation flexibility.
- The edit/save workflow in a view has always been strange. You couldn't
edit, look at another file, then return to your edit without being
forced to save the change first. This is not intuitive behavior, and
nowhere else in the platform are your forced to save changes just
to see another file.
- It made sense to use a perspective to accomplish what the old sync
view did by showing all related views (structure diff, structure compare,
merge viewers) together.
Show helpful change counts
It would be helpful for the user to understand the scope of changes
that are remote (incoming) and local (outgoing) regardless of the filters
being applied to the changes. This would help cases were users forget
changes because they have been filtered out of the view.
Because the Synchronize view can show the changes for all resources
in the workspace that are configured with the current synchronization
target the user can use working sets to scale the list of projects to
synchronize. When a working set is active the change counts change to
show both the counts in the current working set and those in the entire
workspace. This can help decide when to disable a working set.
The CVS commands that used to prime the Synchronize View now use
a dynamically created working set to automatically reduce the list of
projects in the view. For example, when sharing a project for the first
time or synchronizing via the Team > Synchronize with Remote menu,
a working set is created.
Conflicts are propagated to parent folders
To ensure that conflicts are never ignored, the conflict icon is propagated
to parent folders. This will also allow users to quickly find the conflicts.
General usability enhancements
The structure compare part of the 2.1 Synchronize View has many usability
problems that we would like to improve.
- We propose that synchronization state be displayed in different
formats (table, compressed tree) so that things you are interested
in (the leaves of the tree) are not constantly separated by a great
distance. This would make it easier to see at a glance what has changed.
It also makes multi-selection operations less painful, for example
comitting a set of related files that should have the same comment.
- There are several also several expansion/selection usability problems
that we would like to fix. (35187, 33041, 4931, 5431, 20847, 27237, 16249)
- Synchronize view modes (filtering) and conflict workflows must be
improved. This is somewhat related to showing helpful change counts.
The Synchronize API
The API is composed of both core and ui components. Firstly, the core
building blocks for synchronization is the management of synchronization
state of local resources. Synchronization state is usually modeled as
the change between a local resource and one or more variants of that resource. Thus, the core
APIs are focused on managing variants (e.g. given a local resource what
is the variant used to calculate synchronization state) and how synchronization
state is calculated given a local resource and associated variants.
Given the physical representation of resource variants then the UI aspect
of the API allow you to control how the model is presented to the user.
Since the core synchronization classes are event driven, you can build
synchronization UIs that optionally update dynamically. Integration into
the Synchronize View is possible by using the synchronize view extension
points and using the synchronize APIs to and possibly re-usable UIs to
build the synchronize page.
The interesting API packages are:
This package specifies the API for managing the synchronization state
between the local workspace resources and a corresponding variants of
those resources. The classes in this package can be used by Subscribers
(see the org.eclipse.team.core.subscribers package) or others. The classes
are roughly divided into three categories:
- describing the synchronization state of a one or more resources,
- notifying interested parties of changes in the synchronization state.
- filtering a set of resource based on a sync state criteria
This package specifies the API for Team subscribers. A Subscriber provides
access to the synchronization state between the local workspace resources
and a set of variants of those resources, whether it be a code repository
or some other type of server (e.g. FTP). A subscriber is typically associated
with only a subset of the resources in the local workspace, referred
to as the set of resources the subscriber supervises. The supervised
local resources have a corresponding variant state which describes the
state of the variant resources that correspond to the local resources.
A Subscriber provides:
- a set of root resources that define the subset of resources in the
workspace that the subscriber supervises (some children of the roots
may not be supervised, as indicated by the isSupervised method).
- access to the synchronization state (using SyncInfo) between the
resources it supervises and their corresponding variant resources.
- the ability to refresh the the remote state. This implies that
the Subscriber would be responsible for caching variants.
- change notification to registered listeners (of type ISubscriberChangeListener)
when the variant state changes or when roots are added or removed.
The packaage specifies a set of classes and interfaces to support participating
in the Synchronize View. Synchronize participants are declared by extending
the synchronizeParticipants extension point. There
are two classes of synchronize participants: static participants
will be created when the synchronize view is created, and dynamic
participants that are created by user code at some other time. A synchronize
manager (ISynchronizeManager) manages all active synchronize
participants, and provides notification of participants which are added
and removed. Participants are displayed in a page book view. Each participant
implementation is reponsible for creating its page (IPageBookView),
which provides freedom of presentation to the synchronize view implementation.
A single participant may be displayed simultaneously in multiple synchronize
views, and in different workbench windows.
This package contains classes that support displaying synchronization
information described by SyncInfo. If you consider the synchronization
model described by the classes in org.eclipse.team.core.synchronize
as the physical representation of resource variants then the classes
in this package allow you to control how the model is presented to the
user. One key feature of these APIs is that they support managing dynamic
There is no specific support for the Synchronize View in this package.
Instead, the classes here can be used to build a page that can be shown
in the view, but they can also be used to show synchronization information
in dialogs, wizards, and editors.
This package contains a classes that provides an implementation of a synchronize
participant that enables synchronization for a Subscriber. For
providers that implement a Subscriber, this is the
easiest method of integrating into the Synchronize View. The TeamSubscriberParticipant
provides a view of changes (incoming, outgoing, conflicting), modes
for showing only a subset of the changes, decorations for identifying
the changes, and working sets.
This section describes how to make use of the Synchronize API including
how to integrate into the Team Synchronize view. It is divided into
two parts. The first describes how to implement a Subscriber, either
from scratch or by building on top of an existing local workspace synchronization
infrastructure. The second part describes how to make use of the UI
components of the Synchronize API for showing Subscriber state in the
Team Synchronize view and other views, editors and dialogs.
Implementing a Subscriber
This section presents a few different perspectives on implementing
a subscriber. First, the general anatomy of a subscriber is discussed,
followed by a few examples that build on API provided in the org.eclipse.team.core
The Anatomy of a Subscriber
A subscriber provides access to the synchronization state between the
local workspace and another location that mirrors the local workspace
resources. This other location is usually, but not necessarily, on a
different machine such as a server. In order to describe what a subscriber
is, we must first present some of the terminoligy we are going to use:
Resource Variant: A local resource that is mapped to a resource
that exists at another location can be referred to as a variant of
that resource. That is, the resources are usually very similar but
may differ slightly (either due to modificatons to the local resource
or changes made the remote copy made by other users). We take a local
workspace centric view of this, referring to the local copy as the
resource and any remote copy as resource variants.
Two-way vs. Three-way Synchronization: There are two basic
types of synchronization state determination: two-way and three-way.
A two-way comparison only considers the local resource and a single
resource variant, referred to as the remote resource variant. This
type of comparison can only show the differences between the two resources
but cannot offer hints as to how the changes interelate. Most code
repository systems support a three-way comparison for synchronization
state determination. This type of comparision involves the local resource,
a remote resource variant and a base resource variant. The base resource
variant represents a common ancestor for the local and remote resources.
This allows for more sophisticated synchronization states that indicate
the direction of the change.
A subscriber can then be described as providing access to the synchronization
state between the resources in the local workspace and a set of resource
variants for these resources using either a two-way or three-way comparison,
depending on the nature of the subscriber. A subscriber provides the
local workspace traversal: a subscriber supports
the traversal of the local workspace resources that are supervised
by the subscriber. As such, the subscriber has a set of root resources
that define the workspace subtrees under the subscriber's control,
as well as a members method that returns the supervised members
of a workspace resource.
resource synchronization state determination: For
supervised resources, the subscriber provides access to the synchronization
state of the resource, including access to the variants of the resource.
For each supervised resource, the subscriber provides a SyncInfo
object that contains the synchronization state and the variants used
to determine the state. The subscriber also provides an IResourceVariantComparator
which determines whether two-way or three-way comparison is to be
used and provides the logic used by the SyncInfo to comparing
resource variants when determining the synchronization state.
refresh of synchronization state and change notification:
Clients can react to changes that happen to local resources by listening
to the Core resource deltas. When a local resource is changed, the
synchronization state of the resource can then be re-obtained from
the subscriber. However, clients must explicilty query the server
to know if there are changes to the resource variants. For subscribers,
this process is broken up into two parts. A client can explicitly
refresh a subscriber. In response the subscriber will obtain
the latest state of the resource variants from the remote location
and fire synchronization state change events for any resource variants
that have changed. The change notification is separate from the refresh
since there may be other operations that contact the remote location
and obtain the latest remote state.
As stated above, the purpose of the Subscriber API is to provide access
to the synchronization state between the local workspace and a remote
location. Obviously, underneath the hood, their must be an infrastructure
for managing that state. In general, the synchronization state consists
of the following basic states.
- local resource is synced with the remote: when a resource
is loaded from or written to a remote location, it is consider in-sync
with the remote variant. At this point, the local timestamp and remote
timestamp are recorded to support future synchronization state checks.
- local resource is modified: If the recorded in-sync local
modification timestamp differs from the actual timestamp, the resource
is locally modified.
- remote resource is modified: If the timestamp of the remote
resource variant differs from the recorded remote timestamp, then
the resource has been remotely modified (most likely by a 3rd party).
The infrastructure that maintains the states must persist the timestamps
obtained when a resource is synced.
Implementing a Subscriber From Scratch
When implementing a subscriber from scratch, you can make use of some
helpful API provided in the org.eclipse.team.core plugin. The
org.eclipse.team.core.variants package contains two subclasses
of Subscriber which can be used to simplify implementation.
The first is ResourceVariantTreeSubscriber which will be discussed
in the second example below. The second is a subclass of the first:
ThreeWaySubscriber. This subscriber implementation provides
several helpful classes for managing the synchronization state of a
local workspace. If you do not have any existing infrastructure, this
is a good place to start.
Implementing a subscriber from scratch will be illustrated using our
file system example available in the org.eclipse.team.examples.filesystem
plugin. The code in the following description is kept to a minimum since
it is available from the dev.eclipse.org repository. Although not technicaly
a three-way subscriber, the file system example can still make good
use of this infrastructure. The FTP and WebDav plugins also are built
using this infrastructure.
For the file system example, we already had an implementation of a
RepositoryProvider that associated a local project with a file
system location where the local contents were mirrored. FileSystemSubscriber
was created as a subclass of ThreeWaySubscriber in order to
make use of a ThreeWaySynchronizer to manage workpace synchronization
state. Subsclasses of this class must do the following:
- create a ThreeWaySynchronizer instance to manage the local
workspace synchronization state.
- create an instance of a ThreeWayRemoteTree subclass to provide remote
- The class FileSystemRemoteTree was defined for this purpose
- implement a method to create the resource variant handles used to
calculate the synchronization state.
- The class FileSystemResourceVariant (a subclass of
CachedResourceVariant) was defined for this
- implement the roots method.
- The roots for the subscriber are all the projects mapped to
the FileSystemProvider. Callbacks were added to FileSystemProvider
in order to allow the FileSystemSubscriber to generate change
events when projects are mapped and unmapped.
In addition to the subscriber implementation, the get and put operations
for the file system provider were modified to update the synchronization
state in the ThreeWaySynchronizer. The operations are implemented in
the class org.eclipse.team.examples.filesystem.FileSystemOperations.
ThreeWaySynchronizer manages the synchronization state between the
local workspace and a remote location. It caches and persists the local,
base and remote timestamps in order to support the efficient calculation
of the synchronization state of a resource. It also fires change notifications
to any registered listeners. The ThreeWaySubscriber translates these
change events into the proper format to be sent to listeners registered
with the subscriber.
The ThreeWaySynchronizer makes use of Core scheduling rules and locks
to ensure thread safety and provide change notification batching.
A ThreeWayRemoteTree is a subclass of ResourceVariantTree that is tailored
to the ThreeWaySubscriber. It must be overriden by clients to provide
the mechanism for fetching the remote state from the server. ResourceVariantTree
is discussed in more detail in the next example.
A CachedResourceVariant is a partial implementation of IResourceVariant
that caches any fetched contents for a period of time (currently 1 hour).
This is helpful since the contents may be accessed several times in
a short period of time (for example, to determine the synchronization
state and display the contents in a compare editor).
Building on Top of Existing Workspace Synchronization
Many repository providers may already have a mechanism for managing
their synchronization state (e.g. if they have exisitng plugins). The
ResourceVariantTreeSubscriber and its related classes provide
the ability to build on top of an existing synchronization infrastrucutre.
This is the superclass of all of the CVS subscribers, for instance.
subclasses must provide
base and remote trees
base and remote tree
creation of resource variant handles
Traversal of a resource variant tree
caches resource variant bytes in a ResourceVariantByteStore
can use Persistant or Session stores or can provide a custom implementation
that access the bytes from some other store. For example the ThreeWayRemoteTree
makes use of a byte store implementation that stores the remote bytes
in the ThreeWaySynchronizer.
subclasses must provide code that fetches the current state of the
tree from the server and that creates the resource variant handles
How Does IRemoteSyncElement Relate to Subsciber
Example: synchronizing with a remote file-system
To show how all the APIs comme together we will show you how to create
a file system synchronization support using these APIs.
- Create a subscriber to manage the sync states for the files on the
workspace. for now we will use the timestamp at the time the file
was copied into the workspace from the remote location
- decide how to refresh the variants
- create a participant
- create a set of actions that work on diff nodes