Temporal queries

The latest versions of Hawk have the capability to index every version of all the models in the locations being monitored. To enable this capability, your Hawk index must meet certain conditions:

  • You must be using a time-aware backend (currently, Greycat).
  • You must be using the time-aware updater (TimeAwareModelUpdater) and not the standard one.
  • You must be using the time-aware indexer factory and not the standard one (TimeAwareHawkFactory).
  • You must query the index with a time-aware query language:
  • org.hawk.timeaware.queries.TimeAwareEOLQueryEngine
  • org.hawk.timeaware.queries.TimelineEOLQueryEngine

If you meet these constraints, you can index a SVN repository with models and Hawk will turn the full history of every model into an integrated temporal graph database, or index a workspace/local folder and have Hawk remember the history of every model from then onwards. You will be able to query this temporal graph through an extension of Hawk's EOL dialect.

This functionality was first discussed in our MRT 2018 paper, "Reflecting on the past and the present with temporal graph-based models".

Data model

The usual type -> model element graph in Hawk is extended to give both types and model elements their own histories. The histories are defined as follows:

  • Types are immortal: they are created at the first endpoint in the graph and last to the "end of time" of the graph. There is a new version whenever an instance of the type is created or destroyed.
  • Model elements are created at a certain timepoint, and either survive or are destroyed at another timepoint. Model elements are assumed to have a persistent identity: either its natural/artificial identifier, or its location within the model. New versions are produced when an attribute or a reference changes.

Timepoints are provided by the Hawk connectors, and they tend to be commit timestamps or file timestamps. In SVN, these are commit timestamps to millisecond precision.

Basic history traversal primitives

The actual primitives are quite simple. In the time-aware dialect of Hawk, types and model elements expose the following additional attributes and operations:

  • x.versions: returns the sequence of all versions for x, from newest to oldest
  • x.getVersionsBetween(from, to): versions within a range of timepoints
  • x.getVersionsFrom(from): versions from a timepoint (included)
  • x.getVersionsUpTo(from): versions up to a timepoint (included)
  • x.earliest, x.latest: earliest / latest version
  • x.next, x.prev/x.previous: next / previous version
  • x.time: version timepoint

Temporal assertions

It is possible to evaluate assertions over the history of a type or model element:

  • x.always(version | predicate over version): true if and only if ("iff") the predicate is true for every version of x.
  • x.never(version | predicate over version): true iff the predicate is false for every version of x.
  • x.eventually(version | predicate over version): true iff the predicate is true for some version of x.
  • x.eventuallyAtLeast(version | predicate over version, count): true iff the predicate is true in at least count versions of x.
  • x.eventuallyAtMost(version | predicate over version, count): true iff the predicate is true in at least one version and at most count versions of x.

Scoping views (predicate-based)

The versions in scope for the above assertions and primitives can be limited with:

  • x.since(version | predicate over version) will return the type/model element in the oldest timepoint since that of x for which the predicate holds, or null if it does not exist. The returned type/model element will only report versions from its timepoint onwards. This esentially imposes a left-closed version interval.
  • x.after(version | predicate over version) will return the type/model element in the timepoint immediately after the oldest timepoint for which the predicate holds, or null if it does not exist. It is essentially a variant of x.since that implements a left-open interval.
  • x.until(version | predicate over version) will return the the same type/model element, but it will only report versions up to and including the first one for which the predicate holds, or null if such a version does not exist. This implements a right-closed version interval.
  • x.before(version | predicate over version) will return the same type/model element, but it will only report versions before (excluding) the first one for which the predicate holds, or null if such a version does not exist. This implements a right-open interval.
  • x.when(version | predicate over version) will return the type/model element in the oldest timepoint since that of x for which the predicate holds, or null if it does not exist. The returned type/model element will only report versions from its timepoint onwards that match the predicate. This is a left-closed, filtered interval.

Scoping views (context-based)

You can also limit the available versions from an existing type / model element:

  • x.sinceThen: version of x that will only report the versions from x onwards (included).
  • x.afterThen: next version of x that will only report the versions after x (excluded). null if a next version does not exist.
  • x.untilThen: version of x that will only report the versions up to x (included).
  • x.beforeThen: previous version of x that will only report the versions before x (excluded). null if a previous version does not exist.

You can undo the scoping with .unscoped. This will give you the same model element or type, but with all the versions available once more.

Scoping views (based on derived attributes)

Some of the events we may be interested in may be very rare. In long histories, it may be very expensive to find such rare events by iterating over all the versions of a model element. In these cases, it is possible to define a derived Boolean attribute (e.g. HasManyChildren for a Tree, with definiton return self.children.size > 100;) on a type, and then use these additional operations:

  • x.whenAnnotated('AttributeName'): returns a view of the model element x that exposes all the versions when the derived attribute named AttributeName defined on the type of x was true. The view will be at the earliest timepoint when this happened.
  • x.sinceAnnotated('AttributeName'): equivalent to since, but using the derived attribute AttributeName.
  • x.afterAnnotated('AttributeName'): equivalent to after. See above.
  • x.untilAnnotated('AttributeName'): equivalent to until. See above.
  • x.beforeAnnotated('AttributeName'): equivalent to before. See above.

IMPORTANT: until #83 is resolved, you will need to define these derived attributes before you index any model versions.

Global operations on the model

The Model global reference is extended with new operations:

  • Model.allInstancesNow returns all instances of the model at the timepoint equal to current system time.
  • Model.allInstancesAt(timepoint) returns all instances of the model at the specified timepoint, measured in the integer amount of milliseconds elapsed since the epoch.
  • Model.getRepository(object) will return a node representing the repository (VCS) that the object belongs to at its current timepoint. From the returned node, you may retrieve the .revision (SVN revision, folder timestamp or Git SHA-1), and the .message associated with the corresponding revision.

Some examples

A simple query to find the number of instances of X in the latest version of the model would be:

return X.latest.all.size;

If we want to do find the second last time that instances of X were created, we could write something like:

return X.latest.prev.time;

If we want to find an X that at some point had y greater than 0 and still survives to the latest revision, we could write something like:

return X.latest.all.select(x|x.versions.exists(vx|vx.y > 0));

More advanced queries can be found in the Git repository for the MRT 2018 experiment tool.

Timeline queries

If you want to obtain the results of a certain query for all versions of a model, you can use the TimelineEOLQueryEngine instead. This operates by repeating the same query while changing the global timepoint of the graph, so you can write your query as a normal one and see how it evolves over time. For instance, if using return Model.allInstances.size;, you would see how the number of instances evolved over the various versions of the graph.

NOTE: due to current implementation restrictions, this will only process versions where type nodes changed (i.e. objects were created or deleted). We plan to lift this restriction in the near future.

Current limitations

  • Subtree contexts, file-first/derived allOf and traversal scoping are not yet implemented for this query engine. File/repository patterns do work.
  • Derived features will only work if added before any VCSes are added, and the impact of adding multiple VCS with their own histories has not been tested yet. Please make sure to report any issues!