| Re: [eclipselink-dev] Code review: initial partitioning support, bug#328937 |
|
Hello
Samba, Thank you for your comments, please also
submit them to the design document discussion page so they can be tracked
accordingly. Database clustering is obviously a big
complex area, and this is our first entrance into this space. We will not support everything imaginable in
our first release. I think the policies
outlined in the design doc cover the common use cases, and are probably even a
little to ambitious for a first release. What you seem to be requesting is support
for both partitioning and replication... and load balancing for the same
data. As you can understand this would
be pretty complex. Scenarios like this are
something that the design of the partitioning framework are capable of
supporting, but given the complexity, this is not something we will direct support
in our first release. To implement this
you can define your own PartitioningPolicy subclass (or subclass the policy
that matches your requirements the closest).
Then for a given query you can define, in your own code, which databases
to send the request to. You would need to
define which set of servers each range should go to. For a read request, you could load balance across these
servers. For a write request, you could
write to each of the servers. If you
detect an error in one of the servers, you can failover to the other. I understand that the root class name PartitioningPolicy
does not describe everything it can do adequately. But naming the class PartitioningLoadBalanacingReplicationAndFailoverPolicy
would be a little too wordy; I think partitioning is the main usecase, hence
its name. -----Original
Message----- Hi James, I
have few comments on the implementation; please read these as constructive
criticism. A
PartitioningPolicy is supposed to be used for partitioning both the read
queries as well as write statements such that queries get distributed and end
up at different database instances. A ReplicationPolicy,
as noted in the comments above the class, is instrumented to duplicate all
writes across all the configured nodes of replication. A
LoadBalancingPolicy can be an implementation of either of the above
or a combination of both the partitioning and the replication features. So, we
need to create a base class for LoadBalancingPolicy that can be
exteded to support various ways of load balancing by utilizing replication or
partitioning or both. The
difference I'm trying to bring out is that a ReplicationPolicy cannot be an
extension of PartitioningPolicy. Similarly, a LoadBalancingPolicy
can have an instance each of PartitioningPolicy and ReplicationPolicy but
it is not an extension of either of these; instead it can only be a
composition of these two features. How we
provide load balancing can be dealt in the implementations like: 1.
RounRobinLoadBalancingPolicy, CluserLoadBalancingPolicy,
etc are possible if the data is only
replicated and not partitioned; they support active/passive fail-over. Their primary
purpose is to support
fail over and optionally to reduce load on a single
server. These implementations rely on having ReplicationPolicy implementations
and cannot have PartitioningPolicy instance. 2.
RangeLoadBalancingPolicy, HashLoadBalancingPolicy,etc are possible
with partitioning the data across
several nodes, their primary purpose is to provide scalability and
performance.However we can also replicate
each partitioned node and thus can support passive fail-over.
These policies will primarily reply
on having PartitioingPolicy implementation but can optionally also
include ReplicationPolicy features as
well in order to support fail over in addition to scale and performance. I hope I'm
making some sense here :) Thanks and
Regards, On Mon, Nov
8, 2010 at 7:57 AM, James Sutherland <JAMES.SUTHERLAND@xxxxxxxxxx>
wrote:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=328937 design
doc, http://wiki.eclipse.org/EclipseLink/DesignDocs/328937 Changes: - added
partitioningPolicy to ClassDescriptor - added
null check to FetchGroupManager to avoid null-pointer on failed deploy - added
PartitionPolicy abstract class, defines getConnections API - added
ReplicationPolicy, replicates writes to multiple pools - added
pool reference to Accessor, so it knows where it came from - added
acquire/release connection logging - added
partitioningPolicy to AbstractSession - changed
AbstractSession accessor to Collection accessors (and updated references) -
changed transaction to work with multiple accessors (2 stage commit) -
changed call execution to work with multiple accessors - made
client sessions (isolated, exclusive) execute calls consistently, added support
for partitioning - added
@Overrides to sessions, some micro - fixed
finally connection release in ReferenceMapping -
changed DatabaseQuery accessor to Colleciton accessors -
changed SessionBroker getAccessor API to use same getAccessor for partitioning,
only a single call, pass query - added
setURL to DatabaseLogin -
changed ClientSession writeAccessor to Map writeAccessors keyed on pool name -
changed ClientSession connection to be lazily assigned -
changed getAccessor on ClientSession to assign a connection if in a transaction
to support backward compatiblity, and internal usage -
changed ServerSession call execution to support partitioning - added
JPA partitioned model -
changed JPA test framework methods to be instances methods and use inherited
getPersistenceUnitName define in test avoid common mistakes in non default unit
tests - added
JPA paritioning test switch using derby "cluster" and round robin and
replication, tests only run on Derby as need to create multiple databases - added
batch-fetch example - added
partitioned, and isolated partitioned version of UnitOfWork test model, uses
"virtual rack" (multiple connection pools to the same database) Code
review: Andrei (pending) |