Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Problems with concurrent transactions
[CDO] Problems with concurrent transactions [message #1735733] Wed, 22 June 2016 10:39 Go to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 34
Registered: July 2009
Member
I am seeing some problems running multiple CDOTransactions concurrently within the same session (using JVM connector and MEMStore backend. Running with Mars and CDO 4.4 SR2)

I have created an empty MemStore based repository.

I open a transaction, create a new resource and store a element in there. 

ELEMENT_PATH = "/root.xmi"

final CDOTransaction t = session.openTransaction();
final CDOResource r = t.createResource(ELEMENT_PATH);

final MyCDOObject instance = ...; 
r.getContents().add(instance);

// Record the timestamp
CDOCommitInfo commitInfo = t.commit();
timestamp = commitInfo.getTimeStamp();
t.close();


Then I get a new view and pull out the object instance

final CDOView baseView = session.openView(timestamp);
baseView.waitForUpdate(timestamp);
final CDOResource baseResource = baseView.getResource(ELEMENT_PATH);

final MyCDOObject baseInstance = (MyCDOObject ) baseResource.getContents().get(0);



I then create a number of runnables with the following code below and submit to a ExecutorService

for (int retryNumber = 0; retryNumber <commitRetryCount; ++ retryNumber) {
	CDOTransaction transaction = session.openTransaction();

	// Ensure the first commit has been processed (is this needed given shared session?)
	transaction.waitForUpdate(timestamp);

	// Grab the transaction version of the object.
	MyCDOObject transactionInstance = transaction.getObject(baseInstance);
	Assert.assertNotNull(transactionInstance);

	try {
		<ADD NEW ELEMENT TO ROOT OBJECT>
		transaction.commit();
		// No need to retry 
		break;
	} catch (CommitException e) {
		// CommitException - assume concurrent modification and retry
		transaction.rollback();
		// Report error if we hit our limit
		if (retryNumber >= commitRetryCount - 1) {
			e.printStackTrace();
			Assert.fail(e.getMessage());
		}
		// Add delay to allow other commits to execute
		try {
			Thread.sleep(new Random().nextInt(100));
		} catch (final Exception ee) {
			ee.printStackTrace();
		}
	} catch (Exception e) {
		// Some other kind of exception, abort immediately
		transaction.rollback();
		e.printStackTrace();
		Assert.fail();
	} finally {
		// Clean up
		transaction.close();
	}
}


This will sometimes pass, but I am also getting a range of exceptions. These are EOFException, ArrayIndexOutOfBounds and InvalidURIException (which seems strange as it does exist and I have even used the waitForUpdate call) - see below for example stack traces.

Can anyone advise?

Thanks

Simon



EOFException stack trace
java.util.concurrent.ExecutionException: org.eclipse.emf.cdo.common.util.TransportException: java.io.EOFException
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.example.CDOBasedTest.test(CDOBasedTest.java:259)
	at org.example.CDOBasedTest.theTest(CDOBasedTest.java:156)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: org.eclipse.emf.cdo.common.util.TransportException: java.io.EOFException
	at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.send(CDOClientProtocol.java:565)
	at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.unlockObjects2(CDOClientProtocol.java:370)
	at org.eclipse.emf.internal.cdo.view.CDOViewImpl.unlockObjects(CDOViewImpl.java:602)
	at org.eclipse.emf.internal.cdo.view.CDOViewImpl.unlockObjects(CDOViewImpl.java:562)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.handleRollback(CDOTransactionImpl.java:1702)
	at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactionStrategyImpl.rollback(CDOSingleTransactionStrategyImpl.java:110)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.rollback(CDOTransactionImpl.java:1357)
	at org.example.CDOBasedTest.lambda$0(CDOBasedTest.java:217)
	at org.example.CDOBasedTest$$Lambda$1/21422054.run(Unknown Source)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.eclipse.net4j.util.io.ExtendedDataInput$Delegating.readInt(ExtendedDataInput.java:107)
	at org.eclipse.emf.cdo.internal.net4j.protocol.UnlockObjectsRequest.confirming(UnlockObjectsRequest.java:99)
	at org.eclipse.emf.cdo.internal.net4j.protocol.UnlockObjectsRequest.confirming(UnlockObjectsRequest.java:1)
	at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientRequest.confirming(CDOClientRequest.java:97)
	at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedInput(RequestWithConfirmation.java:125)
	at org.eclipse.net4j.signal.Signal.doInput(Signal.java:369)
	at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(RequestWithConfirmation.java:105)
	at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.java:53)
	at org.eclipse.net4j.signal.Signal.runSync(Signal.java:275)
	at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalProtocol.java:466)
	at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(RequestWithConfirmation.java:89)
	at org.eclipse.net4j.signal.RequestWithConfirmation.send(RequestWithConfirmation.java:75)
	at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.send(CDOClientProtocol.java:557)
	... 13 more


ArrayIndexOutOfBoundsException: 
java.util.concurrent.ExecutionException: org.eclipse.net4j.signal.RemoteException: java.lang.ArrayIndexOutOfBoundsException: 126
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.example.CDOBasedTest.test(CDOBasedTest.java:259)
	at org.example.CDOBasedTest.theTest(CDOBasedTest.java:156)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: org.eclipse.net4j.signal.RemoteException: java.lang.ArrayIndexOutOfBoundsException: 126
	at org.eclipse.net4j.signal.RequestWithConfirmation.getRemoteException(RequestWithConfirmation.java:141)
	at org.eclipse.net4j.signal.RequestWithConfirmation.setRemoteException(RequestWithConfirmation.java:130)
	at org.eclipse.net4j.signal.SignalProtocol.handleRemoteException(SignalProtocol.java:489)
	at org.eclipse.net4j.signal.RemoteExceptionIndication.indicating(RemoteExceptionIndication.java:62)
	at org.eclipse.net4j.signal.Indication.doExtendedInput(Indication.java:57)
	at org.eclipse.net4j.signal.Signal.doInput(Signal.java:369)
	at org.eclipse.net4j.signal.Indication.execute(Indication.java:51)
	at org.eclipse.net4j.signal.Signal.runSync(Signal.java:275)
	at org.eclipse.net4j.signal.Signal.run(Signal.java:154)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 126
	at org.eclipse.net4j.util.io.ExtendedIOUtil.readEnum(ExtendedIOUtil.java:363)
	at org.eclipse.net4j.util.io.ExtendedDataInputStream.readEnum(ExtendedDataInputStream.java:49)
	at org.eclipse.net4j.util.io.ExtendedDataInput$Delegating.readEnum(ExtendedDataInput.java:150)
	at org.eclipse.emf.cdo.spi.common.protocol.CDODataInputImpl.readCDOLockType(CDODataInputImpl.java:388)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.UnlockObjectsIndication.indicating(UnlockObjectsIndication.java:52)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CDOServerIndication.indicating(CDOServerIndication.java:86)
	at org.eclipse.net4j.signal.IndicationWithResponse.doExtendedInput(IndicationWithResponse.java:100)
	at org.eclipse.net4j.signal.Signal.doInput(Signal.java:369)
	at org.eclipse.net4j.signal.IndicationWithResponse.execute(IndicationWithResponse.java:73)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CDOServerWriteIndication.execute(CDOServerWriteIndication.java:39)
	... 5 more


InvalidURIException
org.eclipse.emf.cdo.util.InvalidURIException: Invalid URI "cdo://423fc2be-9fb8-451a-9fe7-6c5ce8547402/root.xmi": org.eclipse.emf.cdo.common.util.CDOException: No top level ResourceNode with the name root.xmi
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.registerProxyResource2(AbstractCDOView.java:1576)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.attachResource(AbstractCDOView.java:1541)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.attachResource(CDOTransactionImpl.java:1011)
	at org.eclipse.emf.cdo.eresource.impl.CDOResourceImpl.basicSetResourceSet(CDOResourceImpl.java:1671)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl$ResourcesEList.inverseAdd(ResourceSetImpl.java:586)
	at org.eclipse.emf.common.notify.impl.NotifyingListImpl.addUnique(NotifyingListImpl.java:286)
	at org.eclipse.emf.common.util.AbstractEList.add(AbstractEList.java:303)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.createResource(ResourceSetImpl.java:435)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandCreateResource(ResourceSetImpl.java:243)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:400)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getResource(AbstractCDOView.java:889)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.newResourceInstance(AbstractCDOView.java:1321)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.createObject(AbstractCDOView.java:1281)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getObject(AbstractCDOView.java:1168)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.getObject(CDOTransactionImpl.java:1233)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.convertIDToObject(AbstractCDOView.java:1516)
	at org.eclipse.emf.internal.cdo.view.CDOStoreImpl.convertIDToObject(CDOStoreImpl.java:689)
	at org.eclipse.emf.internal.cdo.view.CDOStoreImpl.getResource(CDOStoreImpl.java:139)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.createObject(AbstractCDOView.java:1296)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getObject(AbstractCDOView.java:1168)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.getObject(CDOTransactionImpl.java:1233)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getObject(AbstractCDOView.java:1217)
	at org.example.CDOBasedTest2.lambda$0(CDOBasedTest2.java:195)
	at org.example.CDOBasedTest2$$Lambda$1/212011969.run(Unknown Source)
	at org.example.ConcurrencyTestUtil$1.run(ConcurrencyTestUtil.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.eclipse.emf.cdo.common.util.CDOException: No top level ResourceNode with the name root.xmi
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getRootOrTopLevelResourceNodeID(AbstractCDOView.java:827)
	at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.getRootOrTopLevelResourceNodeID(CDOTransactionImpl.java:1190)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getResourceNodeID(AbstractCDOView.java:766)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getResourceNodeIDChecked(AbstractCDOView.java:733)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.getResourceNodeID(AbstractCDOView.java:718)
	at org.eclipse.emf.internal.cdo.view.AbstractCDOView.registerProxyResource2(AbstractCDOView.java:1559)
	... 29 more

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1735746 is a reply to message #1735733] Wed, 22 June 2016 11:57 Go to previous messageGo to next message
Pavel Vlasov is currently offline Pavel VlasovFriend
Messages: 23
Registered: December 2015
Junior Member
I am not an expert, but I tihink you probably should use locks on the root object or on the resource - cdoWriteLock().lock(), https://www.eclipse.org/forums/index.php/t/207168/

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1735753 is a reply to message #1735746] Wed, 22 June 2016 12:56 Go to previous messageGo to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 34
Registered: July 2009
Member
Locks would require an object from the CDOTransaction? Several of my exceptions occur trying to get such an object.

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1735976 is a reply to message #1735753] Fri, 24 June 2016 07:42 Go to previous messageGo to next message
Per Sterner is currently offline Per SternerFriend
Messages: 91
Registered: October 2011
Member
Hi,

you could setup an Oomph-CDO-Workspace and reproduce the problem with a junit-test.

I copied your fragments (ZEclipseForum1078624.java) and no exception occurs.

(I used: cdo-4-4-maintenance, H2, JVM-Connector)

Regards,

Per Sterner

[Updated on: Fri, 24 June 2016 07:44]

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1736031 is a reply to message #1735976] Fri, 24 June 2016 13:31 Go to previous messageGo to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 34
Registered: July 2009
Member
Thanks for the Oomph pointer, I had been trying to get this kind of workspace setup myself. I had some problems with the oomph setup trying to resolve the nebula gallery widget for the target platform section and project checkout, but managed to somehow work around that.

Looking at your test and my test, I created the new object outside of the transaction and reused it in my retry loop where as you create a new category each time. My tests look like they run fine now with that change. Is this a requirement in CDO - to create a new instance each time? I noticed the rollback did not undo an eOpposite reference in the new object in my data model.

Thanks,
Simon

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1736061 is a reply to message #1736031] Fri, 24 June 2016 18:02 Go to previous messageGo to next message
Per Sterner is currently offline Per SternerFriend
Messages: 91
Registered: October 2011
Member
Quote:
Looking at your test and my test, I created the new object outside of the transaction and reused it in my retry loop where as you create a new category each time


Yes, this is necessary and makes sense, after the commit, all EObjects are valid and 'referenced' by CDO/The EResource. For new objects you must create new instances. If you add the old committed object in some list, it will change the eContainer/parent or add or set a reference.

Quote:
I noticed the rollback did not undo an eOpposite reference in the new object in my data model.


In this example it will reset the eOpposite:

        Supplier su = getModel1Factory().createSupplier();
        companyTr.getSuppliers().add(su);
        PurchaseOrder po = getModel1Factory().createPurchaseOrder();
        companyTr.getPurchaseOrders().add(po);
        transaction.commit();
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        // -> null ; []
        po.setSupplier(su);
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        // -> -> Supplier@OID5[DIRTY] ; [PurchaseOrder@oid1[NEW]]
        transaction.rollback();
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        // -> -> null ; []
        System.err.println();
        transaction.commit();


But if one of the elements is not contained any were, this may fail:

        Supplier su = getModel1Factory().createSupplier();
        companyTr.getSuppliers().add(su);
        PurchaseOrder po = getModel1Factory().createPurchaseOrder();
        // companyTr.getPurchaseOrders().add(po); // (*)
        transaction.commit();
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        // -> null ; []
        po.setSupplier(su);
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        // -> Supplier@OID5[DIRTY] ; [PurchaseOrder[TRANSIENT]]
        transaction.rollback();
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
       // -> Supplier@OID5 ; []
        System.err.println();
        transaction.commit();


In this case 'rollback' can't rollback everything you wan't, because there are dangling items.

This will result in a DanglingIntegrityException:

        Supplier su = getModel1Factory().createSupplier();
        companyTr.getSuppliers().add(su);
        PurchaseOrder po = getModel1Factory().createPurchaseOrder();
        // companyTr.getPurchaseOrders().add(po); // (*)
        transaction.commit();
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        po.setSupplier(su);
        System.err.println("-> " + po.getSupplier() + " ; " + su.getPurchaseOrders());
        transaction.commit();

Report message to a moderator

Re: [CDO] Problems with concurrent transactions [message #1736096 is a reply to message #1736061] Sat, 25 June 2016 18:54 Go to previous message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 34
Registered: July 2009
Member
Thanks for the explanation. That helps clear things up for me.

Report message to a moderator

Previous Topic:[CDO] Query returns always the same object
Next Topic:Setting EReference
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Jun 19 23:19:59 GMT 2021

Powered by FUDForum. Page generated in 0.01529 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly