Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » SeMantic Information Logistics Architecture (SMILA) » Deleting stored records
Deleting stored records [message #654599] Wed, 16 February 2011 13:15 Go to next message
Andrej Rosenheinrich is currently offline Andrej RosenheinrichFriend
Messages: 22
Registered: August 2010
Junior Member
Hello,

I have a (probably) simple question today. I do a lot of crawling and process the gathered data. After processing I dont need the records anymore, but still have them in the record store. Is there an easy way to configure SMILA to delete records when a BPEL workflow finished? Or would the obvious solution be to add a pipelet at the end of the workflow that will do this job?

Thanks in advance!
Andrej

Re: Deleting stored records [message #654645 is a reply to message #654599] Wed, 16 February 2011 14:51 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: juergen.schumacher.attensity.com

Hi,

Am 16.02.2011, 14:15 Uhr, schrieb Andrej Rosenheinrich <andrej.rosenh
> Or would the obvious solution be to add a pipelet at the end of the =

> workflow that will do this job?

Yes I think so. At least I don't have a better idea currently.

Regards,
J=C3=BCrgen
Re: Deleting stored records [message #655448 is a reply to message #654645] Mon, 21 February 2011 08:16 Go to previous messageGo to next message
Andrej Rosenheinrich is currently offline Andrej RosenheinrichFriend
Messages: 22
Registered: August 2010
Junior Member
Thanks for your answer, Jürgen, but I have to ask one more question on this topic. In the BPEL workflow I have the ids of the records. How can i get the ids from the attachments I need to delete them in the binary store? In my understanding these are not the record ids, right?
Another question would be if deleting a record (removeRecord(Id id))has an effect on the binary store or if it only deletes the record from the record store?

Thanks in advance!
Andrej
Re: Deleting stored records [message #656235 is a reply to message #655448] Thu, 24 February 2011 16:04 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: juergen.schumacher.attensity.com

Am 21.02.2011, 09:16 Uhr, schrieb Andrej Rosenheinrich =

<andrej.rosenheinrich@unister-gmbh.de>:

> Thanks for your answer, J=C3=BCrgen, but I have to ask one more questi=
on on =

> this topic. In the BPEL workflow I have the ids of the records. How ca=
n =

> i get the ids from the attachments I need to delete them in the binary=
=

> store? In my understanding these are not the record ids, right?
> Another question would be if deleting a record (removeRecord(Id id))ha=
s =

> an effect on the binary store or if it only deletes the record from th=
e =

> record store?

In a pipelet you should be able to remove attachments by using
blackboard.removeAttachment(Id, String attachmentName). This
should delete the file on disk, too. The RecordStorage does not
know anything about the BinaryStorage, so removeRecord(Id) on the =

RecordStorage
should have no effect on the BinaryStorage. There should probably be a
removeRecord(Id) method on the blackboard that coordinates these things.=


Regards,
J=C3=BCrgen.
Re: Deleting stored records [message #656350 is a reply to message #656235] Fri, 25 February 2011 08:48 Go to previous messageGo to next message
Igor Novakovic is currently offline Igor NovakovicFriend
Messages: 53
Registered: July 2009
Member
>> Thanks for your answer, Jürgen, but I have to ask one more question on
>> this topic. In the BPEL workflow I have the ids of the records. How
>> can i get the ids from the attachments I need to delete them in the
>> binary store? In my understanding these are not the record ids, right?
>> Another question would be if deleting a record (removeRecord(Id
>> id))has an effect on the binary store or if it only deletes the record
>> from the record store?
>
> In a pipelet you should be able to remove attachments by using
> blackboard.removeAttachment(Id, String attachmentName). This
> should delete the file on disk, too.


> The RecordStorage does not know anything about the BinaryStorage, so removeRecord(Id) on the
> RecordStorage should have no effect on the BinaryStorage. There should probably be a
> removeRecord(Id) method on the blackboard that coordinates these things.
Yep!

In the course of simplifying SMILA's data model - which will take place
in the coming days - we should introduce this method in a refactored
Backboard service.

Cheers
Igor
Re: Deleting stored records [message #657600 is a reply to message #656235] Thu, 03 March 2011 12:49 Go to previous messageGo to next message
Andrej Rosenheinrich is currently offline Andrej RosenheinrichFriend
Messages: 22
Registered: August 2010
Junior Member
Well, I tried the idea writing a pipelet that deletes the records in RecordStore and BinaryStore, using the methods Jürgen pointed out. The good thing first: it works Wink meaning the records are deleted. The bad, or shall I say strange, things are the following exceptions i am getting now:

2011-03-03 13:07:02,493 ERROR [QueueWorker-Listener@1909330074 ] impl.Process - [TASK/Process] Error
org.eclipse.smila.processing.ProcessingException: Error processing BPEL workflow AddPipeline: Invocation of pipeline element AddPipeline/extensionActivity-activity-line-159 failed due to runtime error.
at org.eclipse.smila.processing.bpel.ODEWorkflowProcessor.proce ss(ODEWorkflowProcessor.java:241)
at org.eclipse.smila.connectivity.queue.worker.internal.task.im pl.Process.executeInternal(Process.java:43)
at org.eclipse.smila.connectivity.queue.worker.internal.task.im pl.Process.executeInternal(Process.java:1)
at org.eclipse.smila.connectivity.queue.worker.internal.task.Ab stractTask.execute(AbstractTask.java:47)
at org.eclipse.smila.connectivity.queue.worker.internal.task.Ta skListExecutionService.execute(TaskListExecutionService.java :89)
at org.eclipse.smila.connectivity.queue.worker.internal.listene r.ListenerWorker.run(ListenerWorker.java:138)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.eclipse.smila.processing.ProcessingException: Invocation of pipeline element AddPipeline/extensionActivity-activity-line-159 failed due to runtime error.
at org.eclipse.smila.processing.bpel.ExtensionManager.newProces singException(ExtensionManager.java:278)
at org.eclipse.smila.processing.bpel.ExtensionManager.invokeAda pter(ExtensionManager.java:247)
at org.eclipse.smila.processing.bpel.ExtensionManager.invokeAct ivity(ExtensionManager.java:164)
at org.eclipse.smila.processing.bpel.SMILAExtensionBundle$Invok ePipeletActivity.run(SMILAExtensionBundle.java:79)
at org.eclipse.smila.processing.bpel.SMILAExtensionBundle$Invok ePipeletActivity.run(SMILAExtensionBundle.java:91)
at org.apache.ode.bpel.rtrep.v2.EXTENSIONACTIVITY.run(EXTENSION ACTIVITY.java:62)
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.ode.jacob.vpu.JacobVPU$JacobThreadImpl.run(JacobV PU.java:451)
at org.apache.ode.jacob.vpu.JacobVPU.execute(JacobVPU.java:139)
at org.apache.ode.bpel.rtrep.v2.RuntimeInstanceImpl.execute(Run timeInstanceImpl.java:639)
at org.apache.ode.bpel.engine.BpelRuntimeContextImpl.execute(Bp elRuntimeContextImpl.java:593)
at org.apache.ode.bpel.engine.BpelRuntimeContextImpl.executeCre ateInstance(BpelRuntimeContextImpl.java:581)
at org.apache.ode.bpel.engine.ODEProcess.executeCreateInstance( ODEProcess.java:373)
at org.apache.ode.bpel.engine.ODEProcess$2.call(ODEProcess.java :295)
at org.apache.ode.bpel.engine.ODEProcess$2.call(ODEProcess.java :294)
at org.apache.ode.bpel.engine.ODEProcess$ProcessCallable.call(O DEProcess.java:1206)
at org.apache.ode.bpel.engine.BpelInstanceWorker.doInstanceWork (BpelInstanceWorker.java:174)
at org.apache.ode.bpel.engine.BpelInstanceWorker.execInCurrentT hread(BpelInstanceWorker.java:108)
at org.apache.ode.bpel.engine.ODEProcess.doInstanceWork(ODEProc ess.java:487)
at org.apache.ode.bpel.engine.ODEProcess.invokeProcess(ODEProce ss.java:293)
at org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl.doInvok e(MyRoleMessageExchangeImpl.java:122)
at org.apache.ode.bpel.engine.UnreliableMyRoleMessageExchangeIm pl$1.call(UnreliableMyRoleMessageExchangeImpl.java:44)
at org.apache.ode.bpel.engine.UnreliableMyRoleMessageExchangeIm pl$1.call(UnreliableMyRoleMessageExchangeImpl.java:43)
at org.apache.ode.bpel.engine.ODEProcess$ProcessCallable.call(O DEProcess.java:1206)
at org.apache.ode.bpel.engine.Contexts.execTransaction(Contexts .java:106)
at org.apache.ode.bpel.engine.BpelServerImpl$TransactedCallable .call(BpelServerImpl.java:968)
at org.apache.ode.bpel.engine.BpelServerImpl$ServerCallable.cal l(BpelServerImpl.java:948)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.jav a:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Threa dPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo lExecutor.java:908)
... 1 more
Caused by: java.lang.NullPointerException
at org.eclipse.smila.datamodel.tools.record.filter.RecordFilter Utils.filter(RecordFilterUtils.java:141)
at org.eclipse.smila.datamodel.tools.record.filter.RecordFilter Helper.filter(RecordFilterHelper.java:68)
at org.eclipse.smila.blackboard.impl.TransientBlackboardImpl.fi lterRecord(TransientBlackboardImpl.java:806)
at org.eclipse.smila.blackboard.impl.TransientBlackboardImpl.ge tRecord(TransientBlackboardImpl.java:795)
at org.eclipse.smila.processing.bpel.MessageHelper.createWorkfl owRecord(MessageHelper.java:344)
at org.eclipse.smila.processing.bpel.MessageHelper.createWorkfl owObjects(MessageHelper.java:318)
at org.eclipse.smila.processing.bpel.MessageHelper.createMessag e(MessageHelper.java:196)
at org.eclipse.smila.processing.bpel.ExtensionManager.createMes sage(ExtensionManager.java:617)
at org.eclipse.smila.processing.bpel.ExtensionManager.invokeAda pter(ExtensionManager.java:230)
... 32 more

My question would be what is happening after the last pipelet in the BPEL workflow is processed. And why the hell is the Recordfilter invoked after the workflow is finished? Or, what would be really bad, are RecordIds not unique so that deleting one record might affect records in another workerthread?
Re: Deleting stored records [message #658480 is a reply to message #657600] Tue, 08 March 2011 16:39 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: juergen.schumacher.attensity.com

Hi,

Am 03.03.2011, 13:49 Uhr, schrieb Andrej Rosenheinrich =

<andrej.rosenheinrich@unister-gmbh.de>:
> My question would be what is happening after the last pipelet in the =

> BPEL workflow is processed. And why the hell is the Recordfilter invok=
ed =

> after the workflow is finished? Or, what would be really bad, are =

> RecordIds not unique so that deleting one record might affect records =
in =

> another workerthread?

Sorry for the late answer, I somehow did not see this message ...

I think the behaviour is correct. Your pipelet just should not return th=
e =

IDs of
records it has deleted, because the WorkflowProcessor must write =

informatin about the returned
IDs into the output BPEL variable of the invocation. The workflow is not=
=

finished after the last
invoke but only after the <reply> element that selects the variable to =

return as the result of
the complete workflow. This cannot be done anymore if the record does no=
t =

exist anymore on the
blackboard and in the storages, of course. So it should work if you just=
=

return a new empty ID
array at the end of your pipelet (assumed that it deleted all input =

records). It would be nice
if you could test this, I'm not completely sure, if the processor can wo=
rk =

with an empty result
array. However, if not it should be quite easy to fix.

Cheers,
J=C3=BCrgen.
Re: Deleting stored records [message #659323 is a reply to message #658480] Sat, 12 March 2011 09:27 Go to previous message
Andrej Rosenheinrich is currently offline Andrej RosenheinrichFriend
Messages: 22
Registered: August 2010
Junior Member
Thanks Jürgen, your advice works. Seems like SMILA can handle empty arrays Wink

Greets
Andrej
Previous Topic:General memory consumption of SMILA
Next Topic:Announcing changes in SMILA
Goto Forum:
  


Current Time: Wed Nov 26 17:45:30 GMT 2014

Powered by FUDForum. Page generated in 0.01844 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software