[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [tracecompass-dev] TotalADS: Total Anomaly Detection System
|
Hi Shariyar,
looking forward to seeing you next week!
here's how to add stuff to orbit.
https://wiki.eclipse.org/Orbit/Adding_Bundles_to_Orbit
BR,
Matthew
On 14-12-04 10:58 AM, Shariyar wrote:
> Hi,
>
> I have removed the dependency on embedded libraries by using the
> target file. However, Mahout is missing in Orbit. I cannot add them
> there. So can any one add Mahout there? Specifically, I require the
> following jars:
>
> mahout-core-0.9-job.jar
> mahout-core-0.9.jar
> mahout-integration-0.9.jar
> mahout-math-0.9.jar
>
> Thanks,
> Shariyar
>
> On Thu, Nov 27, 2014 at 2:04 PM, Alexandre Montplaisir
> <alexmonthy@xxxxxxxxxxxx <mailto:alexmonthy@xxxxxxxxxxxx>> wrote:
>
>
> On 11/27/2014 11:34 AM, Shariyar wrote:
>
> Hi Alexandre,
>
> My replies are inline.
>
> On Wed, Nov 26, 2014 at 10:47 PM, Alexandre Montplaisir <
> alexmonthy@xxxxxxxxxxxx <mailto:alexmonthy@xxxxxxxxxxxx>> wrote:
>
> Hi Shariyar,
>
> First, congrats on getting this out! I've seen demos of
> TotalADS many
> times so far, and I'm very happy to see this coming out as
> open-source.
> Props to you and all the people at Concordia who worked on
> this!
>
> Thank you :).
>
> Some questions:
>
> Is this meant to be a project of its own? I'm wondering if
> it wouldn't be
> better to have this in a separate git repository. It could
> for example be
> under the Trace Compass project umbrella at Eclipse, but
> have its own
> separate git repo. Especially since it has its own
> dependencies that aren't
> shared with the rest. It could also be advertised on its
> own, or as a
> plugin to Trace Compass.
> Just throwing the idea out there, we'll have to see more
> in details how
> everything integrates together.
>
>
> It actually depends on Trace Compass for parsing CTF traces,
> XML files and
> text files via regular expression. So I am wondering whether a
> separate git
> repo will work or not. Will a new developer has to pull both
> repos. for
> development? It would be better to have its own git repository
> but I think
> it has to be under Trace Compass right now.
>
>
> It would be perfectly fine to have a separate repo and depend on
> the main Trace Compass plugins. Just as you have other
> dependencies already, TC would just be another one of those.
>
> If a developer wants to pull the TotalADS code to work on it, they
> could get the TC dependencies automatically from the target
> definition, for example.
>
> To me, a separate git repo under the "Eclipse Trace Compass"
> project would make a lot of sense for this. Anyone else has an
> opinion?
>
> Also how do I create a new git
> repo in Gerrit. From the website I learnt that I have to file
> a new bug
> report for a particular plugin for Eclipse to create a new
> repo. Is that
> right or is there any other way?
>
>
> It's the project maintainers (us) who have to take care of this.
> It's quite straightforward to do. I'm just waiting to see the
> conclusion of this discussion first ;)
>
> Related to the previous question, who will be maintaining
> this? What is
> the plan going forward? I assume it's not "done done", and
> that there will
> be people around for fixing bugs? Will more people from
> Concordia be
> working on this? Ideally it shouldn't be a case of "dump &
> run" ;)
>
> Unfortunately, I shall not be doing bug fixing and
> development for it in
>
> future. It will be maintained by someone else at Concordia,
> essentially it
> will be the responsibility of Wahab. Having said that, I will
> be making
> sure that any new features and algorithms added to TotalADS are
> theoretically and syntactically correct before pushing them
> further to
> Trace Compass. So, yes there will be someone maintaining this.
>
>
> Excellent! Initially, all code will have to go through existing
> committers, but if people from Concordia become regularly
> involved, then eventually some could become Trace Compass
> committers themselves, and continue working on the TotalADS repo
> directly.
>
> Some remarks:
> - Everything Geneviève said!
>
> I am not sure how to divide it further into small patches. I
> can push one
> plugin or two plugins at a time(e.g., core and test) .
> However, if we are
> going to keep a separate repository then I can push all the
> code there and
> others can review it gradually as they like.
>
>
> Even if we make a new repo, the initial push should be relatively
> in good shape (no embedded libraries for instance, we wouldn't be
> allowed to even push this). You could start by squashing all the
> commits on your Github repo into one, to eliminate for example
> multiple edits to the same file.
>
> Afterwards, you could split that one big change into smaller
> commits, such as:
> - Provide empty plugins (no code, just the settings and Manifests,
> etc. Just that is usually quite big!)
> - Add the run-time code
> - Add the test cases
>
> - It seems to be based on a very recent version of Trace
> Compass, good job
> getting it up to date!
>
> Yes, I went to push it to Linuxtools. tmf and then I found out
> that it has
> moved out. So I decided to change it in the beginning before
> being too late.
>
>
> - Binary test traces should not be committed in the git
> tree. We host our
> test traces on archive.eclipse.org
> <http://archive.eclipse.org>, and we have our build system
> download
> them on-demand. We can help you set that up once we get to
> that point.
>
> Yes, please I am not sure how to do that. Are there any
> instructions any
> where?
>
>
> You can take a look at the
> "org.eclipse.tracecompass.ctf.core.tests" plugin. We use an Ant
> script to download the test files. The only tricky part is to then
> use assumeTrue(traceExists) in the test cases, to make sure the
> test is skipped if the traces are not present (as in these cases
> it's possible for the traces to "not be there").
>
> Once the test cases are ready to be pushed, we can upload those
> traces to our repo on a.e.o
>
>
> Cheers,
> Alexandre
>
>
>
> - Embedded libraries are a big no-no. You could look into
> using a target
> definition to allow Eclipse to download the dependencies
> (see the
> "org.eclipse.tracecompass.target" plugin in our git tree).
> Eclipse
> projects that depend on external libraries have to get
> them packaged in
> Orbit [1] first. All the ones you are depending on are
> already in Orbit,
> with the exception of Mahout. But that one seems to be
> under the Apache
> License, so there should be no problem getting it into
> Orbit, once we get
> there.
>
> Okay, I shall try to create the target definition file for
> libraries.
>
>
>
>
> Cheers,
> Alexandre
>
>
> [1] Latest Orbit build: http://download.eclipse.org/
> tools/orbit/downloads/drops/R20140525021250/
>
>
>
> Regards,
> Shariyar
>
> On 2014-11-25 11:31 AM, Shariyar wrote:
>
> Dear all,
>
> I would like to introduce a new plugin for
> Tracecompass, called TotalADS
> (Total Anomaly Detection System). Here is a brief
> introduction:
>
> TotalADS is a novel framework for automated host-based
> anomaly detection.
> TotalADS is an open source tool developed as a plug-in
> for Eclipse. It
> integrates different anomaly detection algorithms (or
> techniques),
> different trace readers and a rich set of trace views
> in one common
> platform.
>
> Currently, TotalADS encompasses three different
> algorithms, such as
> Sequence Matching (SQM), Kernel State Modeling (KSM),
> and Hidden Markov
> Model (HMM). It supports execution traces and logs in
> CTF, XML and text
> format. It also supports live anomaly detection using
> trace streaming
> along
> with real time training and testing.
>
> TotalADS also extends another Eclipse plugin called
> Tracecompass by using
> the rich set of views present in it for the
> visualization of traces, such
> as control flow of processes,resource usages and etc.
>
> TotalADS has a number of applications, such as
> automatic detection of zero
> day attacks, diagnosis of anomalous paths in failure
> traces, and diagnosis
> of performance faults in the system.
>
> TotalADS is extendible through simple Java interfaces:
> new algorithms and
> trace readers can be easily added.algorithms and trace
> readers can be
> easily added.
>
> Here is the wiki of TotalADS:
> https://github.com/sshahriyar/org.eclipse.tracecompass/wiki
>
> The fork repository of Tracecompass and the new plugin
> TotalADS:
> https://github.com/sshahriyar/org.eclipse.tracecompass
>
> TotalADS adds four new plugins to Tracecompass, namely:
> totalads.core
> totalads.core.tests
> totalads.ui
> totalads.ui.swtbot.tests
> TotalADS does not make changes to the source code of
> existing plugins of
> Tracecompass except adding references to the plugins
> where necessary.
>
> Screenshots are available here:
> http://users.encs.concordia.ca/~abdelw/sba/totalads/features.html
> <http://users.encs.concordia.ca/%7Eabdelw/sba/totalads/features.html>
>
> A Use Case:
> https://github.com/sshahriyar/org.eclipse.tracecompass/wiki/Use-Case
>
> Let me know how can I push the code for review.
>
> Regards,
> Shariyar
>
>
>
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@xxxxxxxxxxx
> <mailto:tracecompass-dev@xxxxxxxxxxx>
> To change your delivery options, retrieve your
> password, or unsubscribe
> from this list, visit
> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
>
>
>
>
>
>
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
