that one is also a good example of a "leaky abstraction"
so to be avoided ;-)
The
second one is what my salary buys in Oxfordshire.
Matt,
you're right :-). It was a bad comparison ... I should have
better taken a "booth" instead of a "shack". Then it looks
rather like this:
than like this:
Cheers,
Philip
Bazaar
rather than cathedrals I’m into that, yea cool… but
shacks?
Yep, that's
right. It's not about storing the data but to enable
interoperability between different scientific techniques.
Furthermore, I don't think it will be successful if we try
to create THE ONE ALLMIGHTY interchange data format
specification. Many tried this before and still try it,
especially in my area:
https://en.wikipedia.org/wiki/Mass_spectrometry_data_format
(JCAMP-DX, ANDI/AIA, mzData, mzXML, ...)
http://www.allotrope.org
IMHO ... they didn't reached/will not reach their goals as
they are either too specific or too generic.
The better way is how we do it - it's the bazaar strategy
... the Eclipse bazaar. There are several shacks at the
Science marketplace ... Matt and the DAWNSci shack, Jay and
the ICE shack, Philip and the ChemClipse shack ... and there
is e.g. Tobias who likes to combine the results of different
analytical techniques (shacks) to perform some statistic
evaluations. He is getting the data from each shack in a
generic format bundled with the recipe how to read it.
That's why I really like the HDF5 approach:
https://www.hdfgroup.org/HDF5
Now imagine how to combine data from shacks of different
marketplaces like Science, LocationTech and IoT. We can do
the same as before.
Best,
Philip
Am 16.03.2016 um 02:05 schrieb Jay Jay
Billings:
So
we are not talking about data storage per se, but
the tools and libraries that make it possible?
Something along the lines of
"Data storage tools, libraries
and associated technologies to promote
interoperability."
My
concern is the size of scientific data sets so I want
to be explicit that we don't plan to store anything. I
don't want to put this line in the document and give
them the feeling that they could ask us to store a
Petabyte of data.
Jay
On Tue, Mar 15, 2016 at 10:04 AM,
<Matt.Gerring@xxxxxxxxxxxxx>
wrote:
Hello,
I
do agree with Erwin. Synchrotron and neutron
sources do have existing forums for meeting each
other and sharing code, however we would be an
easy win since we are doing similar things, at
least computing wise.
All
the best,
Matt
Thanks
Erwin :-)
I skip the draft I was just about to send.
This is exactly what I meant.
Best,
Philip
Am
15.03.2016 um 14:34 schrieb Erwin de Ley:
Jay,
Philip, all,
Permit me to also add some 2-cents, based on
what I've seen at the (mainly synchrotron)
institutions that we've been working with.
I'm sure Matt and others have a much deeper
knowledge on this, and they can correct me,
add more info etc. I just want to trigger
some thoughts here ;-)
At the synchrotrons I've seen ideas and
efforts that not only want to allow sharing
datasets between different users, but also
between different tools (interactive and
automated).
Another topic is that datasets are becoming
ever bigger, and the volume of them as well.
Insofar that engineering has to provide
storage and access services for the
(visiting) scientists and that the moment is
getting near that the raw data is no longer
easily transportable, e.g. to a scientists
home institution.
Some aspects of this :
1. common / standard storage formats, i.e.
the technical format / filetype
2. within those technical "containers",
trying to standardize nomenclature &
functional structures
and provide abstraction tools that can work
with different storage variations in a
common way
3. large storage infrastructures and
corresponding tools/APIs to allow storing
huge volumes of datasets and make them
accessible in a controlled manner
4....
E.g. :
1. is about HDF5, Nexus...
2. DAWN's loaders, Soleil's CDMA, I remember
that Philip also described ChemClips's
related features to work with many different
file formats..
3. e.g. ICAT, http://pan-data.eu/about,
...
The Science IWG would be a great place to
collect cross-domain requirements and to
extract knowhow, APIs, libraries, ... to
integrate some of these aspects!
cheers
erwin
Op
15/03/2016 om 13:14 schreef Jay Jay
Billings:
Philip,
That's an interesting idea. Will you
write a little more about what you have in
mind?
Jay
On
Mar 15, 2016 4:58 AM, "Erwin de Ley"
<erwin.de.ley@xxxxxxxxxx>
wrote:
All,
Great to see a top-level project being
planned!
And a definite +1 for Tobias' and
Philip's suggestions.
cheers
erwin
Op
15/03/2016 om 08:38 schreef Philip
Wenig:
Jay,
everyone,
it would be great if we could
include under section "Scope" also:
* Storing scientific data and enable
an easy exchange between
researchers.
Best,
Philip
Am
15.03.2016 um 08:33 schrieb Tobias
Verbeke:
Thanks for
the revisions.
Here is
an updated version
with most of your
changes worked in
one way or the
other. Please let
me know what you
think. Two things:
*BSD is
already mentioned
next to EDL.
*Exporting
cryptographic
algorithms is a
violation of US export
control laws. So we
need to explicitly say
that we will not work
on cryptography or its
applications.
OK...
Adding a sentence that
provides the background
(along 'In order to comply
with ..., ') may be
useful.
Doing
compression or
anonymization are
separate subjects,
regardless of the
origins of the
techniques.
Jay
On Mon,
Mar 14, 2016 at 7:26
PM, Jay Jay Billings
<jayjaybillings@xxxxxxxxx>
wrote:
Tobias,
Good
suggestions. I'll work
them in.
Jay
On
Mar 14, 2016
17:24, "Tobias
Verbeke" <tobias.verbeke@xxxxxxxxxxxxxxxx>
wrote:
Some
comments on
the Science
TLP draft:
-
modeling and
simulation is
only one way
to collect (in
this case
generate) and
analyze
scientific
data (be it in
the physical
or social
sciences);
this could be
broadened to
collection and
analysis of
sample survey
data and
experimental
data
-
maybe add
economics to
the social
sciences ?
-
tools and
libraries for
statistics,
machine
learning,
artificial
intelligence,
data mining,
text mining
-
data
structures
should not be
limited to 3D;
we want to
live in more
dimensions
-
visualization:
a detail, but
4D is not
uncommon
(including
e.g. time
dimension)
-
I don't
understand why
we should
explicitly
exclude (the
mathematics
of)
cryptography;
certain
anonymization
or
privacy-protecting
procedures as
applied to
scientific
data can use
cryptographic
techniques
-
I would use
Revised BSD or
3-clause BSD
next to (or
rather than)
EDL; I
understand the
inclination to
use EDL, but
the name is
not widely
known (e.g.
Wikipedia
knows EPL but
not EDL) and,
if it does not
make any
practical
difference,
the other
names are much
more
recognizable
and therefore
reassuring to
people.
Everyone,
This
draft is under
review by the
Steering
Committee too
and we are
going to
review it one
final time on
Wednesday. We
will share our
thoughts with
you then.
Ideally
we would have
as much
community
feedback
addressed in
the document
as possible
before we
submit it to
the
Foundation. So
please speak
up if you have
ideas!
On
Mon, Mar 14,
2016 at 12:17
PM, Andrea
Ross <andrea.ross@xxxxxxxxxxx>
wrote:
Dear
Everyone,
A draft of the
Science Top
Level Project
can be
viewed and
commented on
here. If
you're
interested in
edit access,
just ask and
I'll grant it
to you.
A few folks
asked what a
Top Level
Project (TLP)
is in our
annual
meeting. It's
a code-less
project that
provides
vetting for
important
practices of
the projects
hosted beneath
it. The
members of the
TLP are called
the Project
Management
Committee
(PMC). Here's
a page listing
the kinds of
things the PMC
does:
https://wiki.eclipse.org/PMC
A couple of
key ones are:
-
reviewing,
discussing,
and
approving/rejecting
CQ requests
before they go
to the
intellectual
property (IP)
team.
-
some
checks &
balances
related to
committer
elections
Kind
regards,
Andrea
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery
options,
retrieve your
password, or
unsubscribe
from this
list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay
Jay Billings
Oak
Ridge National
Laboratory
Twitter
Handle:
@jayjaybillings
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery
options,
retrieve your
password, or
unsubscribe
from this
list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery
options,
retrieve your
password, or
unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay Jay
Billings
Oak
Ridge National
Laboratory
Twitter
Handle:
@jayjaybillings
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery
options, retrieve your
password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options,
retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
This e-mail and any
attachments may contain confidential, copyright
and or privileged material, and are for the use of
the intended addressee only. If you are not the
intended addressee or an authorised recipient of
the addressee please notify us of receipt by
returning the e-mail and do not use, copy, retain,
distribute or disclose the information in or
attached to the e-mail.
Any opinions expressed within this e-mail are
those of the individual and not necessarily of
Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that
this e-mail or any attachments are free from
viruses and we cannot accept liability for any
damage which you may sustain as a result of
software viruses which may be transmitted in or
with the message.
Diamond Light Source Limited (company no.
4375679). Registered in England and Wales with its
registered office at Diamond House, Harwell
Science and Innovation Campus, Didcot,
Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your
password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay Jay Billings
Oak Ridge National Laboratory
Twitter Handle: @jayjaybillings
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
This e-mail and any attachments may contain
confidential, copyright and or privileged material, and are for
the use of the intended addressee only. If you are not the
intended addressee or an authorised recipient of the addressee
please notify us of receipt by returning the e-mail and do not
use, copy, retain, distribute or disclose the information in or
attached to the e-mail.
Any opinions expressed within this e-mail are those of the
individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or
any attachments are free from viruses and we cannot accept
liability for any damage which you may sustain as a result of
software viruses which may be transmitted in or with the
message.
Diamond Light Source Limited (company no. 4375679). Registered
in England and Wales with its registered office at Diamond
House, Harwell Science and Innovation Campus, Didcot,
Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
|
