Matt,
you're right :-). It was a bad comparison ... I should have better
taken a "booth" instead of a "shack". Then it looks rather like
this:
than like this:
Cheers,
Philip
Bazaar
rather than cathedrals I’m into that, yea cool… but shacks?
Yep, that's
right. It's not about storing the data but to enable
interoperability between different scientific techniques.
Furthermore, I don't think it will be successful if we try to
create THE ONE ALLMIGHTY interchange data format
specification. Many tried this before and still try it,
especially in my area:
https://en.wikipedia.org/wiki/Mass_spectrometry_data_format
(JCAMP-DX, ANDI/AIA, mzData, mzXML, ...)
http://www.allotrope.org
IMHO ... they didn't reached/will not reach their goals as
they are either too specific or too generic.
The better way is how we do it - it's the bazaar strategy ...
the Eclipse bazaar. There are several shacks at the Science
marketplace ... Matt and the DAWNSci shack, Jay and the ICE
shack, Philip and the ChemClipse shack ... and there is e.g.
Tobias who likes to combine the results of different
analytical techniques (shacks) to perform some statistic
evaluations. He is getting the data from each shack in a
generic format bundled with the recipe how to read it. That's
why I really like the HDF5 approach:
https://www.hdfgroup.org/HDF5
Now imagine how to combine data from shacks of different
marketplaces like Science, LocationTech and IoT. We can do the
same as before.
Best,
Philip
Am 16.03.2016 um 02:05 schrieb Jay Jay
Billings:
So we
are not talking about data storage per se, but the
tools and libraries that make it possible? Something
along the lines of
"Data storage tools, libraries and
associated technologies to promote interoperability."
My
concern is the size of scientific data sets so I want to
be explicit that we don't plan to store anything. I
don't want to put this line in the document and give
them the feeling that they could ask us to store a
Petabyte of data.
Jay
On Tue, Mar 15, 2016 at 10:04 AM,
<Matt.Gerring@xxxxxxxxxxxxx>
wrote:
Hello,
I
do agree with Erwin. Synchrotron and neutron
sources do have existing forums for meeting each
other and sharing code, however we would be an
easy win since we are doing similar things, at
least computing wise.
All
the best,
Matt
Thanks
Erwin :-)
I skip the draft I was just about to send. This
is exactly what I meant.
Best,
Philip
Am
15.03.2016 um 14:34 schrieb Erwin de Ley:
Jay,
Philip, all,
Permit me to also add some 2-cents, based on
what I've seen at the (mainly synchrotron)
institutions that we've been working with. I'm
sure Matt and others have a much deeper
knowledge on this, and they can correct me,
add more info etc. I just want to trigger some
thoughts here ;-)
At the synchrotrons I've seen ideas and
efforts that not only want to allow sharing
datasets between different users, but also
between different tools (interactive and
automated).
Another topic is that datasets are becoming
ever bigger, and the volume of them as well.
Insofar that engineering has to provide
storage and access services for the (visiting)
scientists and that the moment is getting near
that the raw data is no longer easily
transportable, e.g. to a scientists home
institution.
Some aspects of this :
1. common / standard storage formats, i.e. the
technical format / filetype
2. within those technical "containers", trying
to standardize nomenclature & functional
structures
and provide abstraction tools that can work
with different storage variations in a common
way
3. large storage infrastructures and
corresponding tools/APIs to allow storing huge
volumes of datasets and make them accessible
in a controlled manner
4....
E.g. :
1. is about HDF5, Nexus...
2. DAWN's loaders, Soleil's CDMA, I remember
that Philip also described ChemClips's related
features to work with many different file
formats..
3. e.g. ICAT, http://pan-data.eu/about,
...
The Science IWG would be a great place to
collect cross-domain requirements and to
extract knowhow, APIs, libraries, ... to
integrate some of these aspects!
cheers
erwin
Op
15/03/2016 om 13:14 schreef Jay Jay
Billings:
Philip,
That's an interesting idea. Will you write
a little more about what you have in mind?
Jay
On
Mar 15, 2016 4:58 AM, "Erwin de Ley" <erwin.de.ley@xxxxxxxxxx>
wrote:
All,
Great to see a top-level project being
planned!
And a definite +1 for Tobias' and
Philip's suggestions.
cheers
erwin
Op
15/03/2016 om 08:38 schreef Philip
Wenig:
Jay,
everyone,
it would be great if we could include
under section "Scope" also:
* Storing scientific data and enable
an easy exchange between researchers.
Best,
Philip
Am
15.03.2016 um 08:33 schrieb Tobias
Verbeke:
Thanks for
the revisions.
Here is
an updated version
with most of your
changes worked in
one way or the
other. Please let me
know what you think.
Two things:
*BSD is
already mentioned next
to EDL.
*Exporting
cryptographic algorithms
is a violation of US
export control laws. So
we need to explicitly
say that we will not
work on cryptography or
its applications.
OK...
Adding a sentence that
provides the background
(along 'In order to comply
with ..., ') may be useful.
Doing
compression or
anonymization are
separate subjects,
regardless of the
origins of the
techniques.
Jay
On Mon,
Mar 14, 2016 at 7:26 PM,
Jay Jay Billings <jayjaybillings@xxxxxxxxx>
wrote:
Tobias,
Good
suggestions. I'll work
them in.
Jay
On Mar
14, 2016 17:24,
"Tobias Verbeke"
<tobias.verbeke@xxxxxxxxxxxxxxxx>
wrote:
Some
comments on
the Science
TLP draft:
-
modeling and
simulation is
only one way
to collect (in
this case
generate) and
analyze
scientific
data (be it in
the physical
or social
sciences);
this could be
broadened to
collection and
analysis of
sample survey
data and
experimental
data
-
maybe add
economics to
the social
sciences ?
-
tools and
libraries for
statistics,
machine
learning,
artificial
intelligence,
data mining,
text mining
-
data
structures
should not be
limited to 3D;
we want to
live in more
dimensions
-
visualization:
a detail, but
4D is not
uncommon
(including
e.g. time
dimension)
-
I don't
understand why
we should
explicitly
exclude (the
mathematics
of)
cryptography;
certain
anonymization
or
privacy-protecting
procedures as
applied to
scientific
data can use
cryptographic
techniques
-
I would use
Revised BSD or
3-clause BSD
next to (or
rather than)
EDL; I
understand the
inclination to
use EDL, but
the name is
not widely
known (e.g.
Wikipedia
knows EPL but
not EDL) and,
if it does not
make any
practical
difference,
the other
names are much
more
recognizable
and therefore
reassuring to
people.
Everyone,
This
draft is under
review by the
Steering
Committee too
and we are
going to
review it one
final time on
Wednesday. We
will share our
thoughts with
you then.
Ideally
we would have
as much
community
feedback
addressed in
the document
as possible
before we
submit it to
the
Foundation. So
please speak
up if you have
ideas!
On
Mon, Mar 14,
2016 at 12:17
PM, Andrea
Ross <andrea.ross@xxxxxxxxxxx>
wrote:
Dear
Everyone,
A draft of the
Science Top
Level Project
can be
viewed and
commented on
here. If
you're
interested in
edit access,
just ask and
I'll grant it
to you.
A few folks
asked what a
Top Level
Project (TLP)
is in our
annual
meeting. It's
a code-less
project that
provides
vetting for
important
practices of
the projects
hosted beneath
it. The
members of the
TLP are called
the Project
Management
Committee
(PMC). Here's
a page listing
the kinds of
things the PMC
does:
https://wiki.eclipse.org/PMC
A couple of
key ones are:
-
reviewing,
discussing,
and
approving/rejecting
CQ requests
before they go
to the
intellectual
property (IP)
team.
-
some
checks &
balances
related to
committer
elections
Kind
regards,
Andrea
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery
options,
retrieve your
password, or
unsubscribe
from this
list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay
Jay Billings
Oak
Ridge National
Laboratory
Twitter
Handle:
@jayjaybillings
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery
options,
retrieve your
password, or
unsubscribe
from this
list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg
mailing list
science-iwg@xxxxxxxxxxx
To change your
delivery options,
retrieve your
password, or
unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay Jay
Billings
Oak
Ridge National
Laboratory
Twitter
Handle:
@jayjaybillings
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery
options, retrieve your
password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve
your password, or unsubscribe from this
list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
This e-mail and any
attachments may contain confidential, copyright and
or privileged material, and are for the use of the
intended addressee only. If you are not the intended
addressee or an authorised recipient of the
addressee please notify us of receipt by returning
the e-mail and do not use, copy, retain, distribute
or disclose the information in or attached to the
e-mail.
Any opinions expressed within this e-mail are those
of the individual and not necessarily of Diamond
Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this
e-mail or any attachments are free from viruses and
we cannot accept liability for any damage which you
may sustain as a result of software viruses which
may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679).
Registered in England and Wales with its registered
office at Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire, OX11 0DE,
United Kingdom
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password,
or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Jay Jay Billings
Oak Ridge National Laboratory
Twitter Handle: @jayjaybillings
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
|
