RE: [stellation-res] Timestamp handling

["Jonathan Gossage" <jgossage@xxxxxxxx>]

> -----Original Message-----
> From: stellation-res-admin@xxxxxxxxxxxxxxx
> [mailto:stellation-res-admin@xxxxxxxxxxxxxxx]On Behalf Of Mark C.
> Chu-Carroll
> Sent: Saturday, March 15, 2003 4:36 PM
> To: stellation-res@xxxxxxxxxxxxxxx
> Subject: Re: [stellation-res] Timestamp handling
>
>
> Jonathan Gossage wrote:
>
> >
> >
> >>-----Original Message-----
> >>From: stellation-res-admin@xxxxxxxxxxxxxxx
> >>[mailto:stellation-res-admin@xxxxxxxxxxxxxxx]On Behalf Of Mark C.
> >>Chu-Carroll
> >>Sent: Friday, March 14, 2003 9:01 PM
> >>To: stellation-res@xxxxxxxxxxxxxxx
> >>Subject: Re: [stellation-res] Timestamp handling
> >>
> >>
> >>Jonathan Gossage wrote:
> >>
> >>
> >>
> >>>There are a number of places where we are using the following
> sequence of
> >>>code that leads to problems and inefficiency.
> >>>
> >>>Date t = // get from somewhere
> >>>attributes.put("time", t.toString());
> >>> .
> >>> .
> >>> .
> >>> .
> >>>Date date = null;
> >>>try {
> >>> date = DateFormat.getDateInstance().parse(attributes.get("time"));
> >>>}
> >>>catch (ParseException pe) {
> >>>logger.debug("Invalid date in attribute: " +
> attributes.get("time") + ",
> >>>using now as default");
> >>>  date = new Date();
> >>>}
> >>>
> >>>This code is failing everywhere sice DateFormat.parse() cannot
> handle the
> >>>output from Date.toString(). In addition it is very expensive in
> >>>
> >>>
> >>processing
> >>
> >>
> >>>and resultsa in losing information. I believe that we should
> use the Date
> >>>objects directly instead of the string representation, and when
> >>>
> >>>
> >>we need to
> >>
> >>
> >>>write externally (e.g. to XML files we should write the long integer
> >>>representation of the date.
> >>>
> >>>
> >>>
> >>I agree completely.
> >>
> >>The whole time handling code was, frankly, a complete mess. I've been
> >>working on cleaning it up, but there's still more to go.
> >>
> >>The remaining problems that I know of (which, I think, is the case
> >>you're pointing at - please correct me if I'm wrong) are cases where,
> >>for some reason, the command-line code was
> >>using a StringMap as a general map. StringMap is intended to be a
> >>type-specific map
> >>from strings to strings. It had evolved into a rather strange generic
> >>string-to-object
> >>map where the object  value was stringified on insertion and parsed on
> >>retrieval. I first
> >>got rid of the type-specific methods in the StringMap. I now need to go
> >>through
> >>the uses of StringMap, and find all of the instances where it's really
> >>not a stringmap.
> >>
> >>I wasn't rushing on this, because as it stands, it's working on my linux
> >>box, so I assumed
> >>it was in a relatively stable state. If it's causing problems, I'll bump
> >>it up and
> >>take care of it ASAP.
> >>
> >>    -Mark
> >>
> >>
> >
> >I think you are getting into JVM specific behaviour here. I am
> using the Sun
> >1.4.1-02 JVM since I have had nothing but problems with the IBM
> JVM's under
> >Windows. The underlying problem that we are seeing here is that there is
> >nothing in the contracts for the Date.toString() method and the
> >DateFormat.parse() method that says that the output of the first
> method is
> >guaranteed to be parsable by the second method. It seems that
> the IBM JVM on
> >Linux lets you get away with it while the Sun JVM on Windows
> does not. The
> >problem may become even worse in a client/server environment
> where different
> >JVM's may be used on the client and the server.
> >
> >
> Yeah, I got that. I'm working on it. I must say that it strikes me as
> dreadful incompetence
> on the part of the Java guys to provide an API that includes a date type
> which provides a standard string representation, and a standard parser,
> where the standard parser and the standard output format are
> *incompatible*.
>
> >The specific area that needs to be addressed is the "time" attributes for
> >the CompoundArtifact, TextArtifact, DataAtrifact and LinkArtifact. On
> >Windows we are getting ParseExceptions all over the place which
> is resulting
> >in the original "time" attributes becoming lost. These all use the
> >DateFormat.parse() method to recover a time stamp from a string
> in a string
> >representation of the "time" attribute. In all cases, when the
> attribute is
> >created, it is created with a Date.toString() method.
> >
> Yeah, I get it.
>
> >I think this is a priority 1 item because it is causing critical
> data to be
> >lost but it is also the tip of the iceberg since I think that the whole
> >method of handling attributes for the artifacts is brittle and
> may cause us
> >problems down the road.
> >
> >
> Handling attributes is already causing me no end of headaches. They're a
> total
> disaster area. The whole damn mess is, for some reason that I find
> incomprehensible,
> implemented in term of maps from strings to values, where the values are
> stored
> as strings and then parsed on retrieval. It really makes no sense at all
> as far as I can
> tell. but it's used so pervasively through the workspace code that I'd
> been hoping to
> postpone dealing with it until we had a better substrate to work with.
>
> The current workspace wasn't ever really intended to be a long-term
> solution. It
> was started just to provide a test engine for the repository. The
> initial workspace worked so well that we kept it, and Dave built on it.
> The catch, of course, is that the basic code
> was never really designed to grow that way - and as it grew, the design
> never changed - each new feature just got added in. As you'd expect from
> that kind of growth, it gradually
> turned into an incomprehensible mess. The only person who really
> understands it is
> Dave, and he's gone on to do other things. At this point, I honestly
> think it's really
> not salvageable. There's too much mess deeply entwined in it.  (The
> biggest problem,
> however, is that much command-line cruft crept into the core. It's a
> tendency
> that I've tried hard to fight, but I just don't have the time or energy
> to personally
> examine every change, and so it slipped under my radar - and then once
> it was
> there, it just deteriorated.)
>
> Our plans have been to used Jim's new client-core to build an
> Eclipse-based command-line
> workspace, and to get rid of the current workspace.
>
> So I have been trying to avoid putting too much time into fixing the
> deep problems
> with the current workspace. Each time a problem shows up, I wind up neck
> deep
> in the mess, and it takes an absolutely inordinate amount of time to
> figure out how
> to fix it. I had hoped that the last date fix was enough to fix the date
> related
> problems. But I was wrong, so I'll dive back in and try to do a cleanup of
> the attribute stuff.
>
>     -Mark

I agree completely with you on this. I think that you should do the absolute
minimum needed to have this stuff at least work reliably for the alpha
release. I would like to undertake a complete re-write of the workspace code
based around the configuration stuff I am working on. First, though I think
we should put together a formal spec on what we think the command line
support should do and what we want to get out of standalone Stellation
workspaces.

The current workspace support is an utter disaster from a performance point
of view. For example, I know how to make a 20:1 improvement in performance
for the "add files" command alone. But I don't want to do anything until we
get a clear consensus on what the likely use cases for standalone support
will be.

Regards

Jonathan