Re: [stellation-res] UNITTESTS DROP: support for nested directories

[Dave Shields <shields@xxxxxxxxxxxxxx>]

On Sat, Aug 03, 2002 at 04:25:09PM -0400, Mark C. Chu-Carroll wrote:
> On Saturday 03 August 2002 03:54 pm, Florin Iucha wrote:
> > New features: nested directories.
> >
> > Question: why is stellation using unix commands for file manipulation
> > instead of java native io API? Is it faster this way?
> 
> The Unix commands are all in the workspace code, which is written
> and maintained by Dave Shields. So Dave can give the conclusive
> answer. (To be honest, I don't know nearly as much about the
> command line parts of the system as I'd like; my attention has
> been focused on the repository core and server. )
> 
> >From what Dave has told me, I think the Unix commands are used for handling 
> things that you can't do with the Java IO api. For example, modifying the 
> filesystem permissions for a file can't be done with the Java IO API.  Nor 
> can the executable bit bit manipulated. Nor can symbolic links be created.
> And even ignoring those things, when faced with a java.io.File which
> is the name for a symbolic link, the Java IO stuff doesn't handle
> it particularly well for something like Stellation that needs to be able
> to distinguish the link from its target.
> 
> All of the Unix commands are supposed to be in isolated code,
> where there's a non-Unix alternative. (If it's not, then it's a bug,
> and should be fixed.)
> 
> 	-Mark


Mark's summary is accurate. The problem is that java.io.File doesn't support links or executable
files. As a result, the current code has two versions of 'change': JavaChange, which uses java.io.File to
list the workspace files, and UnixChange, which uses five 'find' commands to map the file structure of the
workspace.

I was about to check in a new version, which combines JavaChange and UnixChange into a single file Change.java
(which also includes code formerly in Clean.java), with system dependencies all in revised Files.java, when
I uncovered that mangled Project.java that has been the subject of a separate discussion.

Note the unix commands are used only at the file level, and not to manipulate the file contents, and in general
are only needed for the 'change' operations: add, clean, move and remove.

I take the view that file-level operations are much less important to overall performance that operations 
on the file content. For example, version 2.4.9 of the Linux kernel source has about 7000 files 
taking 106 megabytes. I doubt that JNI, which would affect only the file-level operations, would help much, as the
real cost is dealing with the 106 megabytes of file content.

However, the large number of files in this example revealed that JDOM was quite slow parsing large project files, and
this prompted the conversion of the workspace code to use w3c.dom and not JDOM.

dave

-- 
Dave Shields, IBM Research, shields@xxxxxxxxxxxxxx, http://www.eclipse.org/stellation