Applied [HEAD] Re: [cdt-patch] patch for scanner

[John Camelon <jcamelon@xxxxxxxxxx>]

Thanks Dave. 

FYI, change descriptions can be put into ChangeLog files.

JohnC
www.eclipse.org/cdt

cdt-patch-admin@xxxxxxxxxxx wrote on 03/07/2004 11:40:48 AM:

> 
> I have put together a patch for the scanner (for HEAD) as part of 
> some performance analysis that I was doing. 
> All Junit test have passed on Windows. 
> 
> My example for performance testing was: 
>         int abc; 
>         #include <windows.h>   // from cygwin w32api 
>         #include <stdio.h> 
>         int main() 
>         { 
>                 a    ;  // do code assist here 
>         } 
> 
> Note that there are about 7,700,000 Java objects created by this 
> test  -- I believe that reducing this number is essential to optimal
> performance.  The parsing takes about 5.5 seconds (it varies by as 
> much as 1 second either way) on a Pentium M - 1.6 GHz. 
> 
> The patch for the scanner includes the following: 
> 
> 1.  Removes the Strings associated with constant value tokens and 
> keywords  -- it looks cleaner, and reduces the number of objects 
> created (only by 30 to 40 K) (about 2% quicker) 
> 
> 2.  Buffered the File I/O associated with reading inclusions: 
> (about 5% quicker total) 
>         a. for "windows.h" and associated includes the current 
> implementation was 280 ms, 
>         b. the buffered form is 70 ms, and 
>         c.  if we wish to rewrite a lot of supporting code -- a 
> series of memory mapped files  would be 35 ms 
> 
> 3. restructured the scanner into a large case statement with a few 
> helper functions -- this is the start of removing the strings (or 
> providing a preallocated buffer for string manipulation) (about 2% 
quicker) 
>         a. This is a major change in structure, but behavior is the same 

>         b.  I removed some of the "state" information that was 
> explicitly managed by the code 
>         c.  I started combining some of the common code fragments 
> (for easier maintenance and tuning) 
> 
> 
> - Dave 
> 
> [attachment "scanner-update.txt" deleted by John Camelon/Ottawa/IBM]