Re: [stellation-res] Latest checkin *does* run the unit tests - with one

["Mark C. Chu-Carroll" <mcc@xxxxxxxxxxxxxx>]

On Wednesday 14 August 2002 03:33 pm, Florin Iucha wrote:
> On Wed, Aug 14, 2002 at 08:27:52AM +0000, Mark C. Chu-Carroll wrote:
> > On Wednesday 14 August 2002 09:51 am, Florin Iucha wrote:
> > > On Tue, Aug 13, 2002 at 01:54:40PM +0000, Mark C. Chu-Carroll wrote:
> > > > Unit tests pass with MAX_AGE = 12.
> > >
> > > Because you have 512 MB RAM + ??? swap.
> >
> > I run with 512MB ram + 1G swap. According to gkrellm, it never even
> > went into swap, but just ran in the 512M.  (Besides, we're running
> > Java with a max heap size set to well below that... So there's something
> > very strange going on.)
>
> I am using Alan Cox kernels that do not allow to allocate more memory
> than you really have. I think you are using RedHat stock kernels which
> allow overcommit: the process can allocate as much memory as it wants as
> long as it doesn't use it ;) This can cause out-of-memory when the
> process really dirties too many pages. With strict overcommit control
> that does not happen - a request to allocate more memory than is
> available fails right away.

There's still a JVM memory restriction, which is independent of the kernel. 
(Meaning that it's normally set so that it should kick in before you've used
so much memory that you'll run out. But when you consider the Eclipse 256M
max heap, plus the same for the JVM running JUnit under Eclipse, plus the, I 
think, 128M max heap for the server... Yeah. You can trip that limit.

> > > I have 256 MB RAM and 512 MB swap and the last test fails with 12 and
> > > the last two tests fail with 15. I am using Alan Cox kernels which have
> > > strict overcommit (you cannot allocate more memory than you have).
> > >
> > > This is with both SUN and IBM JDKs.
> > >
> > > Invoking Svn.run leaks both threads and memory. I have a
> > >    ps x | grep java | wc -l
> > > in another console and I have counted 1000 java threads!!!
> >
> > Gag! I'll try to look at that. Something is *seriously* wrong there.  I'm
> > having a hard imagining where 1000 threads could be coming from!
> >
> > Out of curiousity... Do you see the same kind of out-of-control thread
> > stuff running from the command line? Or is it just when run from the
> > unittest framework?
>
> It happens only with the unittest framework right now because it uses a
> single jvm process for all operations.

Does the unit test framework close the workspace between invocations? What I'm
wondering is if the old invocations of svc aren't fully terminating, in which 
case you'd see a communication core thread, plus the message queue thread 
pool, for each of the invocations. The default message pool is 5 threads, 
producing, I think, 8 client-side threads per invocation.  (Hmmm... I wonder
if the message handler thread pool is terminating threads correctly on 
engine shutdown... I'll check that.)

This wouldn't explain 1000 threads, but it's a start towards an explanation. 
If I have time, I'll try to look at the test stuff, and see if it's closing 
off the comm layer properly between invocations.


> When running from the command line, all the resources are cleaned up
> upon jvm's exit.
>
> [That's why I don't like garbage collection user's mindset ;)]
>
> I have added a System.gc() in tests tearDown method but it doesn't
> appear to help. There is real memory pinned down there, quite possibly
> in the leaked threads.
>
> > (Unfortunately, I'm home with the baby today,
>
> Get him better soon!

*Her*. She's a lot better today... Yesterday, she was running a high fever; 
today, the fever is barely detectable. (less than 1 degree F.) So hopefully,
by tommorow, things will be back to normal. 

	-Mark
-- 
Mark Craig Chu-Carroll,  IBM T.J. Watson Research Center  
*** The Stellation project: Advanced SCM for Collaboration
***		http://www.eclipse.org/stellation
*** Work Email: mcc@xxxxxxxxxxxxxx  ------- Personal Email: markcc@xxxxxxxxxxx