Home » Eclipse Projects » EclipseLink » Too many open files using embedded derby
| | |
| Re: Too many open files using embedded derby [message #385438 is a reply to message #385437] |
Wed, 04 February 2009 17:55   |
 Johannes Stamminger
Messages: 17
Registered: July 2009 |
Junior Member |
|
|
Ok, I sorted this out a little bit more now: I collect all instances
creating/releasing entitymanager(factory) in a WeakHashMap (for verifying
at end of tests them all to be closed - initially without any cleanup
there, shame on me). Not collecting them there at all prevents from the
failure.
I guess the instances do not get garbage collected with being ref'ed in
the *WEAK* map (as there is no need from mem point of view). And releasing
the database resources is done on the objects' finalization only, not on
close calls.
Another observation: the error disappears only with removing the instance
from the map *before* entitymanager(factory) close calls. With doing it
afterwards the tests fail again.
Does this sound reasonable?
To me (with my current standing of jpa-beginners knowledge) this does not
look like expected behavior but a bug as the documentation on
EntityManager#close() states:
* Close the factory, releasing any resources that it holds.
* After a factory instance is closed, all methods invoked on
* it will throw an IllegalStateException, except for isOpen,
* which will return false. Once an EntityManagerFactory has
* been closed, all its entity managers are considered to be
* in the closed state.
And the resources seem not being released as "promised" here. Or am I
wrong with this?
The good for me: I can prevent from this very easy in my tests.
Though the bad: I see problems with this behavior might occur with an
application opening/closing database connections heavily and for some
reasons instances referencing entitymanager(factory) instances not being
garbage collected nearly immediately ...
Best regards,
Johannes
|
|
| |
| Re: Too many open files using embedded derby [message #385440 is a reply to message #385439] |
Wed, 04 February 2009 18:54 |
Johannes Stamminger
Messages: 17
Registered: July 2009 |
Junior Member |
|
|
I stripped down my test to a handable portion (running on a kubuntu 8.04
desktop with open files limits set to 2048):
public class JPARegistryBaseTest extends TestCase {
private static final File TEST_DIR;
static {
try {
TEST_DIR =
File.createTempFile(JPARegistryBaseTest.class.getName(), "");
TEST_DIR.delete();
TEST_DIR.mkdirs();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private static int fInstance;
private volatile EntityManagerFactory fEntityManagerFactory;
private volatile EntityManager fEntityManager;
private void open() throws Exception {
final boolean impliciteCreation = !TEST_DIR.exists();
final Map<String, String> dbProperties = new HashMap<String,
String>();
final String name = "input-registry";
final File registryDir = new File(TEST_DIR, "run-" + fInstance);
registryDir.mkdirs();
final String dbDir = new File(registryDir, name).getPath();
dbProperties.put(PersistenceUnitProperties.JDBC_URL, "jdbc:derby:"
+ dbDir + ";create=true");
dbProperties.put(PersistenceUnitProperties.DDL_GENERATION,
PersistenceUnitProperties.CREATE_ONLY);
if (impliciteCreation) {
dbProperties.put(PersistenceUnitProperties.DDL_GENERATION_MO DE,
PersistenceUnitProperties.DDL_BOTH_GENERATION);
} else {
dbProperties.put(PersistenceUnitProperties.DDL_GENERATION_MO DE,
PersistenceUnitProperties.DDL_SQL_SCRIPT_GENERATION);
}
dbProperties.put(PersistenceUnitProperties.APP_LOCATION, dbDir);
// any effect?
dbProperties.put(PersistenceUnitProperties.DEFAULT_APP_LOCAT ION,
dbDir); // affects ddl scripts location
dbProperties.put(PersistenceUnitProperties.LOGGING_FILE, dbDir +
"-derbydb.log");
dbProperties.put(PersistenceUnitProperties.CREATE_JDBC_DDL_F ILE,
".." + File.separator + name + "-create.jdbc");
dbProperties.put(PersistenceUnitProperties.DROP_JDBC_DDL_FIL E,
".." + File.separator + name + "-drop.jdbc");
fEntityManagerFactory =
Persistence.createEntityManagerFactory(name, dbProperties);
fEntityManager = fEntityManagerFactory.createEntityManager();
fEntityManager.setFlushMode(FlushModeType.COMMIT);
}
public void testDirectEntityManagerUsage() throws Exception {
fInstance++;
open();
}
@Override
protected void tearDown() throws Exception {
close();
}
private void close() {
if (fEntityManager != null) {
fEntityManager.close();
fEntityManager = null;
}
if (fEntityManagerFactory != null) {
fEntityManagerFactory.close();
fEntityManagerFactory = null;
}
}
public static Test suite() {
return new RepeatedTest(new TestSuite(JPARegistryBaseTest.class),
100);
}
}
persistence.xml looks like:
<persistence-unit name="input-registry"> <!-- name relates to
JPAPacketInputRegistry#DB_NAME -->
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider >
<class> ... a very simple entity class here ... </class>
<properties>
<property name="eclipselink.jdbc.driver"
value="org.apache.derby.jdbc.EmbeddedDriver"/>
<property name="eclipselink.target-database" value="Derby"/>
<property name="eclipselink.jdbc.user" value="" />
<property name="eclipselink.jdbc.password" value="" />
<property name="eclipselink.logging.level" value="INFO"/>
<property name="eclipselink.orm.throw.exceptions"
value="true"/>
<property name="eclipselink.jdbc.batch-writing" value="JDBC"/>
</properties>
</persistence-unit>
Enough for me today, best regards,
JOhannes
|
|
|
| Re: Too many open files using embedded derby [message #385442 is a reply to message #385440] |
Thu, 05 February 2009 13:28 |
|
Calling close() on the EntityManagerFactory will logout from the JDBC
connections. Try turning logging on finest to see what is going on, you
should see a logout call for each login. Please include the log.
We have run our suit of several 1,000 of tests on Derby, as well as
running the JPA TCK, and have not encountered this issue before, so their
must be something specific to what you are doing, or your environment.
What does the RepeatedTest do, is it multi-threaded? In general a single
EntityManager should not used by multiple threads, each
user/thread/transaction should have its own EntityManager, the
EntityManagerFactory should be shared by the entire server, and only
startup/shutdown with the server.
---
James
James : Wiki : Book : Blog : Twitter
|
|
|
| Re: Too many open files using embedded derby [message #385443 is a reply to message #385442] |
Thu, 05 February 2009 14:03 |
Johannes Stamminger
Messages: 17
Registered: July 2009 |
Junior Member |
|
|
James wrote:
> Calling close() on the EntityManagerFactory will logout from the JDBC
> connections. Try turning logging on finest to see what is going on, you
> should see a logout call for each login. Please include the log.
I do see the logout log message (even with the INFO message level), I
think I already mentioned in a previous post. I'm now in a hurry, I will
attach complete (*very* long) log output with FINEST later.
> We have run our suit of several 1,000 of tests on Derby, as well as
> running the JPA TCK, and have not encountered this issue before, so their
> must be something specific to what you are doing, or your environment.
For sure.
Perhaps because of doing this embedded?
Or because with each of the test run, a *new* database instance is
created. The problem does not become obvious with re-using the same
database in all tests. I tell it "not obvious" because in fact the problem
is then there, too, as the file handles are not released eighther until vm
shutdown - but this does not lead to a failure as no limit is violated
then.
> What does the RepeatedTest do, is it multi-threaded? In general a single
> EntityManager should not used by multiple threads, each
> user/thread/transaction should have its own EntityManager, the
> EntityManagerFactory should be shared by the entire server, and only
> startup/shutdown with the server.
Oh, it is a junit.extensions.RepeatedTest. So in the example I provided
the create/close of the database (without enetering any data, no
transaction involved) is run synchronously one after the other.
As I hopefully said already: with lsof we observe all database files
created due to implicite database creation are kept open - until the limit
is reached.
With using my sources provided (I left away the import statements, I
attach them below), adding the derby, eclipselink and junit (I use 3.8.1
here currently) dependent jars and filling with a very simple entity
implementation (I yesterday simply lacked time to extract one for myself)
you should be able to reproduce my problem immediately.
My first way of coming around this will be to re-use same database but
clear it completely on every test begin.
Next I would look if this problem occurs, too, with not using JPA layer in
between the embedded derby database (to identify responsibility of this
behavior).
Any other suggestions are of cause very welcome ;-).
Best regards,
JOhannes
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.FlushModeType;
import javax.persistence.Persistence;
import junit.extensions.RepeatedTest;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
import org.eclipse.persistence.config.PersistenceUnitProperties;
public class JPARegistryBaseTest extends TestCase {
...
|
|
| | |
Goto Forum:
Current Time: Tue Apr 16 22:04:10 GMT 2024
Powered by FUDForum. Page generated in 0.08121 seconds |
] 
Search
Help
Register
Login
Home
