| jdbc emc problem: "Too many connections" [message #1772183] |
Wed, 06 September 2017 08:28  |
Eclipse User |
|
|
|
Hi all,
I'm using ETL code and MySQL emc driver in order to transform from a MySQL source to a RDF file target
After some correct record trasformation I get this error from the MySQL server: "message from server: "Too many connections"".
This is the ETL code:
rule servicelocation
transform s : MySQL!servicelocation
to t : adms!ConformLoad
{
System.out.println('Rule servicelocation: ' + s.name );
t.UUID = s.mRID;
t.name = s.name;
t.aliasName = s.aliasName;
t.Terminals.add(CreateTerminal(s.Terminal1));
}
operation CreateTerminal(Terminal_ID: String ) {
System.out.println("CreateTerminal " + Terminal_ID);
var MyTerminal;
var TerminalSource = MySQL!terminal.allInstances. // Select the terminal in MySQL input db
selectOne(o|o.mRID = Terminal_ID);
if (TerminalSource.isUndefined()) {
System.out.println("Terminal " + Terminal_ID + " doesn't exist!");
} else { // Se esiste, ne crea uno equivalente in output
System.out.println("Terminal found: " + TerminalSource);
MyTerminal = new adms!Terminal;
MyTerminal.UUID = TerminalSource.mRID;
MyTerminal.name = TerminalSource.name;
// look for the Connectivity Node related to the terminal
//MyTerminal.ConnectivityNode = Connect2CN(TerminalSource.ConnectivityNode);
}
return MyTerminal;
}
I'm also attaching the full console output.
Any suggestion is appreciated, thanks,
Gianluigi
|
|
|
|
|
|
|
|
| Re: jdbc emc problem: "Too many connections" [message #1778519 is a reply to message #1772881] |
Fri, 15 December 2017 10:27  |
| Eclipse User |
|
|
|
Hi,
For future reference, the JDBC driver uses ResourceSets (each of which uses a connection) to provide dynamic collections that are not cached in Java memory but instead accessed directly in the DB. It is good for memory management so the complete DB is not loaded into memory but has the down side that instantiating many collections from the JDBC model will result in "too many connections" errors. As a workaround be sure that in your EOL, ETL, EGL, etc scripts you keep the number of JDBC element access under control. In the OP's case, there was an allInstances access in a transformation rule (pseudo-code):
rule JDBC2SOmething
transform r : Row
to x: X {
x.name = JDBC!Tables.all.select(....);
}
So for each Row in the JDBC model, the "all" operation was invoked. Since each "all" method call created a new connection, then it will result in the mentioned error. If the select operation is static, you could move this to a "pre" block so it is called only once:
pre {
var tables = JDBC!Tables.all.select(....);
}
rule JDBC2SOmething
transform r : Row
to x: X {
x.name = tables.select(...);
}
|
|
|
Powered by
FUDForum. Page generated in 0.27616 seconds
Home
