Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » BIRT » How to increase Java heap space(out of memory error on larger data sets)
How to increase Java heap space [message #629318] Mon, 27 September 2010 19:07 Go to next message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

I have a report that works on small data sets but not on large ones. I trimmed down the amount of work that BIRT is doing (removing conditional formatting, pushing aggregations into the SQL statement) but I'm still getting the following trace when generating to XLS.

PDF also blows up, and I haven't tested HTML yet.

The problem occurs in the RCP editor and when the report is run from the offline libraries (DOS batch mode). I know there's a way to tell Java at startup to increase memory, but I don't know where that's located in this kind of environment. Any thoughts?

at org.eclipse.birt.core.framework.Platform.shutdown(Platform.j ava:101)
at org.eclipse.birt.report.engine.api.ReportRunner.execute(Repo rtRunner.java:172)
at org.eclipse.birt.report.engine.api.ReportRunner.main(ReportR unner.java:121)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuffer.append(Unknown Source)
at java.io.StringWriter.write(Unknown Source)

AND

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedInputStream.<init>(Unknown Source)
at java.io.BufferedInputStream.<init>(Unknown Source)
at com.ibm.icu.impl.ICUResourceBundleReader.<init>(ICUResourceBundleReader.java:167)
at com.ibm.icu.impl.ICUResourceBundleReader.getReader(ICUResour ceBundleReader.java:190)
at com.ibm.icu.impl.ICUResourceBundle.createBundle(ICUResourceB undle.java:829)
at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUReso urceBundle.java:737)
at com.ibm.icu.impl.ICUResourceBundle.getBundleInstance(ICUReso urceBundle.java:705)
at com.ibm.icu.util.UResourceBundle.getRootType(UResourceBundle .java:463)
at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResource Bundle.java:511)
at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResource Bundle.java:137)
at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResource Bundle.java:217)
at com.ibm.icu.impl.ICUResourceBundle.getFunctionalEquivalent(I CUResourceBundle.java:173)
at com.ibm.icu.util.Calendar.getCalendarType(Calendar.java:1659 )
at com.ibm.icu.util.Calendar.createInstance(Calendar.java:1755)
at com.ibm.icu.util.CalendarServiceShim$1$RBCalendarFactory.han dleCreate(CalendarServiceShim.java:98)
at com.ibm.icu.impl.ICULocaleService$LocaleKeyFactory.create(IC ULocaleService.java:378)
at com.ibm.icu.impl.ICUService.getKey(ICUService.java:460)
at com.ibm.icu.impl.ICUService.getKey(ICUService.java:380)
at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java: 72)
at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java: 56)
at com.ibm.icu.util.CalendarServiceShim.createInstance(Calendar ServiceShim.java:67)
at com.ibm.icu.util.Calendar.getInstanceInternal(Calendar.java: 1632)
at com.ibm.icu.util.Calendar.getInstance(Calendar.java:1594)
at com.ibm.icu.text.DateFormat.get(DateFormat.java:1288)
at com.ibm.icu.text.DateFormat.getDateTimeInstance(DateFormat.j ava:1108)
at org.eclipse.birt.core.format.DateFormatter.createPattern(Dat eFormatter.java:194)
Re: How to increase Java heap space [message #629319 is a reply to message #629318] Mon, 27 September 2010 19:11 Go to previous messageGo to next message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

Trying to output to HTML I get this:

Sep 27, 2010 2:09:16 PM org.eclipse.birt.report.engine.api.ReportRunner runAndRenderReport
SEVERE: Error happened while running the report
org.eclipse.birt.report.engine.api.EngineException: Error happened while running the report
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doR un(RunAndRenderTask.java:183)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run (RunAndRenderTask.java:75)
at org.eclipse.birt.report.engine.api.ReportRunner.runAndRender Report(ReportRunner.java:222)
at org.eclipse.birt.report.engine.api.ReportRunner.execute(Repo rtRunner.java:162)
at org.eclipse.birt.report.engine.api.ReportRunner.main(ReportR unner.java:121)
Caused by: java.lang.NullPointerException

I was able once to run the report from the RCP editor with output as HTML and it displayed successfully.

ADDED: Running offline to PDF gives this same error. I'l try from the RCP editor again and see if that still generates the out of heap space issue.

[Updated on: Mon, 27 September 2010 19:14]

Report message to a moderator

Re: How to increase Java heap space [message #629325 is a reply to message #629319] Mon, 27 September 2010 19:22 Go to previous messageGo to next message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

Running from the RCP editor with output to XLS, I get the following:

HTTP ERROR 500
Problem accessing /viewer/preview. Reason:

Java heap space

Caused by:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.(Unknown Source)
at java.lang.StringBuilder.toString(Unknown Source)
at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaRe sultSet.getString(OdaResultSet.java:239)
at org.eclipse.birt.data.engine.odaconsumer.ResultSet.getString (ResultSet.java:267)
at org.eclipse.birt.data.engine.odaconsumer.ResultSet.fetch(Res ultSet.java:180)
at org.eclipse.birt.data.engine.executor.cache.OdiAdapter.fetch (OdiAdapter.java:142)
at org.eclipse.birt.data.engine.executor.cache.RowResultSet.nex t(RowResultSet.java:104)
at org.eclipse.birt.data.engine.executor.cache.ExpandableRowRes ultSet.next(ExpandableRowResultSet.java:63)
at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper .populateData(SmartCacheHelper.java:308)
at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper .initInstance(SmartCacheHelper.java:280)
at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper .initOdaResult(SmartCacheHelper.java:152)
at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper .getResultSetCache(SmartCacheHelper.java:77)
at org.eclipse.birt.data.engine.executor.cache.SmartCache.(Smar tCache.java:53)
at org.eclipse.birt.data.engine.executor.transform.pass.PassUti l.populateOdiResultSet(PassUtil.java:98)
at org.eclipse.birt.data.engine.executor.transform.pass.PassUti l.pass(PassUtil.java:61)
at org.eclipse.birt.data.engine.executor.transform.pass.PassMan ager.populateResultSetCacheInResultSetPopulator(PassManager. java:320)
at org.eclipse.birt.data.engine.executor.transform.pass.PassMan ager.doPopulation(PassManager.java:276)
at org.eclipse.birt.data.engine.executor.transform.pass.PassMan ager.doMultiPass(PassManager.java:229)
at org.eclipse.birt.data.engine.executor.transform.pass.PassMan ager.pass(PassManager.java:95)
at org.eclipse.birt.data.engine.executor.transform.pass.PassMan ager.populateResultSet(PassManager.java:72)
at org.eclipse.birt.data.engine.executor.transform.ResultSetPop ulator.populateResultSet(ResultSetPopulator.java:196)
at org.eclipse.birt.data.engine.executor.transform.CachedResult Set.(CachedResultSet.java:85)
at org.eclipse.birt.data.engine.executor.DataSourceQuery.execut e(DataSourceQuery.java:834)
at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery$OdaDSQu eryExecutor.executeOdiQuery(PreparedOdaDSQuery.java:331)
at org.eclipse.birt.data.engine.impl.QueryExecutor.execute(Quer yExecutor.java:1024)
at org.eclipse.birt.data.engine.impl.ServiceForQueryResults.exe cuteQuery(ServiceForQueryResults.java:230)
at org.eclipse.birt.data.engine.impl.QueryResults.getResultIter ator(QueryResults.java:157)
at org.eclipse.birt.report.engine.data.dte.QueryResultSet.(Quer yResultSet.java:98)
at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExec uteQuery(DteDataEngine.java:131)
at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.e xecute(AbstractDataEngine.java:253)
at org.eclipse.birt.report.engine.executor.ExecutionContext.exe cuteQuery(ExecutionContext.java:1825)

Re: How to increase Java heap space [message #629334 is a reply to message #629318] Mon, 27 September 2010 19:46 Go to previous messageGo to next message
Jason Weathersby is currently offline Jason WeathersbyFriend
Messages: 9167
Registered: July 2009
Senior Member

Richard,

The option is -XX:MaxPermSize=256m
what command line are you currently using?

Jason

On 9/27/2010 3:07 PM, Richard A. Polunsky wrote:
> I have a report that works on small data sets but not on large ones. I
> trimmed down the amount of work that BIRT is doing (removing conditional
> formatting, pushing aggregations into the SQL statement) but I'm still
> getting the following trace when generating to XLS.
>
> PDF also blows up, and I haven't tested HTML yet.
>
> The problem occurs in the RCP editor and when the report is run from the
> offline libraries (DOS batch mode). I know there's a way to tell Java at
> startup to increase memory, but I don't know where that's located in
> this kind of environment. Any thoughts?
>
> at org.eclipse.birt.core.framework.Platform.shutdown(Platform.j ava:101)
> at org.eclipse.birt.report.engine.api.ReportRunner.execute(Repo
> rtRunner.java:172)
> at org.eclipse.birt.report.engine.api.ReportRunner.main(ReportR
> unner.java:121)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Unknown Source)
> at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
> at java.lang.AbstractStringBuilder.append(Unknown Source)
> at java.lang.StringBuffer.append(Unknown Source)
> at java.io.StringWriter.write(Unknown Source)
>
> AND
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedInputStream.<init>(Unknown Source)
> at java.io.BufferedInputStream.<init>(Unknown Source)
> at
> com.ibm.icu.impl.ICUResourceBundleReader.<init>(ICUResourceBundleReader.java:167)
>
> at com.ibm.icu.impl.ICUResourceBundleReader.getReader(ICUResour
> ceBundleReader.java:190)
> at com.ibm.icu.impl.ICUResourceBundle.createBundle(ICUResourceB
> undle.java:829)
> at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUReso
> urceBundle.java:737)
> at com.ibm.icu.impl.ICUResourceBundle.getBundleInstance(ICUReso
> urceBundle.java:705)
> at com.ibm.icu.util.UResourceBundle.getRootType(UResourceBundle .java:463)
> at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResource
> Bundle.java:511)
> at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResource
> Bundle.java:137)
> at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResource
> Bundle.java:217)
> at com.ibm.icu.impl.ICUResourceBundle.getFunctionalEquivalent(I
> CUResourceBundle.java:173)
> at com.ibm.icu.util.Calendar.getCalendarType(Calendar.java:1659 )
> at com.ibm.icu.util.Calendar.createInstance(Calendar.java:1755)
> at com.ibm.icu.util.CalendarServiceShim$1$RBCalendarFactory.han
> dleCreate(CalendarServiceShim.java:98)
> at com.ibm.icu.impl.ICULocaleService$LocaleKeyFactory.create(IC
> ULocaleService.java:378)
> at com.ibm.icu.impl.ICUService.getKey(ICUService.java:460)
> at com.ibm.icu.impl.ICUService.getKey(ICUService.java:380)
> at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java: 72)
> at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java: 56)
> at com.ibm.icu.util.CalendarServiceShim.createInstance(Calendar
> ServiceShim.java:67)
> at com.ibm.icu.util.Calendar.getInstanceInternal(Calendar.java: 1632)
> at com.ibm.icu.util.Calendar.getInstance(Calendar.java:1594)
> at com.ibm.icu.text.DateFormat.get(DateFormat.java:1288)
> at com.ibm.icu.text.DateFormat.getDateTimeInstance(DateFormat.j ava:1108)
> at org.eclipse.birt.core.format.DateFormatter.createPattern(Dat
> eFormatter.java:194)
>
Re: How to increase Java heap space [message #629337 is a reply to message #629334] Mon, 27 September 2010 20:05 Go to previous messageGo to next message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

I haev a batch file calling the "genReport.bat" file that ships with the 2.5.0 libraries.
Re: How to increase Java heap space [message #629341 is a reply to message #629337] Mon, 27 September 2010 20:31 Go to previous messageGo to next message
Jason Weathersby is currently offline Jason WeathersbyFriend
Messages: 9167
Registered: July 2009
Senior Member

Richard,

Can you try changing the line that contains
%JAVACMD% -cp
to
%JAVACMD% -cp -XX:MaxPermSize=256m

Jason


On 9/27/2010 4:05 PM, Richard A. Polunsky wrote:
> I haev a batch file calling the "genReport.bat" file that ships with the
> 2.5.0 libraries.
Re: How to increase Java heap space [message #629344 is a reply to message #629341] Mon, 27 September 2010 21:04 Go to previous messageGo to next message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

you mean between "-cp" and BIRTCLASSPATH?

EDITED: yes, that won't work. Putting the argument before the -cp and testing

[Updated on: Mon, 27 September 2010 21:07]

Report message to a moderator

Re: How to increase Java heap space [message #629350 is a reply to message #629344] Mon, 27 September 2010 21:34 Go to previous messageGo to next message
Jason Weathersby is currently offline Jason WeathersbyFriend
Messages: 9167
Registered: July 2009
Senior Member

Yes.


On 9/27/2010 5:04 PM, Richard A. Polunsky wrote:
> you mean between "-cp" and BIRTCLASSPATH?
>
Re: How to increase Java heap space [message #629369 is a reply to message #629318] Tue, 28 September 2010 01:42 Go to previous message
Richard A. Polunsky is currently offline Richard A. PolunskyFriend
Messages: 199
Registered: July 2009
Location: Houston TX
Senior Member

That seems to help. Something is still an issue - this is a Windows box, and the task manager shows the Java process eating 350MB of memory, steadily increasing page faults, 35MB In bytes and 15MB out bytes (those numbers unchanging). I forced a data retrieval limit into the query and am going to see if I can find out where the failure or hang point is. This will help - thanks.

Unfortunately I don't have time to try and replicate this in the Derby database but I'll keep it on my "if time allows" queue.

Richard
Previous Topic:BIRT Vs Cognos
Next Topic:Starting Issues
Goto Forum:
  


Current Time: Mon Dec 22 12:59:56 GMT 2014

Powered by FUDForum. Page generated in 0.01831 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software