| Out of memory for large report [message #1416814] |
Thu, 04 September 2014 20:43  |
 andy Moses
Messages: 26
Registered: November 2013 |
Junior Member |
|
|
Hi, does any one have experience on how to handle "Out of Memory" when the birt report is very large, e.g. thousands of pages.
My web server hosting website and using birt report engine (running on same server) to render report. When there are a few requests to request same birt large report, the server threw "Out of memoey" error.
The server has a restriction of 3G memory (32 bits jvm) and has no quick plan to change it to 64bits.
Under this situation, any recommended solution to avoid the out of memory of server. If there are multiple requests of the same large report around same time, it has risks of bring down the server, which is a big problem.
I wonder if below ways might work or not, hope any one could give some advices:
1) pagination
so don't return all pages, only show one page;
Not sure how easy it could be implemented because the app need to record total pages count and pass the prev/next page number
2) Birt web viewer
which has built-in pagination. But not sure if still the memory still hold the whole report. Same amount of memoery is consumed as not using web viewer.
3) independent Birt iServer, not using my server to both host the website and render report using same memory.
This need to pay money
any others? Appreciate any suggestions or experiences
[Updated on: Tue, 14 October 2014 14:55] Report message to a moderator |
|
|
| Re: Out of memory for large report [message #1417666 is a reply to message #1416814] |
Fri, 05 September 2014 21:37  |
|
What output are you going to? Also, are you using runAndRender or separate run and render tasks in your Java class? runAndRender is all done in memory. Separate run and render will use a .rptdocument between tasks. Not sure if this will help with how large your report is, but it'd be worth a shot if you're not already doing it. You could also try the new BIRT iHub F-Type to see if it gives you better performance, but it requires 64bit, so that doesn't seem to be an option.
Another thought would be, is this a static report or parameterized for different users? If static, you could run the report once each day, then just supply the output file on request.
Michael
Developer Evangelist, Silanis
|
|
|
|
| Re: Out of memory for large report [message #1434040 is a reply to message #1419359] |
Mon, 29 September 2014 15:24 |
|
Sorry for the delay. The paginated HTML output is part of the sample viewer. If using it, it uses separate tasks when using the viewer frameset for the html pagination. Have you tried changing your code to just use separate tasks without the viewer?
Michael
Developer Evangelist, Silanis
|
|
|
| Re: Out of memory for large report [message #1436394 is a reply to message #1434040] |
Thu, 02 October 2014 21:14 |
andy Moses
Messages: 26
Registered: November 2013 |
Junior Member |
|
|
Thank you Michael.
I tried using two separate tasks for run and render. And originally the birt report creator return file content (in byte[] type) to front end code, I changed to let renderOption directly output to httpServletResponse's outputstream.
The result turned out to be even worse:
max Memory usage increased 60%. ( The max memory usage is that I use tools to see the max memory used, minus the memory usage just before the report request was made)
Performance: time spent in run and rendering the report increase a little longer.
The runTask spend the same amount of the time as previouse runAndRenderTask.
And renderTask itself take its own time. So the extra time is on the renderTask, which is quite fast compared with runTask
An experiment of dataengine hint on memory seem not helpful. There's no memory usage different noticed with or without below hint.
task.getAppContext().put(DataEngine.MEMORY_USAGE, DataEngine.MEMORY_USAGE_CONSERVATIVE);
So in my understanding, even pagination will not help the performance since pagination requires to have to two task (run and render) and it showed me that using two tasks significantly use more memory, even if I only set to show one single page on screen, the memory usage will still increase greatly. (I didn't test it but above experiment made me this impression)
Then I might be out of options. Is there any other birt engine API provide some hint on memory usage, or other API methods which may help use less memory?
[Updated on: Thu, 02 October 2014 21:16] Report message to a moderator |
|
|
|
| Re: Out of memory for large report [message #1445622 is a reply to message #1441660] |
Wed, 15 October 2014 17:59 |
andy Moses
Messages: 26
Registered: November 2013 |
Junior Member |
|
|
Michael, thank you for your input.
So far it's required to have one report. The largest report has about 800 pages. It's probably not very big.
More exactly, I wonder what the possible options we have, to user Birt to create large report. Below options were experimented but didn't seem to help.
1) Pagination. Not working since this requires separating run and render tasks, which seems need much more memory (maybe when the rptdocument is generated)
2) RenderOption output to http servlet response's stream directly. Did not notice memory usage was less.
3) Web viewer: this has not tried, but I imagine the memory usage should be about the same.
4) memory hint setting for RunTask didn't seem to improve memory usage.
task.getAppContext().put(DataEngine.MEMORY_USAGE, DataEngine.MEMORY_USAGE_CONSERVATIVE);
I guess the options left are:
1) use a separate report server to generate birt report
2) enhance report design file to shrink the size of the report especially HTML format.
3) stop supporting real-time reporting; schedule back-end process to generate
report. Give users pre-generated reports when they request.
4) as Michael suggested, split the reports or sql data range
Appreciate your input
[Updated on: Wed, 15 October 2014 18:00] Report message to a moderator |
|
|
Powered by
FUDForum. Page generated in 0.04711 seconds
] 
Search
Help
Register
Login
Home
