Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » EMF Resource load from InputStream taking too long(Resource load from InputStream in Java takes approx. 20 minutes)
EMF Resource load from InputStream taking too long [message #1634807] Wed, 25 February 2015 13:13 Go to next message
Syed Rizvi is currently offline Syed RizviFriend
Messages: 7
Registered: February 2015
Junior Member
Hi,
I am trying to load an EMF resource from an InputStream in Java as stated in the topic and here is how the code looks:
			DataPackage.eINSTANCE.eClass();
			String nsURI = DataPackage.eINSTANCE.getNsURI();
			Resource.Factory.Registry reg = Resource.Factory.Registry.INSTANCE;
			Map<String, Object> m = reg.getExtensionToFactoryMap();
			m.put("*", new XMIResourceFactoryImpl());
			
			ResourceSet resSet = new ResourceSetImpl();
			resSet.getPackageRegistry().put(nsURI, DataPackage.eINSTANCE);
			
			//Get URL contents
			URL mashupUrl = new URL(url);
			URLConnection connection = mashupUrl.openConnection();
			connection.setConnectTimeout(15000);
			connection.setReadTimeout(600000);
			
			bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
			
			while ((inputLine = bufferedReader.readLine()) != null)
			{
				content.append(inputLine);
			}
			
			bufferedReader.close();
                        connection.getInputStream().close();
			//Get URL contents
			
			Resource res = resSet.createResource(URI.createURI(url));
			
			if(res == null) {
				throw new MashupConnectionException("Could not create emf resource", url);
			}
			
			final long start = System.currentTimeMillis();
			
			//Load resource
			try {
				
				System.out.println("Resource load start: " + new Date());
				res.load(new URIConverter.ReadableInputStream(content.toString(), StandardCharsets.UTF_8.name()), null);
				System.out.println("Resource load end: " + new Date() + " Seconds elapsed: " + (System.currentTimeMillis() - start)/1000.0);
				
			} catch (IOException e) {
				...
			}


As you can see above, "content" contains the XML data retrieved from a URL: muc2014.communitymashup.net/x3/mashup

My main question: The same code used to load the resource in < 2 minutes in my previous project (which somehow got corrupted and I had no backup unfortunately). I have now reproduced this from memory, it works, but the resource loading time has increased ten folds and I am totally unable to recall what I was doing differently in my previous version of the project.

If any other resources are required, please let me know! I would really appreciate your help.

Thank you!

[Updated on: Wed, 25 February 2015 14:44]

Report message to a moderator

Re: EMF Resource load from InputStream taking too long [message #1636380 is a reply to message #1634807] Thu, 26 February 2015 06:55 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30452
Registered: July 2009
Senior Member
Syed,

Comments below.

On 25.02.2015 15:12, Syed Rizvi wrote:
> Hi,
> I am trying to load an EMF resource from an InputStream in Java as
> stated in the topic and here is how the code looks:
>
> DataPackage.eINSTANCE.eClass();
> String nsURI = DataPackage.eINSTANCE.getNsURI();
> Resource.Factory.Registry reg =
> Resource.Factory.Registry.INSTANCE;
> Map<String, Object> m = reg.getExtensionToFactoryMap();
> m.put("*", new XMIResourceFactoryImpl());
Is this a stand alone application rather than something running in an
Eclipse IDE?
>
> ResourceSet resSet = new ResourceSetImpl();
> resSet.getPackageRegistry().put(nsURI, DataPackage.eINSTANCE);
>
> //Get URL contents
> URL mashupUrl = new URL(url);
> URLConnection connection = mashupUrl.openConnection();
> connection.setConnectTimeout(15000);
> connection.setReadTimeout(600000);
>
> bufferedReader = new BufferedReader(new
> InputStreamReader(connection.getInputStream()));
>
> while ((inputLine = bufferedReader.readLine()) != null)
> {
> content.append(inputLine);
> }
>
> bufferedReader.close();
> //Get URL contents
>
> Resource res = resSet.createResource(URI.createURI(url));
>
> if(res == null) {
> throw new MashupConnectionException("Could not create
> emf resource", url);
> }
>
> final long start = System.currentTimeMillis();
>
> //Load resource
> try {
>
> System.out.println("Resource load start: " + new Date());
> res.load(new
> URIConverter.ReadableInputStream(content.toString(),
> StandardCharsets.UTF_8.name()), null);
> System.out.println("Resource load end: " + new Date() +
> " Seconds elapsed: " + (System.currentTimeMillis() - start)/1000.0);
>
> } catch (IOException e) {
> ...
> }
>
>
> As you can see above, "content" contains the XML data retrieved from a
> URL: muc2014.communitymashup.net/x3/mashup

Why is this all so complicated? I.e., why not pass an input stream
direction to load? Then it would properly respect the encoding of the
XML/XMI?

> My main question: The same code used to load the resource in < 2 minutes
> in my previous project (which somehow got corrupted and I had no backup
> unfortunately). I have now reproduced this from memory, it works, but
> the resource loading time has increased ten folds and I am totally
> unable to recall what I was doing differently in my previous version of
> the project.

There are often questions about which options can be used to affect the
performance of loading. You could look for such answers. Often simple
things like deferring ID resolution have a huge impact.

>
> If any other resources are required, please let me know! I would really
> appreciate your help.
>
> Thank you!
>
Re: EMF Resource load from InputStream taking too long [message #1637220 is a reply to message #1636380] Thu, 26 February 2015 15:59 Go to previous messageGo to next message
Syed Rizvi is currently offline Syed RizviFriend
Messages: 7
Registered: February 2015
Junior Member
Dear Ed,
Thank you for your response!


  1. This is currently running in an Eclipse IDE and will later be a stand-alone web app that will serve a dataset to its clients

  2. Correct, but passing such a big input stream to the resource.load() function would keep the stream open for a long time and the application cannot afford that since its URL fetch cannot exceed 60 seconds in duration (AppEngine restriction)

    P.S. I know connection.setReadTimeout() is set to 600,000ms instead of 60,000ms in my original message and that was a mistake

  3. I will have a look at the answers related to improvement of loading performance. But would you be so kind to suggest a few that might be most relevant, or do I need take a trial and error approach? Smile


Best regards.
Re: EMF Resource load from InputStream taking too long [message #1637750 is a reply to message #1637220] Thu, 26 February 2015 21:55 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30452
Registered: July 2009
Senior Member
Syed,

Comments below.

On 26.02.2015 16:59, Syed Rizvi wrote:
> Dear Ed,
> Thank you for your response!
>
>
> This is currently running in an Eclipse IDE and will later be a
> stand-alone web app that will serve a dataset to its clients

Best to rely on registrations local to the resource set (which also has
a resource factory registry).

>
> Correct, but passing such a big input stream to the resource.load()
> function would keep the stream open for a long time and the application
> cannot afford that since its URL fetch cannot exceed 60 seconds in
> duration (AppEngine restriction)
Converting it to characters without taking into account the encoding
specified in the XML/XMI itself is a bad idea.
>
> P.S. I know connection.setReadTimeout() is set to 600,000ms instead of
> 60,000ms in my original message and that was a mistake
>
> I will have a look at the answers related to improvement of loading
> performance. But would you be so kind to suggest a few that might be
> most relevant, or do I need take a trial and error approach? :)

I don't have my development environment available but Google turns up
things quickly:
https://www.eclipse.org/modeling/emf/docs/performance/EMFPerformanceTips.html
>
>
> Best regards.
Re: EMF Resource load from InputStream taking too long [message #1647559 is a reply to message #1637750] Tue, 03 March 2015 13:48 Go to previous messageGo to next message
Syed Rizvi is currently offline Syed RizviFriend
Messages: 7
Registered: February 2015
Junior Member
Thanks Ed, using the options improved the resource loading by 200 seconds, bringing it down to about 900 seconds instead of 1100.
Though my one question remains unanswered and that is: Why does my PC complete it in such a long time when it used to do it in < 2 minutes previously (the dataset has not changed by even a single bit). There is no way I can bring it down.
I've noticed when the exact same project is run on my friend's machine with only difference of 8 vs 16 Gigs of RAM, the resource load completed in 20 seconds. It is unbelievable!

[Updated on: Tue, 03 March 2015 13:50]

Report message to a moderator

Re: EMF Resource load from InputStream taking too long [message #1647633 is a reply to message #1647559] Tue, 03 March 2015 14:33 Go to previous message
Ed Merks is currently offline Ed MerksFriend
Messages: 30452
Registered: July 2009
Senior Member
Syed,

It's hard to speculate...

On 03/03/2015 2:48 PM, Syed Rizvi wrote:
> Thanks Ed, using the options improved the resource loading by 200
> seconds, bringing it down to about 900 seconds instead of 1100. Though
> my one question remains unaswered and that is: Why does my laptop
> complete it in such a long time when it used to do it in < 2 minutes
> previously. There is no way I can bring it down. I've noticed when the
> exact same project is run on my friend's machine with only difference
> of 8 vs 16 Gigs of RAM, the resource load completed in 20 seconds. It
> is unbelievable!
Previous Topic:EcoreUtil.EqualityHelper and 'ordered' feature
Next Topic:org.eclipselabs.emfjson on Maven repository?
Goto Forum:
  


Current Time: Fri Sep 20 09:54:55 GMT 2019

Powered by FUDForum. Page generated in 0.01578 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top