Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [stem-dev] STEM Feature for single KML file animation -- Dev Questions 
Hello James,
No worries... I'm sure you're very busy. I appreciate the time you have taken to explain the parameters.
A. Thanks. I will see if I can extract the population data using the Java code you've included.
B. Also great info here. I'll hunt down the Sequencer object to get the start and end dates. 

C. I wrote: "Why is my simulation's 'cycle' counter jumping around..."
You were absolutely correct... it turned out to be a threading problem. I resolved it by doing the following: 

  -Increased the STEM simulation step time from 250ms to 750ms
  -Set the STEM Process Priority to High in the OS
  -Allocated STEM 2GB/2GB of memory on launch
-Slowed the new KML writer so it writes only once per 5 cycles (rather than each cycle). -Reduced the complexity of the simulation... the max output of the KML file can only be about 400MB.
I learned that if the single KML file is greater than about 400 MB, Google Earth will crash when loading it.
So unfortunately, this limits the 'granularity' of KML file animations. There is probably a way around this... at present, I write the Polygons to represent a county repeatedly (at each simulation step). This makes county-level data cause the KML animation file to be very large (hundreds of megs) .. There is lots of redundant data since the same polygon coordinates gets written over and over, repeated on each TimeSpan.
I see three workarounds to the large KML output file sizes... The key will be to eliminate the KML polygon data redundancy over TimeSpans...
C.1. Utilize image overlays (jpegs) for countries, regions and counties etc... rather than having Google Earth draw STEM's region polygons over and over, use colored JPG overlays, shape files, something like that.
C.2. OR: Have GE import STEM county-level coordinates as a resource 'as-needed', and use Google Earth's 'Feature' loading capability to cache only the data in view. This way it doesn't all get loaded into memory at once, and might eliminate redundancy.
C.3. OR: Quick and Dirty solution: Reduce the number of points (resolution) that is used to represent county data when writing to the KML file by Ramer–Douglas–Peucker algorithm. So the county polygons will not be so complex in the KML, and will take up less space. ie. Reduce a county polygon from ~50 coordinate pairs to 10 coordinate pairs. 

http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm
D. Thanks. I think I will try to use your standard methods (the Aspect object, scaling, etc) for coloring the KML output polygons. That way everything is consistent across the STEM platform. I want the KML animation to support log scaling , just like the GUI does.
E. Okay. I'm pretty sure the STEM GE Servlet stuff is broken on my Windows 7 x64 environment. For some reason, the web server has not ever run locally (nothing is bound on port 8080 or whatever it is). I think this functionality might be broken or deprecated in the current versions of Java and Google Earth.
The existing GE functionality in the latest STEM release was able to do two major tasks (1) write a directory with one kml file per iteration -- numbered, or (2) launch Google Earth to load a control.kml file once per iteration. I don't think I was able to get the GE Servlet stuff working at all, but I will test again.
F. > What is the difference between KmlDisplay, KmlDisplayCustom, and KmlDisplaySelection? Is any of this functionality deprecated? "Some of these are really old and may go back to older versions of google earth."
Okay, I may try to re-factor some of this code. I think KmlDisplaySelection was meant to display a sub-set of a geographic area to KML, but I don't know if it's working. I think some of this code can be pulled out, fixed, or refactored, especially if the Servlet stuff is actually broken for everyone.
Anyway, I will keep digging and clean up the code as I go. I'd like to pull out or fix any deprecated features that are broken.
Also, I'm tempted to convert the KML write functionality to a Logger object, unless there is a reason not to do that... 

Thanks,
Alex (aka Vincent R)


On 2014-10-07 16:38, James Kaufman wrote:

Vincent,
Sorry for the slow response.
Re your questions

 > A. Where is the 'county' population data stored? For example, how
do I go from a key like 'US-NY-36039' (county identifier) to retrieve
the original population in the county and the current population in
the county at a time-step? (to determine number of fatalities , or
total number of Infections as an absolute value rather than percent).

if you have the node you can use something like the following
 EList<NodeLabel> labelList = node.getLabels();
 FOR(NodeLabel nl:labelList) {
 IF(nl INSTANCEOF PopulationLabel) {
 PopulationLabel pl = (PopulationLabel) nl;
 DOUBLE population = pl.getCurrentPopulationValue().getCount();
 }
 }
You should be able to get the node from the graph. (ie
graph.getNode(nodeURI))

 > B. Where can I find a simulation's actual 'start' and 'end' dates
in String or DateTime format? What object and where is it?
 Get the Sequencer from the scenerio (from the simulation). The
sequencer has the start and end dates.
the sequencer has method getStartTime and getEndTime
they return "STEMtime" objects but STEMtime.getDate() gives you a
Date.

 > C. Why is my simulation's 'cycle' counter jumping around at the
beginning? It starts at cycle=0, then goes to cycle=2, then to
cycle=7,8,9,10 etc?
 The gui runs in it's own thread so you are running either a big or
small simulation. On the menu bar you can go to >window>preferences.
then >STEM>Simulation Managment and either increase the simulation
delay so the gui keeps up. Or decrease it if you need.

 > D. Is there a 'proper' way to go about scaling an 'I' value to a
polygon color? Should I just use the Aspect object? Any tips on this?

I think it just depends what you are trying to show. We usually
normalize I by the population but you might be interested in small
values at the start of an epidemic so you might need a scale factor or
log scaling.

 > E. Are there any known bugs with the GE Servlet stuff? I'm getting
some Servlet exceptions (can't connect to local web server). Is this
feature broken (the network pipe)? It's not
important -- just curious.

 We have not run this in a very long time. Matt, any ideas on this?

 > F. Why are there multiple classes which seem to be involved in
writing KML files... What is the difference between KmlDisplay,
KmlDisplayCustom, and KmlDisplaySelection? Is any of this
functionality deprecated? Some of these are really old and may go back
to older versions of google earth.

I'm sorry not to be more helpful....

Best Regards,
 Jamie

 IBM Almaden Research Center, 650 Harry Rd.
 San Jose, CA 95120-6099
 email: jhkauf@xxxxxxxxxx
 phone: (408) 927-2477 (tie 457-2477)

Back to the top