Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [stem-dev] [Bug 501321] New: Pauses before completion of simulation

Thank you Emily.

From your description below I think what is really varying in the two difference cases is "contact rate" between animals on different farms. As such I would not model this by the transmission rate in the disease model, but rather the mixing rate in the population model. Please take a look at the documentation for creating a custom graph using Pajek. This was implemented courtesy of Chris Thoens of the BfR.

https://wiki.eclipse.org/Importing_a_Pajek_Graph
and the example scenario
http://www.eclipse.org/stem/download_sample.php?file=MultiPopulationExample_PajekGraphs.zip

the following may also be useful
https://wiki.eclipse.org/Visualizing_and_Editing_Graphs


I would start by creating a toy model on a lattice with sites representing different farms. You can later put these as nodes below the real geographic nodes.
You could create some nodes representing common grazing areas, move all the animals from nearby farms in and out of those areas using migration edges you can define in pajek. You can specify migration rates so you might even look for critical mixing rates. Chris's food production downloadable scenarios may be another good example to look at
https://wiki.eclipse.org/Sample_Projects_available_for_Download#1._Beef_Production
and
https://wiki.eclipse.org/Sample_Projects_available_for_Download#2._A_scenario_for_modelling_the_transmission_of_Salmonella_to_pigs.2C_pork_and_humans

http://www.eclipse.org/stem/download_sample.php?file=BeefArgentina.zip


If you would like Judy can also add this to the agenda for discussion at our next call....


Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)





From:        Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        James Kaufman/Almaden/IBM@IBMUS
Date:        10/07/2016 07:15 AM
Subject:        Re: [Bug 501321] New: Pauses before completion of simulation




Hi James,

Yes, it's great that's it's now working!

The scenario I sent you was just one I was working on that is vaguely related to what I want to do to see how STEM works.
Now it's working I am trying to start my actual model. However, I'm not sure whether STEM is currently able to do all the things I want to do.

I'm aiming to make a spatio-temporal model showing the transmission of a parasitic disease in sheep (scab) across the UK. This model will be used by UK policy makers to inform their decision making on how best to control this disease (e.g if a particular high risk region is inoculated, how does this affect infection in surrounding areas?).

The model will include the differences in transmission rates found in different counties (what you call "states" in the USA). I have identified which counties use "common grazing", a practice in which sheep from all local farms graze together on common land. This practice greatly increases the transmission rate between individuals in the county. I've been thinking about how I could model this using STEM.

One idea I've had is that I could have graphs within graphs (although I'm not sure whether this is possible in STEM?)
I have used the GBR_1.graph GBR(1) Nodes graph which is already in STEM (as well as the corresponding area and common border relationship graphs). These have each county as one node.  Ideally, I would like to make a graph for each of the nodes (counties). These graphs would have nodes for all the sheep farms in the county it represents. In counties which use common grazing, the transmission rate between the farms would be greater than in those that don't. Transmission between counties would be occurring at the same time as transmission between farms within in a county.

A simpler thought I've had is to somehow make the initial prevalence of the disease higher in the counties which use common grazing. However, I can only currently initially infect one county. Is there a way to set the initial prevalence of the disease for each county?

Once I've got this model of the transmission of scab up and running I would like to be able to do other things such as:
1. See what the affect of inoculating the population does (I know that this can be done in STEM, although not sure if it can be done for specific locations).
2. See how overall transmission is affected by quarantining certain high risk areas (I think this could be done by just changing edges in graphs?)
3. Make the model more realistic (I would do more research to make my parameter values more accurate).

Sorry for the very long email, but I thought it would just be worth checking with you about whether this is realistic and getting some guidance before wasting my time on something that might never work.
 
I've asked Judy to add me to the monthly call. Although I don't feel I could contribute much myself yet, it will be interesting to hear the discussions.

Best regards,

Emily





From: James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
06 October 2016 16:58
To:
Emily Mitchell
Cc:
Stefan Edlund; Judy Douglas
Subject:
Re: [Bug 501321] New: Pauses before completion of simulation

 
Thank you Emily,
I'm very happy we were able to get things to work !


I'm not 100% sure what the problem was but I have a couple ideas to investigate (If I can find a windows machine :-)
Your sequencer which you created for testing had a very short run duration. It's possible there is an issue with the code related to how often the logger tries to log data.
There might also have been a threading issue (which I why I asked you to check for and remove any delay). I'll document all this in the bug so we can check it out in the future.


In addition to the wiki there are some YouTube tutorials (linked to in the wiki) that might be useful. We also have a monthly phone call. If you do wind up using them you might want to get an Eclipse id
which would allow you to:


i) Join the project
ii) Make changes to the wiki - helping with tips for other future users
iii) add your bio and publications with STEM if you want to
iv) Sign up for the stem-dev mailing list.


You can also contribute demo scenarios if you think that would help others


Also, if you would like, please let Judy know and she could invite you to our Monthly call.


Please feel free to email me if you have future problems. Happy to do what I can....


Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)





From:        
Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
10/06/2016 04:27 AM
Subject:        
Re: [Bug 501321] New: Pauses before completion of simulation




Dear James,

It now finishes at 100%! What do you think the problem was (just so that I don't make the same mistakes again!)?

In case of future issues while I'm learning to use STEM, is there anything else other than the STEM wiki that gives any help on using STEM that you could point me to?


I really appreciate how you've gone out of your way to help me. Would I be able to contact you again in future with other pressing issues (I will try all other avenues first as I'm sure you must be busy)? If any of the work I do is published, I would of course mention you in the acknowledgements.

Best regards,

Emily

 



From:
James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
06 October 2016 00:32
To:
Emily Mitchell
Subject:
Re: [Bug 501321] New: Pauses before completion of simulation


Thanks Emily.
That looks to be an Eclipse Workbench Error (not a STEM error) but I noticed in the stack trace that workbench was disposed.
Can you please try a few more things?

1) Please launch STEM and go to  >Window>Preferences>STEM>SimulationManagment
and make sure that the pause is either unchecked or set to 10msec. By default is it something like 200msec. You don't need a long delay for real science (the delay is for demos)

2) Create a new sequencer that runs for something like 1 year.

3) Replace the short sequencer in your scenario with the 1 year version

4) Click on the project and refresh the workspace

5) Run your scenario in Windows 10 - I hope till it finishes and ends in a pause state.

6) Please do not close the workbench. Just hit the square red button. Then count to ten. Then click on the project and refresh the workspace again.

7) Look for the new log files. See where they end.

Of course if it fails early please send me the stack trace again. I'd like to see how the logger behaves with a more realistic sequencer time window.....




Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)






From:        
Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
10/05/2016 08:50 AM
Subject:        
Re: [Bug 501321] New: Pauses before completion of simulation




Hi James,

I've tried installing STEM on my laptop at home (which is Windows 10).

This time there were some extra files in my recorded simulations folder for the scenario with and the scenario without predicates (see attached).

However, the simulation still stopped at 94%.

These are the two error logs that were produced.  





If you have any idea about what's going on please let me know!

Thanks so much,

Emily



From:
James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
29 September 2016 17:10
To:
Emily Mitchell
Subject:
Re: [Bug 501321] New: Pauses before completion of simulation

Thanks Emily,
This is a mystery. Could you please open the error log view, Clear any error, and duplicate again? If anything shows up in the log please click on it and send me the full stack trace.
Also, could you please try your same project on a Windows 8 or Windows 10 machine?

One last question, where did you install STEM? If could be your operating system and/or  security tools are preventing write access to the log location. You might try installing in your Documents folder and try again from there.
Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)






From:        
Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
09/29/2016 02:14 AM
Subject:        
Re: [Bug 501321] New: Pauses before completion of simulation




Dear Jamie,

Thanks once again for your help. Unfortunately, it still doesn't seem to be working. See my comments on your email below...



From:
James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
27 September 2016 17:05
To:
Emily Mitchell
Subject:
Re: [Bug 501321] New: Pauses before completion of simulation


Hi Emily,
That's pretty strange.
0) What version of STEM are you running? It should be the latest "integration build"
I've just downloaded the newest integration build but I'm experiencing the same problem.  
1) What operating system are you using?
I'm using Windows 7 enterprise
2) What version of Java are you running? (please open a console and type  > java -version
Version 8 update 101
3) After the simulation pauses at the end of the run, do you hit the red square stop button? You should.
4) Once you hit stop, please click on the blue project folder in the project explorer (the project folder not the recordedsimulations folder), then RIGHT CLICK, and then select "Refresh"
5) Open the recorded simulations folder. Do you see in you workspace project something like this?


I followed instructions 3-5 but the same thing occurred- it stopped at 94% and there were only html files like the ones I sent you and no csv files.


The data for those variable you asked the logger to log should show up in csv files like the ones shown above.

Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)






From:        
Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
09/27/2016 02:18 AM
Subject:        
Re: [Bug 501321] New: Pauses before completion of simulation




Hi Jamie,

The only log files that are produced are the html files (attached), but according to the YouTube tutorials there should be more folders and some csv files in the recorded simulations folder following logging. Perhaps this is related to the 94% completion (i.e it isn't completing and so these extra files are not being produced?).

Thank you for Tabea's contact details, I have sent her an email.

I will get in touch with Judy about the monthly call.

Best regards,

Emily



From:
James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
26 September 2016 19:25
To:
Emily Mitchell
Subject:
Re: [Bug 501321] New: Pauses before completion of simulation

Hi Emily,

I think the indicator that shows 94% on completion may only be the GUI not showing the actual completion time. Could you please check your log files to see if data for the correct ending time is being logged?

The scientist working on the global livestock data is Tabea Kropf <Tabea_Kropf@xxxxxxxxxxx>. She sometimes joins our regular calls.

The monthly call is simply an opportunity for STEM users and members of the STEM community to talk, give updates, and ask questions or raise issues.

Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)






From:        
Emily Mitchell <Emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
09/26/2016 08:38 AM
Subject:        
Re: [Bug 501321] New: Pauses before completion of simulation




Hi Jamie,

I'm sorry for the late reply, I have been away at a conference and then on holiday.

Thank you so much for all your help, it is much appreciated.

I have looked at what you sent me and was pleased to see that the logger is now working, so thank you for that. However, the simulation seems to always still pause at 94%. Do you have any idea how to fix this?

I also could not find the information about the scientist a
t SwissRE who is adding global grid livestock data to STEM from the link you gave me. Would you be able to send it again and give me the name of the scientist (if known)?

What usually happens during the monthly call?


Best wishes,

Emily  



From:
James Kaufman <jhkauf@xxxxxxxxxx>
Sent:
14 September 2016 18:05
To:
Emily Mitchell
Cc:
Douglas Judy; Stefan Edlund
Subject:
RE: [Bug 501321] New: Pauses before completion of simulation

Hi Emily,

I tried out your scenario and found a few issues. Please find attached below an updated version of your scenario. For each of the issues I updated the doc on our wiki to help future users who may have similar issues.
fyi You should also feel free to make updates like that to wiki.

Here is what I found:
1) Please don't use spaces in folder, project, or package names. I renamed your project replacing space with '_'. I also updated all of the contained files. Windows has problems with the space character.

2) Please always log to default directory (we have trouble with people not finding the logs when they customize). It will auto increment log data folder names for you
by run time stamp

With these two changes to your original scenario the logger is working and places the files in the right place.
Still stops at 94% though!

Regarding the 'Copy' scenario,
3) You can't really "rename" a scenario the way you did to create "CopyofWholeUKlatest.scenario" as all the plugins inside of it depend on the uri of the scenario. The right way to do this is to create a scenario
and just drag the contents in. 'Copy' would be ambiguous because in principle one would need both deep and shallow copy so we leave it to the use to construct scenarios (thus no ambiguity). This could be a future feature request.

I made a new scenario for you and dragged in all the elements you had in the 'Copy'. I noticed you were missing a Predicate for your Trigger. Also your modifier has a
series of values. Use of a series is a feature we implemented for running in batch mode (which also uses modifiers) but typically not for use with a single trigger.
In the archive attached I added the missing predicate and changed the modifier so you can see how a single value change works in a predicate,modifier,trigger tuple.
Please see the new wiki documentation for interventions (which explains a non-fatal error that can occur when editing a predicate).

https://wiki.eclipse.org/Triggering_interventions#Creating_a_Predicate

Please let us know if you need more help and please feel free to contact Judy if you would like to join our call. fyi there is a scientist at SwissRE who is adding global grid livestock data to STEM.

https://wiki.eclipse.org/Importing_and_Exporting_Projects




Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)






From:        
Emily Mitchell <emjmitch@xxxxxxxxxxx>
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
09/14/2016 05:52 AM
Subject:        
RE: [Bug 501321] New: Pauses before completion of simulation




Dear Jamie,

Thank you for your reply. I did create a logger and drag it into my scenario. I've attached the error log showing the most recent errors after having cleared the log and hitting run on my scenario  "WholeUKlatest.scenario.Scenario

Best regards,

Emily


To: emjmitch@xxxxxxxxxxx
CC: judy.douglas07@xxxxxxxxx
Subject: Fw: [Bug 501321] New: Pauses before completion of simulation
From: jhkauf@xxxxxxxxxx
Date: Tue, 13 Sep 2016 09:24:59 -0700


Welcome to STEM.
I expect this is not a bug. If you would like to zip up the scenario you built and send me that archive I'd be happy to take a quick look.
Did you create a logger and drag it into your scenario?
Also, please open the "error log" view and see if you are getting any exceptions. Clear the log before you hit run.

Also,
Please email Judy Douglas if you would like to be invited to our monthly community call.

Best Regards,
Jamie

IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477  (tie 457-2477)



----- Forwarded by James Kaufman/Almaden/IBM on 09/13/2016 09:22 AM -----


From:        
bugzilla-daemon@xxxxxxxxxxx
To:        
James Kaufman/Almaden/IBM@IBMUS
Date:        
09/13/2016 02:45 AM
Subject:        
[Bug 501321] New: Pauses before completion of simulation




https://bugs.eclipse.org/bugs/show_bug.cgi?id=501321
Product/Component: STEM / Simulation

    Bug ID: 501321
   Summary: Pauses before completion of simulation
Classification: Technology
   Product: STEM
   Version: unspecified
  Hardware: PC
        OS: Windows 7
    Status: NEW
  Severity: normal
  Priority: P3
 Component: Simulation
  Assignee: stem.simulation-inbox@xxxxxxxxxxx
  Reporter: emjmitch@xxxxxxxxxxx

(Version 3.3.0)

I am fairly new to STEM so this may not be a bug, but rather an issue due to my
inexperience.

I've created two different scenarios and for both of these the simulation never
reaches 100% progress, but instead pauses at 94% or 96%. Pressing play does not
have any effect.

This also seems to prevent anything from being logged- there are no files or
folders in the recorded simulations folder even though I've followed all the
instructions in the tutorials.

--
You are receiving this mail because:
You are watching the assignee of the bug.



[attachment "Whole UK 2.zip" deleted by James Kaufman/Almaden/IBM] [attachment "errors.JPG" deleted by James Kaufman/Almaden/IBM]



[attachment "ScenarioLog.html" deleted by James Kaufman/Almaden/IBM] [attachment "ScenarioLog.html" deleted by James Kaufman/Almaden/IBM]






Back to the top