Hi Jamie,
That makes sense, I can definitely see why you don't produce summary data anymore.
I do often choose to change the log interval so that the recorded simulations files are smaller, however, for what I am trying to do (calculating how many farms have an incidence >1 per iteration), I need to have each
iteration recorded.
What about my other suggestion, to change the format of how the results are recorded by STEM in the csv files in the recorded simulations folder? Would it be difficult to change it so that
instead of having nodes as individual columns, that there just be one column which specifies the node? This would greatly reduce the number of columns and increase the rows instead which would enable me to read even very large files into the software I am
trying to use to analyse the data.
So instead of the current format:
Iteration time stem://org.eclipse.stem/node/1 stem://org.eclipse.stem/node/2
0 2018-02-01 1 1
We would have:
Iteration time nodeID Incidence
0 2018-02-01 stem://org.eclipse.stem/node/1 1
0 2018-02-01 stem://org.eclipse.stem/node/2 1
On much smaller files, of course it is possible to modify the output myself in order to have this format. However, due to the sheer number of nodes I am using, and therefore the sheer number
of columns, I am not able to use my database management systems to store the data first before I modify it into this format.
I think this format would be useful to others too, as it is more compatible with a number of data analysing softwares/languages.
How difficult do you think it would be to change the format STEM records the data to what I've suggested above?
Best wishes,
Emily
Emily Nixon
PhD Student
Demonstrator
School of Biological Sciences
University of Bristol
Bristol Life Sciences Building
24 Tyndall Avenue
Bristol
BS8 1TQ
Tel +44 (0)117 394 1389