Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Error while parsing action - only when multithreaded
Hi David,

Glad to help! I believe the issue you're seeing has already been identified and fixed in current master:

https://github.com/locationtech/geomesa/commit/893549ea47cd07bfcd8257087e200c37609fa312

This will be included in the next release (1.3.2), which should be in the next week or so. Alternatively, you can use version 1.3.2-SNAPSHOT and include our snapshot repository in your pom:

  <repository>
    <id>geomesa-snapshots</id>
    <url>https://repo.locationtech.org/content/repositories/geomesa-snapshots</url>
    <releases>
      <enabled>false</enabled>
    </releases>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </repository>


For explanation, we track various stats on ingested data to aid with query planning and counts. The bug involves merging stats together, hence why you don't see it with a single threaded ingest, as there is no need to merge.

Thanks,

Emilio

On 04/25/2017 07:22 PM, David Boyd wrote:
All:

   First a big thank you to everyone on this list for responding to my issues.
Special shout out to Emilio who helped me through several issues.

I have another weird one.

My application reads lines of data from a kafka topic and creates objects and features
from those lines.

The code is set up to support multiple threads (each with it's own runnable) to get some
parallelism (eventually this will be a spark job but this was quickest).

In any case I run my data file through with only a single thread persisting and the
everything runs clean.

However, when I set up multiple threads I get this error:

2017-04-25 15:36:10,184 | ERROR | [Thread-12] | (GDELT_Consumer.java:122) - Error while parsing action 'stat/OneOrMore/ZeroOrMore/Sequence/org$locationtech$geomesa$utils$stats$StatParser$$singleStat/org$locationtech$geomesa$utils$stats$StatParser$$histogram/org$locationtech$geomesa$utils$stats$StatParser$$histogramAction1' at input position (line 1, pos 998):
Count();MinMax("AgentGeoLocation");MinMax("recordKey");MinMax("AgentCode");MinMax("AgentName");MinMax("NameMetaphone");MinMax("AgentCountryCode");MinMax("AgentKnownGroupCode");MinMax("AgentEthnicCode");MinMax("AgentReligion1Code");MinMax("AgentReligion2Code");MinMax("AgentType1Code");MinMax("AgentType2Code");MinMax("AgentType3Code");MinMax("AgentGeoType");MinMax("AgentGeoFullname");MinMax("AgentGeoCountryCode");MinMax("AgentGeoADM1Code");MinMax("AgentGeoADM2Code");MinMax("AgentGeoFeatureID");TopK("recordKey");TopK("AgentCode");TopK("AgentName");TopK("NameMetaphone");TopK("AgentCountryCode");TopK("AgentKnownGroupCode");TopK("AgentEthnicCode");TopK("AgentReligion1Code");TopK("AgentReligion2Code");TopK("AgentType1Code");TopK("AgentType2Code");TopK("AgentType3Code");TopK("AgentGeoType");TopK("AgentGeoFullname");TopK("AgentGeoCountryCode");TopK("AgentGeoADM1Code");TopK("AgentGeoADM2Code");TopK("AgentGeoFeatureID");Histogram("AgentGeoLocation",10000,"POINT (-190 -100)","POINT (-170 -80)");Histogram("recordKey",1000,"00251fe1-1754-46a9-bec0-efbbfc558579","ff7c316b-6245-4838-a9d2-bfb73eb2cd6a");Histogram("AgentCode",1000,"AFG","tam");Histogram("AgentName",1000,"ABBOT","ZIMBABWE");Histogram("NameMetaphone",1000,"0F","XNSP");Histogram("AgentCountryCode",1000,"AFG","ZWE");Histogram("AgentKnownGroupCode",1000,"UNO0","UNOz");Histogram("AgentEthnicCode",1000,"chm","tam");Histogram("AgentReligion1Code",1000,"BUD","MOS");Histogram("AgentReligion2Code",1000,"CPT","LDS");Histogram("AgentType1Code",1000,"AGR","UAF");Histogram("AgentType2Code",1000,"BUS","MIL");Histogram("AgentType3Code",1000,"EDU","LAB");Histogram("AgentGeoType",1000,"0","5");Histogram("AgentGeoFullname",1000,"Acton, Shropshire, United Kingdom","Zimbabwe");Histogram("AgentGeoCountryCode",1000,"AE","ZI");Histogram("AgentGeoADM1Code",1000,"AE03","ZI");Histogram("AgentGeoADM2Code",1000,"12608","WA061");Histogram("AgentGeoFeatureID",1000,"-1201139","ZI");Frequency("recordKey",20);Frequency("AgentCode",20);Frequency("AgentName",20);Frequency("NameMetaphone",20);Frequency("AgentCountryCode",20);Frequency("AgentKnownGroupCode",20);Frequency("AgentEthnicCode",20);Frequency("AgentReligion1Code",20);Frequency("AgentReligion2Code",20);Frequency("AgentType1Code",20);Frequency("AgentType2Code",20);Frequency("AgentType3Code",20);Frequency("AgentGeoType",1);Frequency("AgentGeoFullname",20);Frequency("AgentGeoCountryCode",20);Frequency("AgentGeoADM1Code",20);Frequency("AgentGeoADM2Code",20);Frequency("AgentGeoFeatureID",20)







^

java.lang.IllegalArgumentException: requirement failed: Value out of bounds ([-180.0 180.0]): -190.0
org.parboiled.errors.ParserRuntimeException: Error while parsing action 'stat/OneOrMore/ZeroOrMore/Sequence/org$locationtech$geomesa$utils$stats$StatParser$$singleStat/org$locationtech$geomesa$utils$stats$StatParser$$histogram/org$locationtech$geomesa$utils$stats$StatParser$$histogramAction1' at input position (line 1, pos 998):
Now, I put debugs in my program to dump out all the coordinates, there are no -190 values.
I even put fencing in my code with logging to prevent out of range values.
I examined the file and no -190 coordinate exists.

It looks to be some sort of timing issue related to the indexes getting updated but I have no clue.

It does not occur at the same place or the same number of times in the file.

Full stack trace starts at line 4850 of the attached file.

Any thoughts?

Email me and I will find some way to send source and/or a shaded jar for debugging. 





_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top