[geomesa-dev] [JIRA] (GEOMESA-3200) CQL filters containing OR yields dup

["Kristin Cowalcijk (JIRA)" <jira@xxxxxxxxxxxxxxxxxxxxx>]

Title: Message Title

Kristin Cowalcijk created an issue

GeoMesa / GEOMESA-3200

CQL filters containing OR yields duplicated query results

Issue Type:	Bug
Affects Versions:	3.4.0
Assignee:	Unassigned
Components:	Query Planner
Created:	13/May/22 4:42 AM
Environment:	Reproduced on GeoMesa 3.4.0 and HBase 2.3.4
Priority:	Major
Reporter:	Kristin Cowalcijk

Running queries using CQL containing OR subexpression sometimes yield duplicated results. We’ve attached a sample data file and converter configs for reproducing this problem.

Ingest test data

$ geomesa-hbase ingest -c geomesa_test:tmp -f seq_no_index -s seq_no_index.conf -C seq_no_index.conf seq.csv
$ geomesa-hbase ingest -c geomesa_test:tmp -f seq_with_index -s seq_with_index.conf -C seq_with_index.conf seq.csv

Now we have two schemas containing exactly the same data. Both datasets have an attribute I containing numeric values. seq_with_index has the integer attribute I indexed while seq_no_index do not have attribute indexes.

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index
id,I:Integer,S:String
"#1",1,1
"#10",10,10
"#11",11,11
"#12",12,12
"#13",13,13
...

Reproduce problem 1

Query the dataset using CQL (I > 5 AND I < 10) OR I > 95 on schema with attribute index yield duplicated results (#96 to #99):

s$ geomesa-hbase export -c geomesa_test:tmp -f seq_with_index -q "(I > 5 AND I < 10) OR I > 95"
INFO  Running export - please wait...
id,I:Integer,S:String
"#99",99,99
"#98",98,98
"#96",96,96
"#97",97,97
"#6",6,6
"#9",9,9
"#99",99,99
"#8",8,8
"#98",98,98
"#7",7,7
"#96",96,96
"#97",97,97
INFO  Feature export complete to standard out for 12 features in 6386ms

Those duplicated results goes away when we run the same query on schema without attribute index:

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index -q "(I > 5 AND I < 10) OR I > 95"
INFO  Running export - please wait...
id,I:Integer,S:String
"#6",6,6
"#7",7,7
"#8",8,8
"#9",9,9
"#96",96,96
"#97",97,97
"#98",98,98
"#99",99,99
INFO  Feature export complete to standard out for 8 features in 4647ms

Reproduce problem 2

When the OR subexpression contains several range queries with overlapping ranges, running the query on indexed attribute would produce duplicated results:

$ geomesa-hbase export -c geomesa_test:tmp -f seq_with_index -q "I < 5 OR I < 4"
INFO  Running export - please wait...
id,I:Integer,S:String
"#2",2,2
"#3",3,3
"#1",1,1
"#2",2,2
"#3",3,3
"#1",1,1
"#4",4,4
INFO  Feature export complete to standard out for 7 features in 3765ms

Those duplicated results goes away when we run the same query on schema without attribute index:

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index -q "I < 5 OR I < 4"
INFO  Running export - please wait...
id,I:Integer,S:String
"#1",1,1
"#2",2,2
"#3",3,3
"#4",4,4
INFO  Feature export complete to standard out for 4 features in 4171ms

Expected behavior

No duplicated results should be retrieved when running queries mentioned above.

Add Comment

Get Jira notifications on your phone! Download the Jira Cloud app for Android or iOS
This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100198-sha1:8950bff)