[geomesa-dev] [JIRA] (GEOMESA-3199) Behavior of operator '>' is inconsis

["Kristin Cowalcijk (JIRA)" <jira@xxxxxxxxxxxxxxxxxxxxx>]

Title: Message Title

Kristin Cowalcijk created an issue

GeoMesa / GEOMESA-3199

Behavior of operator '>' is inconsistent with other comparison operators when comparing string values

Issue Type:	Bug
Affects Versions:	3.4.0
Assignee:	Unassigned
Components:	Query Planner
Created:	13/May/22 4:18 AM
Environment:	Reproduced on GeoMesa 3.4.0 and HBase 2.3.4
Priority:	Major
Reporter:	Kristin Cowalcijk

Using binary query operator '>' on string column containing numeric string values yields strange results. We’ve attached a sample data and converter configs for reproducing this problem.

Ingest test data

$ geomesa-hbase ingest -c geomesa_test:tmp -f seq_no_index -s seq_no_index.conf -C seq_no_index.conf seq.csv
$ geomesa-hbase ingest -c geomesa_test:tmp -f seq_with_index -s seq_with_index.conf -C seq_with_index.conf seq.csv

Now we have two schemas containing exactly the same data. Both datasets contains an attribute S containing numeric string values. seq_with_index has the string attribute S indexed while seq_no_index do not have attribute indexes.

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index
id,I:Integer,S:String
"#1",1,1
"#10",10,10
"#11",11,11
"#12",12,12
"#13",13,13
...

Reproduce

String comparison uses lexicographical order when filtering by attribute S using binary comparison operator '<':

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index -q "S < '2'"
id,I:Integer,S:String
"#1",1,1
"#10",10,10
"#11",11,11
"#12",12,12
"#13",13,13
"#14",14,14
"#15",15,15
"#16",16,16
"#17",17,17
"#18",18,18
"#19",19,19
INFO  Feature export complete to standard out for 11 features in 4467ms

When filtering using '>' operator, it seems to convert both sides to numeric value first, then compare on numeric values using natural order:

$ geomesa-hbase export -c geomesa_test:tmp -f seq_no_index -q "S > '8'"
id,I:Integer,S:String
"#10",10,10
"#11",11,11
"#12",12,12
...
"#9",9,9
"#90",90,90
"#91",91,91
"#92",92,92
"#93",93,93
"#94",94,94
"#95",95,95
"#96",96,96
"#97",97,97
"#98",98,98
"#99",99,99
INFO  Feature export complete to standard out for 91 features in 4424ms

If S attribute was indexed, '>' operator would conform to lexicographical order:

$ geomesa-hbase export -c geomesa_test:tmp -f seq_with_index -q "S > '8'"
id,I:Integer,S:String
"#80",80,80
"#87",87,87
"#88",88,88
"#89",89,89
...
"#99",99,99
"#82",82,82
"#83",83,83
"#86",86,86
"#90",90,90
INFO  Feature export complete to standard out for 21 features in 4269ms

We’ve tested various binary comparison operators and found that '>' behaves differently on non-indexed string attribute.

Operator	Non indexed	Indexed
=	lexicographical	lexicographical
<	lexicographical	lexicographical
<=	lexicographical	lexicographical
>	numerical when compared value can be parsed as number	lexicographical
>=	lexicographical	lexicographical

Expected behavior

Operator '>' should always compare string values in lexicographical order.

Add Comment

Get Jira notifications on your phone! Download the Jira Cloud app for Android or iOS
This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100198-sha1:8950bff)