[udig-devel] Efficiency problem with filtered Feature Collections

["Antonio Redondo Lora" <becarlora@xxxxxx>]

Hi udiggers ... 

We are involved in a plugin development in order to apply stadistic
operations (like count, sum, median, ... ) over two ayers from
shapefiles.

We start from a Multi/Polygonal (near 50 feats) and a Point layer (near
50000 feats), and the goal is to extend this Poly ne with new
attributes: the results of this operations over the points which are
contained in each polygonal geometry.

We develope it using geotools filters like DistanceGeometryFilter
(DWithin) and GeometryFilter (contains). The source code is similar to:

    ....
    
    FilterFactory filterFactory =
FilterFactoryFinder.createFilterFactory();
    GeometryDistanceFilter dWithinFilter =
filterFactory.createGeometryDistanceFilter(
        AbstractFilter.GEOMETRY_DWITHIN);
        
    LiteralExpression polyGeom; 
    FeatureCollection pointFeatsAux;
   
    for(Object poly : polyFeatureColl)
    {
        polyFeat = (Feature) poly;
        .....
        
        polyGeom = filterFactory.createLiteralExpression(
            polyFeat.getDefaultGeometry());                    
                
        dWithinFilter.addRightGeometry(polyGeom);
        pointFeatCollAux = FeatureCollections.newCollection();
       
pointFeatCollAux.addAll(pointsSource.getFeatures(dWithinFilter));    
        
        .....
        int size = pointFeatsAux.size();
        .....
        
        for(Object point : pointFeatsAux)             
            // Apply stadistic operation and classification
            // to the features contained in the poly
        }
        
        ....
        
        System.out.println(" Polygonal layer classified!");
    }

We have done several tests of efficiency over shapefiles around 50000
features and the results weren't very good :( It takes almost 2 minutes
to do the polygons classification. 
    
Obviously, there must have more efficient alternatives. We suspect that
the featureCollection result of the filter pointFeatsAux) isn't in fact
filtered ... Each time we iterate over this "filtered Collection", we
check that it takes the same time independly the number of "filtered
features" (pointFeatsAux.size()): we think it run all the primitive
collection again to obtain the filtereds. In summary, we have

    - 50*50000 its. to obtain the pointFeatsAux.size()
    - 50*50000 its. to apply the stadistic operator over each feature
point  

We are expecting to find a solution which only itearate    

    - 50*50000 its. to obtain pointFeatsAux
    - 50*pointFeatsAux.size() its. to process the stadistics
    
In conclussion, the time of filter proccess 
    
    pointFeatCollAux = FeatureCollections.newCollection();
    pointFeatCollAux.addAll(pointsSource.getFeatures(dWithinFilter));  
 
    // This is more efficient that 
    // pointFeatCollAux = pointsSource.getFeatures(dWithinFilter);!!!  
 
    
has zero time cost...but, the cost of the operation

    size = pointFeatsAux.size();
    
is around [ PolyLayer.numFeatures() * PointLayer.numFeatures() ].

Our questions ...

    a) Anyone knows if the Layer Theme Style dialog uses a similar
process to do classifications?
    b) Is there any alternative to this kind of geotools filters? 
    c) Is anyone at the same trouble?


NOTE: we use Visitor Interface to apply stadistics operations
    
    AverageVisitor averageVisitor = new
AverageVisitor(pointAttrib);			
    CountVisitor countVisitor = new
CountVisitor();			
    MinVisitor minVisitor = new
MinVisitor(pointAttrib);			
    MaxVisitor maxVisitor = new
MaxVisitor(pointAttrib);			
    MedianVisitor medianVisitor = new
MedianVisitor(pointAttrib);			
    SumVisitor sumVisitor = new
SumVisitor(pointAttrib);			
    StandardDeviationVisitor desvVisitor;
    
Thanks



 -----------------------------------------------------
  Este correo electrónico y, en su caso, cualquier fichero anexo
  al mismo, contiene información de carácter confidencial
  exclusivamente dirigida a su destinatario o destinatarios. Queda
  prohibida su divulgación, copia o distribución a terceros sin la
  previa autorización escrita de Dap. En el caso de haber
  recibido este correo electrónico por error, se ruega notifíquese
  inmediatamente esta circunstancia mediante reenvío a la dirección
  electrónica del remitente.
  ----------------------------------------------------------
  The information in this e-mail and in any attachments is confidential
  and solely for the attention and use of the named addressee(s).
  You are hereby notified that any dissemination, distribution or copy
  of this communication is prohibited without the prior written consent 
  of Dap. If you have received this communication in error,
  please, notify the sender by reply e-mail.
  -------------------------------------------------------------