Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] -EXT- ClassCastException error in geomesa_pyspark 4.0.5
  • From: "Lahr-Vivaz, Emilio" <Emilio.Lahr-Vivaz@xxxxxxxxx>
  • Date: Thu, 27 Mar 2025 12:32:42 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ga-i3.com; dmarc=pass action=none header.from=ga-i3.com; dkim=pass header.d=ga-i3.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector5401; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=htl6+va359rB4cqYV4sbgdLHSgHddQZAM1MMr2gdPwU=; b=OMEgiyloAcvYFc/ByO2H3sODddrk7t4t5R9GguZN69TBFM0831UoO9SSzXZHRNNiTwygqGL8dWHocGFD2+2OqA07TaBDIKlNkvuwM5FLwjn+Iw6wBOFrVIbdxL2+Ltt/no7bgQ4syK/qwjN7aXw03mthhQpAqYzTgEtZAMXOGLbLcxOjM6FPYHQQMJ7rX0Zvcc3x53mVBave0yjb9yl0t4+xrVh0g9aLIfC5NvKZUkltjH/1cMXk3Dc7BmnjuLnqBMiL1ROb7mAK+7p+LP3v7/C8z4B0W06FcMx182sjoC7B8j2pLzpKOtHR0Ru/KtWDeF818jgjuJIrQRQHAnt/Dg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector5401; d=microsoft.com; cv=none; b=1sPnZU5aMdjzQmNTT9N468BYGLX7nGBwpQDm9hOIGHbSmqudMeRR6/HrlLVlhm1CVkOLSZIZlB+DkeWqVatsoXSZEDZ3XRWcWEn/PuX9SAbo2hcpFk5/aXa7Oq+B6r1Oeir0bwxn08mtXshEg/QM1J0X2ged6vt16yCa7dQb2EU/x6Mzz6f+OC2NiENaHgfffwdTF2dd0MoUQsAFUJZ2w4SL0EdGAyv9q6coeTLoSTgWEqrFHcfD+bD7H+SCApSVmP28uaSZwlYfo0zT9/VZnM/41+EGExOgvNokMvqejCK+JqWmYLpDAd+ek2BwrZNUHGgd8+a/FpJVA1wDb/hQjw==
  • Delivered-to: geomesa-users@xxxxxxxxxxx
  • List-archive: <https://dev.eclipse.org/mailman/private/geomesa-users/>
  • List-help: <mailto:geomesa-users-request@eclipse.org?subject=help>
  • List-subscribe: <https://dev.eclipse.org/mailman/listinfo/geomesa-users>, <mailto:geomesa-users-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://dev.eclipse.org/mailman/options/geomesa-users>, <mailto:geomesa-users-request@eclipse.org?subject=unsubscribe>
  • Msip_labels:
  • Thread-index: AQHbnG1y39bTRsxH90izKuOoNS+eNbOG77IK
  • Thread-topic: -EXT-[geomesa-users] ClassCastException error in geomesa_pyspark 4.0.5
Hello,

I'm not too familiar with the pyspark side of things, but I believe you need to set the serialization options mentioned here:

https://www.geomesa.org/documentation/stable/user/spark/core.html#simple-feature-serialization

Thanks,

Emilio Lahr-Vivaz
General Atomics - I3
From: geomesa-users <geomesa-users-bounces@xxxxxxxxxxx> on behalf of zhou lihuang via geomesa-users <geomesa-users@xxxxxxxxxxx>
Sent: Sunday, March 23, 2025 11:41 PM
To: zhou lihuang via geomesa-users <geomesa-users@xxxxxxxxxxx>
Cc: zhou lihuang <zlh_0923@xxxxxxxxxxx>
Subject: -EXT-[geomesa-users] ClassCastException error in geomesa_pyspark 4.0.5
 

WARNING:  This message is from an external source.  Evaluate the message carefully BEFORE clicking on links or opening attachments.

hello everyone:

I used pyspark to query data in hbase, and the code returned the following exception:
code is as follows:
 
the error message is as below:
java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f of type scala.Function3 in instance of org.apache.spark.rdd.MapPartitionsRDD

The env is :
geomesa_pyspark: 4.0.5
spark: 3.3.0
python: 3.8

How can I fix this problem now? 

Thank you everyone.

Best Regards, 
Mike 

Back to the top