hello.
I need your help.
sample stacktrace
```22/0...
# kotlin-sparkj
jaeyol.youn
06/29/2022, 12:43 AMhello.
I need your help.
sample stacktrace
Copy code
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 4
22/06/29 01:58:58 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on lniuhwk3388.nhnjp.ism:42755 in memory (size: 48.1 KB, free: 366.3 MB)
22/06/29 01:58:58 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on lniuhwk4040.nhnjp.ism:36704 in memory (size: 48.1 KB, free: 1458.6 MB)
22/06/29 01:58:58 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on lniuhwk3393.nhnjp.ism:33023 in memory (size: 48.1 KB, free: 1458.6 MB)
22/06/29 01:58:58 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on lniuhwk3870.nhnjp.ism:34781 in memory (size: 48.1 KB, free: 1458.6 MB)
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 23
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 15
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 5
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 1
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 6
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 21
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 17
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 10
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 8
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 7
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 3
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 13
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 2
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 9
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 22
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 12
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 0
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 24
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 19
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 16
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 14
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 18
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 11
22/06/29 01:58:58 INFO spark.ContextCleaner: Cleaned accumulator 20
22/06/29 01:59:00 ERROR yarn.ApplicationMaster: User class threw exception: kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath
kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath
at kotlin.jvm.internal.ClassReference.error(ClassReference.kt:84)
at kotlin.jvm.internal.ClassReference.isData(ClassReference.kt:70)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.isSupportedClass(ApiV1.kt:172)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.generateEncoder(ApiV1.kt:167)
at com.zshp.agg.batch.bannedword.CatalogBannedWordSparkApplication.main(CatalogBannedWordSparkApplication.kt:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684)
22/06/29 01:59:00 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath
at kotlin.jvm.internal.ClassReference.error(ClassReference.kt:84)
at kotlin.jvm.internal.ClassReference.isData(ClassReference.kt:70)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.isSupportedClass(ApiV1.kt:172)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.generateEncoder(ApiV1.kt:167)
at com.zshp.agg.batch.bannedword.CatalogBannedWordSparkApplication.main(CatalogBannedWordSparkApplication.kt:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684)
)
22/06/29 01:59:00 INFO spark.SparkContext: Invoking stop() from shutdown hook
22/06/29 01:59:00 INFO server.AbstractConnector: Stopped Spark@52631839{HTTP/1.1,[http/1.1]}{0.0.0.0:0}
22/06/29 01:59:00 INFO ui.SparkUI: Stopped Spark web UI at <http://lniuhwk3388.nhnjp.ism:38055>
22/06/29 01:59:00 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
22/06/29 01:59:00 INFO cluster.YarnClusterSchedulerBackend: Shutting down all executors
22/06/29 01:59:00 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
22/06/29 01:59:00 INFO cluster.SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
22/06/29 01:59:00 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
22/06/29 01:59:00 INFO memory.MemoryStore: MemoryStore cleared
22/06/29 01:59:00 INFO storage.BlockManager: BlockManager stopped
22/06/29 01:59:00 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
22/06/29 01:59:00 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
22/06/29 01:59:00 INFO spark.SparkContext: Successfully stopped SparkContext
22/06/29 01:59:00 INFO yarn.ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag message: User class threw exception: kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath
at kotlin.jvm.internal.ClassReference.error(ClassReference.kt:84)
at kotlin.jvm.internal.ClassReference.isData(ClassReference.kt:70)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.isSupportedClass(ApiV1.kt:172)
at org.jetbrains.kotlinx.spark.api.ApiV1Kt.generateEncoder(ApiV1.kt:167)
at com.zshp.agg.batch.bannedword.CatalogBannedWordSparkApplication.main(CatalogBannedWordSparkApplication.kt:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684)
)
22/06/29 01:59:00 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered.
22/06/29 01:59:00 INFO yarn.ApplicationMaster: Deleting staging directory <viewfs://iu/user/z_shopping_ai/.sparkStaging/application_1655162175779_1751556>
22/06/29 01:59:00 INFO util.ShutdownHookManager: Shutdown hook called23 Views