For those interested, there's a new implementation...

# arrow

[Alejandro Serrano.Mena]

For those interested, there's a new implementation of the Optics plugin as a compiler plugin (instead of a KSP plugin). The setup instructions are very easy, just add id("io.arrow-kt.optics") version "2.3.0-alpha.1" , and remove the KSP plugin, and you should be good to go! As a bonus, the optics are generated directly in the companion object, instead of as extensions thereof.

If you also want to see the generated optics in IntelliJ, go to Help > Edit custom properties, and add kotlin.k2.only.bundled.compiler.plugins.enabled=false and restart. You should get something similar to the attached screenshot. It's a known issue that @optics.copy sometimes crashes the IDE, but @optics only should be fine

Please report any problems, or any feedback you may have

[sindrenm]

I am so hyped for this! 🎉🎉🎉

[sindrenm]

We're not supposed to call arrowOptics() inside the kotlin {} block in our build scripts anymore? The arrow.optics.plugin.arrowOptics no longer exists for me.

[sindrenm]

I'm getting the following error when building in our production app, but I'm having issues creating a separate sample app:

Somewhere in file /home/sindre/code/sats-group/member-app-android/core/domain/src/main/kotlin/com/sats/core/domain/activities/models/Abc.kt: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Unexpected returnTypeRef. Expected is FirResolvedTypeRef, but was FirUserTypeRefImpl
	at org.jetbrains.kotlin.util.AnalysisExceptionsKt.wrapIntoFileAnalysisExceptionIfNeeded(AnalysisExceptions.kt:62)
	at org.jetbrains.kotlin.fir.FirCliExceptionHandler.handleExceptionOnFileAnalysis(Utils.kt:270)
	at org.jetbrains.kotlin.fir.resolve.transformers.plugin.FirCompanionGenerationTransformer.transformFile(FirCompanionGenerationProcessor.kt:113)
	at org.jetbrains.kotlin.fir.resolve.transformers.plugin.FirCompanionGenerationTransformer.transformFile(FirCompanionGenerationProcessor.kt:39)
	at org.jetbrains.kotlin.fir.declarations.FirFile.transform(FirFile.kt:47)
	at org.jetbrains.kotlin.fir.resolve.transformers.FirTransformerBasedResolveProcessor.processFile(FirResolveProcessor.kt:53)
	at org.jetbrains.kotlin.fir.resolve.transformers.FirTotalResolveProcessor.process(FirTotalResolveProcessor.kt:39)
	at org.jetbrains.kotlin.fir.pipeline.AnalyseKt.runResolution(analyse.kt:25)

Caused by: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Unexpected returnTypeRef. Expected is FirResolvedTypeRef, but was FirUserTypeRefImpl
	at org.jetbrains.kotlin.fir.symbols.impl.UtilsKt.errorInLazyResolve(Utils.kt:42)
	at org.jetbrains.kotlin.fir.symbols.impl.FirCallableSymbol.calculateReturnType(FirCallableSymbol.kt:48)
	at org.jetbrains.kotlin.fir.symbols.impl.FirCallableSymbol.getResolvedReturnTypeRef(FirCallableSymbol.kt:40)
	at org.jetbrains.kotlin.fir.symbols.impl.FirCallableSymbol.getResolvedReturnType(FirCallableSymbol.kt:53)
	at arrow.optics.plugin.fir.FirOpticsExtractor.constructorFoci(FirOpticsExtractor.kt:232)
	at arrow.optics.plugin.fir.FirOpticsExtractor.foci(FirOpticsExtractor.kt:76)
	at arrow.optics.plugin.fir.OpticsDslGenerator.dslFoci(OpticsDslGenerator.kt:60)
	at arrow.optics.plugin.fir.OpticsDslGenerator.getTopLevelCallableIds(OpticsDslGenerator.kt:67)

Abc.kt was created by me now to try and isolate the issue. It only contains this enum class:

package com.sats.core.domain.activities.models

enum class Abc {
    A, B, C
}

It's not even being referenced anywhere. The same issue happens with both 2.3.0-alpha.1 and 2.3.0-alpha.2. Want me to file an issue on GH, even if I'm not able to recreate the issue in a repro project (yet)?

[sindrenm]

Just to be clear, the issue also happens without Abc, it just happens on another enum class we have called ActivityTagType, that's also just a simple enum class, but that's being referenced somewhere else. I created Abc because I suspected it happened to the first enum it parsed in alphabetical order.

[sindrenm]

If I change enum class Abc to a sealed interface, then Abc is fine, but the error moves forward to the ActivityTagType.kt , which is alphabetically next in the list of files in the project.

[Alejandro Serrano.Mena]

mmm, this should not happen with 2.3.0-alpha.2... I'll try to reproduce and release a new artifact

[Alejandro Serrano.Mena]

And no, no more arrowOptics() in the build fle

[sindrenm]

Thanks!

[Alejandro Serrano.Mena]

@sindrenm could you try to reproduce it with a new project, or after cleaning the project where you get the error? there should be no call to calculateReturnType from getTopCallableIds (this was exactly the change from alpha.1 to alpha.2) 😕

[sindrenm]

I was not able to reproduce in a new project. When I get back to work tomorrow, I will try cleaning the problematic one. However, alpha.2 was the first one I tried (before alpha.1). 🤔

[sindrenm]

Alright, cleared everything, and the call to calculateReturnType() is gone from the stack trace, but the (seemingly) same error persists:

org.jetbrains.kotlin.util.FileAnalysisException: Somewhere in file /home/sindre/code/sats-group/member-app-android/core/domain/src/main/kotlin/com/sats/core/domain/activities/models/ActivityTagType.kt: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Unexpected status. Expected is FirResolvedDeclarationStatus, but was FirDeclarationStatusImpl
	at org.jetbrains.kotlin.util.AnalysisExceptionsKt.wrapIntoFileAnalysisExceptionIfNeeded(AnalysisExceptions.kt:62)
	at org.jetbrains.kotlin.fir.FirCliExceptionHandler.handleExceptionOnFileAnalysis(Utils.kt:270)
	at org.jetbrains.kotlin.fir.resolve.transformers.plugin.FirCompanionGenerationTransformer.transformFile(FirCompanionGenerationProcessor.kt:113)
	at org.jetbrains.kotlin.fir.resolve.transformers.plugin.FirCompanionGenerationTransformer.transformFile(FirCompanionGenerationProcessor.kt:39)
	at org.jetbrains.kotlin.fir.declarations.FirFile.transform(FirFile.kt:47)
	at org.jetbrains.kotlin.fir.resolve.transformers.FirTransformerBasedResolveProcessor.processFile(FirResolveProcessor.kt:53)
	at org.jetbrains.kotlin.fir.resolve.transformers.FirTotalResolveProcessor.process(FirTotalResolveProcessor.kt:39)
	at org.jetbrains.kotlin.fir.pipeline.AnalyseKt.runResolution(analyse.kt:25)
	at org.jetbrains.kotlin.fir.pipeline.FirUtilsKt.resolveAndCheckFir(firUtils.kt:78)
	at org.jetbrains.kotlin.cli.pipeline.jvm.JvmFrontendPipelinePhase.executePhase(JvmFrontendPipelinePhase.kt:168)
	at org.jetbrains.kotlin.cli.pipeline.jvm.JvmFrontendPipelinePhase.executePhase(JvmFrontendPipelinePhase.kt:51)
	at org.jetbrains.kotlin.cli.pipeline.PipelinePhase.phaseBody(PipelinePhase.kt:63)
	at org.jetbrains.kotlin.cli.pipeline.PipelinePhase.phaseBody(PipelinePhase.kt:53)
	at org.jetbrains.kotlin.config.phaser.NamedCompilerPhase.invoke(CompilerPhase.kt:102)
	at org.jetbrains.kotlin.backend.common.phaser.CompositePhase.invoke(PhaseBuilders.kt:22)
	at org.jetbrains.kotlin.config.phaser.CompilerPhaseKt.invokeToplevel(CompilerPhase.kt:53)
	at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.runPhasedPipeline(AbstractCliPipeline.kt:135)
	at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.executeAndReturnPipeLineArtifact(AbstractCliPipeline.kt:89)
	at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.executeAndReturnPipeLineArtifact$default(AbstractCliPipeline.kt:49)
	at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.execute(AbstractCliPipeline.kt:40)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecutePhased(K2JVMCompiler.kt:54)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecutePhased(K2JVMCompiler.kt:45)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:94)
	at org.jetbrains.kotlin.cli.common.CLICompiler.exec(CLICompiler.kt:366)
	at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunnerBase.runCompiler(IncrementalJvmCompilerRunnerBase.kt:211)
	at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunnerBase.runCompiler(IncrementalJvmCompilerRunnerBase.kt:53)
	at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.doCompile(IncrementalCompilerRunner.kt:539)
	at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileImpl(IncrementalCompilerRunner.kt:451)
	at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileNonIncrementally(IncrementalCompilerRunner.kt:327)
	at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:139)
	at org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:782)
	at org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:108)
	at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:2046)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Unexpected status. Expected is FirResolvedDeclarationStatus, but was FirDeclarationStatusImpl
	at org.jetbrains.kotlin.fir.symbols.impl.UtilsKt.errorInLazyResolve(Utils.kt:42)
	at org.jetbrains.kotlin.fir.symbols.impl.UtilsKt.resolvedStatus(Utils.kt:30)
	at org.jetbrains.kotlin.fir.symbols.impl.FirCallableSymbol.getResolvedStatus(FirCallableSymbol.kt:68)
	at arrow.optics.plugin.fir.FirOpticsExtractor.sealedLensFoci$lambda$0(FirOpticsExtractor.kt:352)
	at org.jetbrains.kotlin.fir.scopes.impl.FirClassDeclaredMemberScopeImpl.processPropertiesByName(FirClassDeclaredMemberScope.kt:146)
	at org.jetbrains.kotlin.fir.scopes.FirContainingNamesAwareScopeKt.processAllCallables(FirContainingNamesAwareScope.kt:39)
	at org.jetbrains.kotlin.fir.declarations.DeclarationUtilsKt.processAllDeclarations(declarationUtils.kt:138)
	at org.jetbrains.kotlin.fir.declarations.DeclarationUtilsKt.processAllDeclarations(declarationUtils.kt:163)

[Alejandro Serrano.Mena]

ok, but now it's moved to sealedLensFoci, so it seems I need to do some extra work there

[sindrenm]

Unfortunately, I'm not able to reproduce this in a new project, as I'm not entirely sure what causes it. Want me to raise an issue over on GH anyway?

[Alejandro Serrano.Mena]

github.com/arrow-kt/arrow/pull/3947

[sindrenm]

Awesome, thanks! 🙌

[sindrenm]

Are there snapshot builds I can try out, or should I wait for alpha.3 before testing?

[Alejandro Serrano.Mena]

the alpha.3 is currently being published

[sindrenm]

Much appreciated! 🤩

[sindrenm]

Tested alpha.3 now, and the previous issue seems to be gone. 🙌

[sindrenm]

However, I think I've found another bug: If companion objects on sealed interfaces are declared manually in code (and not generated by the compiler plugin), some things seem to not be generated correctly. This works:

@optics
sealed interface MySealedInterface {
  @optics
  data object MyObject : MySealedInterface

  @optics
  data class MyClass(val myString: String) : MySealedInterface
}

fun main() {
  val source: MySealedInterface = MySealedInterface.MyClass("Hello")

  MySealedInterface.myClass.myString.set(source, "World")
}

However, this fails to compile:

@optics
sealed interface MySealedInterface {
  @optics
  data object MyObject : MySealedInterface

  @optics
  data class MyClass(val myString: String) : MySealedInterface

  companion object // added this, compilation fails
}

fun main() {
  val source: MySealedInterface = MySealedInterface.MyClass("Hello")

  MySealedInterface.myClass.myString.set(source, "World")
}

Error message:

e: file:///path/to/optics-test.kt:18:36 Cannot infer type for type parameter '__S'. Specify it explicitly.

e: file:///path/to/optics-test.kt:18:36 Unresolved reference. None of the following candidates is applicable because of a receiver type mismatch:
val <__S> PPrism<__S, __S, MySealedInterface, MySealedInterface>.myClass: PPrism<__S, __S, MySealedInterface.MyClass, MySealedInterface.MyClass>

e: file:///path/to/optics-test.kt:18:44 Cannot infer type for type parameter '__S'. Specify it explicitly.

e: file:///path/to/optics-test.kt:39:21 Cannot infer type for type parameter '__S'. Specify it explicitly.

e: file:///path/to/optics-test.kt:39:21 Unresolved reference. None of the following candidates is applicable because of a receiver type mismatch:
val <__S> PPrism<__S, __S, MySealedInterface, MySealedInterface>.myClass: PPrism<__S, __S, MySealedInterface.MyClass, MySealedInterface.MyClass>

e: file:///path/to/optics-test.kt:39:29 Cannot infer type for type parameter '__S'. Specify it explicitly.

[sindrenm]

For most of our cases where this is happening, it's fine to just remove the explicit companion object and rely on the generated one. However, some of our optics-annotated classes are also @Serializable from kotlinx-serialization, and the two plugins seem to collide in that case:

@optics
@Serializable
sealed interface MySealedInterface {
  @optics
  data object MyObject : MySealedInterface

  @optics
  data class MyClass(val myString: String) : MySealedInterface
}

java.lang.IllegalStateException: Multiple plugins generated nested class with same name Companion for class com/sindrenm/templates/project/MySealedInterface:
public final companion object Companion : R|kotlin/Any| {
}

public final companion object Companion : R|kotlin/Any| {
}

[sindrenm]

It also looks like generics are disabled on purpose for now?

/** Monomorphic `@optics`-annotated source classes for which the DSL target is enabled. */
private fun annotatedSources(): List<FirRegularClassSymbol> = session.predicateBasedProvider.getSymbolsByPredicate(lookupPredicate)
  .filterIsInstance<FirRegularClassSymbol>()
  .filter { it.typeParameterSymbols.isEmpty() && FirOpticsExtractor.dslEnabled(it) }

(github.com/arrow-kt/arrow/blob/…/OpticsDslGenerator.kt#…)

Is that due to a compiler plugin limitation? We have the following:

@optics
sealed interface UiState<out Data> {
    data object Loading : UiState<Nothing>

    @optics
    data class Content<out Data>(val data: Data) : UiState<Data> {
        companion object
    }

    @optics
    data class Error(val message: RetryableErrorMessage) : UiState<Nothing> {
        companion object
    }
}

And previously we could do UiState.content<Foo>.data()..., but that is no longer generated, it seems. We can create that one on our own for now, though. 👍

[Alejandro Serrano.Mena]

@sindrenm the new alpha.4 fixes both problems

(for serialization you need to write the companion object by hand -- a limitation of compiler plugins -- but I've added a error message when we detect that @Serializable is being used)

[sindrenm]

I'm okay with writing the companion object by hand as long as that works. Will try alpha.4 very soon. Thanks for the insanely quick response time and fixes, btw!

[sindrenm]

alpha.4 seems to fix all of the above issues, and AFAICT, we only have one remaining one in our code base.

import arrow.optics.optics

@optics
data class ScreenContent(
    val state: ScreenState = ScreenState.Loading,
)

@optics
sealed interface ScreenState {
    @optics
    data class Content(val text: String? = null) : ScreenState

    data object Loading : ScreenState
}

// KSP generated this extension, but the compiler plugin does not:
//val <S> Optional<S, ScreenState>.content: Optional<S, ScreenState.Content>
//    get() = this + ScreenState.content

fun setText(state: ScreenContent, text: String?): ScreenContent {
    return ScreenContent.state
        .content // compilation error here without the manually created extension above
        .text.set(state, text)
}

Here are the errors:

e: file:///path/to/OptionalExtensions.kt:22:10 Unresolved reference 'content'.
e: file:///path/to/OptionalExtensions.kt:23:10 Cannot infer type for type parameter '__S'. Specify it explicitly.

[Alejandro Serrano.Mena]

that's strange, it works for me both in the tests, and also when using it in another project 😕 (and I see it in the generated bytecode)

[sindrenm]

Really? Let me try with a fresh project. 🤔

[sindrenm]

Hmmm, yeah, you're right, it does work in a fresh project. I'll investigate further. 🤔

[sindrenm]

So, after quite the journey, I've managed to pinpoint the issue. It only happens if you (1) enable the Kotlin Serialization plugin, and (2) have any enum class anywhere in your compilation. Neither have to be explicitly used, but adding both triggers the issue.

import arrow.optics.optics
import kotlinx.serialization.Serializable
import kotlin.collections.set

// Enabling Serialization and adding this will trigger the error.
enum class ImAnEnum {
    Foo, Bar, Baz
}

@optics
@Serializable
data class ScreenContent(
    val state: ScreenState = ScreenState.Loading,
) {
    companion object
}

@optics
sealed interface ScreenState {
    @optics
    data class Content(val text: String? = null) : ScreenState

    data object Loading : ScreenState
}

// KSP generated this extension, but the compiler plugin does not:
//val <S> Optional<S, ScreenState>.content: Optional<S, ScreenState.Content>
//    get() = this + ScreenState.content

fun main() {
    val content = ScreenContent(state = ScreenState.Content())

    println(content)

    val updated = setText(content, "hello")

    println(updated)
}

fun setText(state: ScreenContent, text: String?): ScreenContent {
    return ScreenContent.state
        .content // compilation error here without the manually created extension above
        .text.set(state, text)
}

[sindrenm]

I published a repro over at github.com/sindrenm/optics-test.

[sindrenm]

Also managed to make it fail with Metro and a dependency graph: github.com/sindrenm/optics-test/…/metro

[Alejandro Serrano.Mena]

Unfortunately it seems that something weird happens when you have several compiler plugins. I've made an attempt at github.com/arrow-kt/arrow/pull/3958, but I randomly get unresolved supertypes (which is the underlying reason why the DSL disappears -- the compiler is not able to know whether something is a supertype correctly)

[sindrenm]

I wish I could help out, but I know nada about building compiler plugins.

[sindrenm]

That said, I'll happily try out any new alpha to see if it works specifically in our case. Thanks again!