:thread: Crash when the screen reader (or the Accessibility Inspector) is enabled

# compose-ios

[faogustavo]

🧵 Crash when the screen reader (or the Accessibility Inspector) is enabled

[faogustavo]

We recently started seeing one crash when users have the screen reader enabled or when the accessibility inspector from Xcode is enabled:

IllegalStateException: Can't merge NativeView semantics property.

I was able to trace it back to a native view (using the UIKitView composable) with accessibility elements (accessibilityLabel). Is it already tracked in any place? I was not able to find it. If not I can try to create a sample project to reproduce the issue

[faogustavo]

Here is the complete stacktrace

stacktrace

[Ivan Matkov]

сс @Elijah Semyonov

[Elijah Semyonov]

Hi, it means that you have multiple interop views inside a LayoutNode that merges its semantics, which is illegal. Interop views are unreachable for AXServices in such case (and they must be). We can relax this requirement(or modify the API) but it would be nice to see an actual scenario that leads to this

[Elijah Semyonov]

For example, you have a clickable element with two interop views inside.

[Elijah Semyonov]

On a second thought, it’s probably way too restrictive, I’ll do a hot fix with a warning instead of a crash if both merged nodes with interop views have ‘isAccessibilityElement’ set to true.

[faogustavo]

Okay, with your insight, I was able to trace it. The problem happens on a UIImageView (used by the UIKitView), inside an IconButton on my toolbar.

But you said that

you have a clickable element with two interop views inside.

We only have one element inside. Shouldn't it work?


BTW, on the UIKitView, I'm setting the interactive = false

[Elijah Semyonov]

It should and seems like a bug, I will investigate it

[Elijah Semyonov]

Interactive is about touches processing, (e.g. plain text is not interactable but visible for accessibility services)

[faogustavo]

Thanks. Please let me know about any updates 🙏

[Elijah Semyonov]

Yeah, it’s a bug, there should be no crash happening there (even based on previous strict policy). I’ll roll out a fix soon.

[Elijah Semyonov]

Actually, your usage case highlighted a much more important problem 🙂
Will include that in a fix too.

[faogustavo]

Thats great, thanks

[Elijah Semyonov]

https://github.com/JetBrains/compose-multiplatform-core/pull/1347
I’ll try to sneak this one into 1.6.10

[Elijah Semyonov]

Be aware, that this fix will doesn’t cover the other issue that you highlighted:
The button containing UIKitView will be inaccessible by accessibility input. UIKitView semantics will taint the Button SemanticsNode and it will redirect the user to the native UIView instead of UIAccessibilityElement proxying the Button semantics.

[Elijah Semyonov]

We don’t support (and don’t plan to) merging all possible accessibility resolution strategies of both Compose semantics trees and native accessibility trees, it’s a combinatorial explosion.
We will provide an argument to UIKitView to opt out the element out of accessibility resolution (so a simple native icon doesn’t override the semantics of the buttons it’s contained in).

[Elijah Semyonov]

But it’s an API change and hence can’t do that in 1.6.10 😞

[Elijah Semyonov]

https://github.com/JetBrains/compose-multiplatform-core/pull/1347 -> crash fix should make it to 1.6.10 (care though, there will be no crash but the button will be inaccessible (will redirect AX services to the interop view inside))
https://github.com/JetBrains/compose-multiplatform-core/pull/1350 -> this will be in next release

[faogustavo]

Awesome, thanks pepeparty

[Elijah Semyonov]

Both merged, for now a workaround would be just not to use interop views inside semantics-merging widgets.