GTXiLib SwiftUI support

[Mary Baptista Martinez]

Hi team,

I've noticed that GTXiLib picks up errors in SwiftUI views (yay!) but it's difficult to discern what view is impacted. Using the view's frame can sometimes be helpful but I'm wondering if there is a way to get more detail?

Here is an example output, where the offending view is always shown as SwiftUI.AccessibilityNode. I've tried searching but it looks like this might be an Apple private API.

2023-08-31 10:44:49.743071-0700 a11y[55045:680500] [GTX_LOG Error] Evaluating <_TtGC7SwiftUI14_UIHostingViewGVS_15ModifiedContentVS_7AnyViewVS_12RootModifier__:0x138d06660 isAXElement=NO AXFrame={{0, 0}, {393, 852}}>

2023-08-31 10:44:49.743366-0700 a11y[55045:680500] [GTX_LOG Error] Skipped <_TtGC7SwiftUI14_UIHostingViewGVS_15ModifiedContentVS_7AnyViewVS_12RootModifier__:0x138d06660 isAXElement=NO AXFrame={{0, 0}, {393, 852}}>

2023-08-31 10:44:49.743837-0700 a11y[55045:680500] [GTX_LOG Error] Evaluating <SwiftUI.AccessibilityNode:0x600003a50870 isAXElement=YES AXFrame={{185.66666666666666, 231}, {21.666666666666657, 21.666666666666657}}>

2023-08-31 10:44:49.744445-0700 a11y[55045:680500] [GTX_LOG Error] Evaluating <SwiftUI.AccessibilityNode:0x600003a50900 isAXElement=YES AXFrame={{149.16666666666669, 254.51212565104166}, {94.666666666666657, 20.333333333333343}}>

2023-08-31 10:44:49.749708-0700 a11y[55045:680500] [GTX_LOG Error] Evaluating <SwiftUI.AccessibilityNode:0x600003a50990 isAXElement=YES AXFrame={{16, 287}, {361, 361}}>

Has your team encountered this?

Thanks,

Mary

[Cooper Knaak]

Hi Mary,

This is an unfortunate gap in GTXiLib (really in all testing tools). It's not possible to evaluate actual SwiftUI content, because it doesn't use the Obj-C runtime. However, instead of being displayed directly,  SwiftUI generates UIKit constructs like you posted above. Hence, GTXiLib can evaluate the output of SwiftUI. SwiftUI.AccessibilityNode is probably a UIAccessibilityElement - a virtual accessibility element that isn't an actual UIView or SwiftUI.View that is used to represent functionality to screenreaders and other accessibility features.

You might be able to get more information by adding an accessibility identifier to your views. That should be printed out alongside isAXElement an AXFrame. You can also call GTXLogger.defaultLogger.setLogLevel(GTXLogLevelDeveloper) (or just .developer, depending on how Swift translates the Obj-C). That will additionally print the accessibility label for each accessibility node (accessibility labels might contain PII, since they are usually the textual content of on screen elements, which is why they are not printed by default, to avoid persisting them to, say, logs).

We'd like to explore actual SwiftUI support, but we don't have anything concrete. If and when GTXiLib supports native SwiftUI evaluations, we will announce it on GitHub and in this group.

Sincerely,

Cooper Knaak