Finally, we had the first real Vision Pro session in Koen’s studio, and to everybody’s surprise, things worked a lot better than expected!
To learn more about my project with Koen, please read our earlier blog post.
Setup and calibration
Setup and calibration have become a lot more straightforward in visionOS 2.0 compared to our first session earlier this year.
We hooked up the Vision Pro with Koen’s Apple ID and mirrored it to his iPhone so we could see what Koen sees. It’s great that Vision Pro remembers it’s mirrored, even after handing off the device to another user.
We gave the device to Koen to calibrate his eyes and hands. To everybody’s surprise, this went a lot better than during the initial test session a while back. I think the reason for this is twofold: visionOS has gotten a lot better at tracking hands and fingers, and Koen has gotten way better at controlling his hands.
Small movements
After calibration, it became clear that Koen has gotten a lot better at controlling visionOS. The beauty of the OS is that it only requires minute movements, while holding one’s hand in one’s lap, to make big gestures inside visionOS. This makes it easy for Koen to control a pencil in Freeform, for instance, which was the first app he opened. He immediately started drawing with the different drawing options. It was awesome to see how easy it was for him to pick it up and run with it.
One could argue that those small but precise movements are also a form of physical therapy for Koen, training the motor skills in his hand while at the same time improving his hand-eye coordination.
What's next?
Koen will keep using and testing Vision Pro with help from friends and family, and together we will document what works and what doesn’t for him.
Parallel to this, we will work out his creative process for the art he wants to create and figure out how Vision Pro can support him in that endeavor.
We’ll also test several controllers to see if they can help Koen. Although we’ve already noticed that the out-of-the-box eye and hand tracking works like a charm for smaller movements, larger gestures like scrolling and zooming might benefit from a dedicated controller stick that works with his right hand.
