The Art of Koen - Final report and next steps

Introduction

Artist and curator Koen van Santvoort (1993) experienced the darkest day of his life in 2021. While working in construction, he fell from a great height, resulting in a months-long coma and a traumatic brain injury. When he finally woke up, he couldn’t even swallow. That marked the beginning of a long rehabilitation journey, now three years underway.

Rehabilitation is a demanding process, but whenever Koen has a little energy left, he devotes it to his lifelong passion: art. It inspires him and offers a welcome sense of relief. He even managed to organize an exhibition during his rehab. With the help of his brother Bart, former classmate Reinier Vrancken, and many friends, he curated Transpainting, a dynamic group show at Nachtzuster in Tilburg.

The exhibition was a big success, but Koen was only able to curate, it wasn’t physically possible for him to finish old work or create new pieces. Now that he has progressed further, the question arises: is there a way for Koen to actively create again? And if so, how can we accommodate his physical limitations?

Koen believes technology might offer a solution, particularly immersive technology. Immersion means being fully surrounded or absorbed. Immersive tech includes tools that draw users into either fully virtual environments or hybrid spaces where digital and physical realities blend. The most important of these are augmented reality (AR) and virtual reality (VR). In AR, users see digital layers over the physical world. In VR, they step into a fully virtual space.

Immersive tech can create a new kind of “realness”, one that feels tangible, even if it’s partly or fully virtual. It’s much more sensory and up-close than traditional screens.

As with Transpainting, a small group of family and friends is supporting Koen in this ambition, including Roger Kemp from Kemp Works. With a background in immersive technologies, Kemp Works is helping Koen explore whether this dream is possible, and if so, how to make it real.

Concept

Apple Vision Pro combines AR and VR into what Apple calls a “spatial computing” experience. The headset is controlled using eyes, hands, and voice, which holds exciting promise for people with limited mobility. For Koen, the focus was on AR, his goal was to expand a physical studio with digital layers. A hybrid space where he can both create and present work.

Experiment

“The art challenges the technology, and the technology inspires the art.”

—John Lasseter

This quote from our original proposal turned out to be very fitting. Through many sessions with Koen and his Apple Vision Pro, we got to know the limits, not only of the tech, but also of Koen’s current abilities. At the same time, we were inspired by what was possible. Apple Vision Pro proved to be a valuable challenge and opened new creative pathways for Koen.

Koen’s second Apple Vision Pro artwork, made in Freeform.

Sub-experiment 1 – Interaction and control

Getting started with the Apple Vision Pro came with some bumps. Setup and calibration took more time and precision than expected. Adjusting the fit, configuring the eye tracking, and finding a comfortable position all required patience.

It was especially hard to see what Koen was actually seeing. External display connections were unreliable, so much of the setup had to be done by feel. In the early sessions, this made guidance tricky. Only later, as we began to understand how the interface responded to Koen’s eye movements, were we able to support him more effectively.

Koen's first experience with Apple Vision Pro.

But once past the setup phase, Apple Vision Pro turned out to be surprisingly intuitive. Despite his physical limitations, Koen was able to navigate the interface with ease. The eye tracking made a huge difference. Koen had previously tried an eye-controlled computer, but it didn’t work well for him. Apple Vision Pro, however, impressed him deeply. The eye tracking is accurate enough for him to select menus, trigger actions, and open apps, just by looking.

Accessibility features were also impressively thoughtful. Most functions could be controlled entirely with eye movements, without needing hand gestures or extra devices. This is incredibly valuable for someone with motor limitations. Unlike other tools that need to be adapted, Apple Vision Pro seems built with inclusion in mind.

Of course, physical endurance was still a big factor. Koen’s attention span is limited, so we worked in short focused bursts. The headset itself is relatively heavy, and although it fits securely, the weight began to wear on Koen’s neck and head over time, causing fatigue or even headaches. This made it crucial to plan sessions carefully and respect Koen’s limits. It highlights how vital comfort and session length are when designing tech for people with heightened physical vulnerability.

Another Apple Vision Pro piece by Koen, created in Freeform.

Sub-experiment 2 – Creative software

Today’s visionOS apps often still feel like tech demos instead of daily tools, especially for artistic expression. However, Apple’s own Freeform app was a pleasant surprise. Koen grasped how it worked almost instantly and started drawing right away. That was a powerful moment, it had been a long time since he was able to visually create something himself.

Although Freeform is simple, it gave just enough creative freedom for Koen to intuitively explore visual ideas. Alongside our sessions, Koen also enjoyed viewing spatial content, like 3D videos and virtual environments. These experiences felt rich and immersive, though sometimes still unfinished, like the technology was ahead of the ecosystem supporting it. Especially in terms of creative tools, there’s still a gap.

That gap is where Koen sees opportunity. The current lack of apps opens the door for creating new ones, tailored to his way of working and thinking. We’re now exploring how to develop one or more apps specifically designed around Koen’s physical abilities, creative goals, and artistic ambitions. This could not only help him make art again, it allows him to actively shape a new generation of creative technology.

Another Apple Vision Pro piece by Koen.

Research Question Answered

How can a spatial computing device, within a physical environment, improve the ability to independently create and share art for someone who has not been able to do so due to physical limitations?

A Personal Approach

Throughout our sessions with Apple Vision Pro, one question stayed central: what does Koen want to make, and how can this technology support that? The goal is for the tech to serve Koen, not the other way around. We want to avoid a situation where the tool dictates what kind of art is possible. Instead, we’re exploring how Apple Vision Pro can help bring Koen’s existing vision to life.

Therapy and Creativity, Hand in Hand

A key part of this project is the potential impact on Koen’s well-being—not just mentally, but physically. Together with Koen’s physical therapist, we’re exploring how creative tools can also serve as physical exercises. Think of movements that stimulate fine motor skills, or poses that promote strength and coordination. In this way, the creative experience also becomes a form of therapy, a kind of playful healing.

A Custom Virtual Studio

The ultimate goal of the tools we’re developing is to enable Koen to independently make art. Over time, it’s become increasingly clear how Koen envisions his virtual studio: a place where physical and digital layers merge. Apple Vision Pro allows him to add virtual dimensions to physical works, creating hybrid pieces where tangible and digital elements come together.

From Studio to Exhibition

Apple Vision Pro won’t just help with creating, it will also play a role in presenting Koen’s work. The digital layers Koen builds will become part of a future exhibition. Visitors will wear Apple Vision Pro to experience these digital elements alongside the physical art. This creates a direct connection between creator and audience, through technology that doesn’t restrict, but enhances

Another Apple Vision Pro piece by Koen.

Next Steps

The experiment with Koen and Apple Vision Pro has laid a promising foundation. In the coming period, we’ll continue on five tracks:

Documenting movements with the physical therapist

We're working with Koen’s therapist to record simple motions that fit his rehabilitation goals. These will form the building blocks for creative interactions that are also physically beneficial.

Building prototypes with visionOS developers

We’re teaming up with a small group of developers to build initial prototypes. These will be tested in short sessions with Koen to quickly find what works and where adjustments are needed.

Shaping Koen’s artistic vision

Together with Koen, we’re mapping out what he wants to create and what artistic autonomy means to him. These insights guide the design of the tools.

Towards a cohesive toolset and exhibition

Eventually, we’ll bring together the exercises, prototypes, and Koen’s ideas into a custom toolset. This will be the foundation for his virtual studio and an upcoming public exhibition.

Sharing progress through Koen’s website

To keep others involved, we’ll regularly share updates on Koen’s website. This way, his network stays connected, and other creators, care professionals, and tech partners can find inspiration in his journey.

Want to join our team?

So you read our report all the way through! Thank you! So would you be interested in helping Koen out? 

Are you a visionOS developer and interested in making a real impact? Then join us in creating small prototypes for Koen. We're looking for people who know visionOS and would like to donate their expertise and some of their time to help us create Koen's virtual studio and expo. Email me or find me on Blue Sky and lets have a chat to see how we could help each other.