At Sundance New Frontier 2015, Felix and Paul presented three VR films to great acclaim: Strangers with Patrick Watson, Herders, and Wild.

Beyza Boyacioglu of the Open Documentary Lab interviewed Felix and Paul about their work, 360-degree filmmaking and the language of the VR medium.

Interview with FELIX

Beyza: Can you talk a little about your background and how you came to be working with virtual reality?

Felix: I’m originally a director and a director of photography, and I spent lots of time working in the Canadian arctic, the Inuit, doing fiction and documentary projects. I discovered a very strong interest in the relationship between space, culture, authentic human beings, and media. So I decided that I would focus on that for the rest of my life. I just didn’t know that I would venture out of film making.

I met Paul and we started doing traditional work, making a living as commercial and music video directors, and we gradually felt kind of bored about the cinema screen, about the cinematic language. We felt the need to create something that was not about watching a story or being sort of a passive viewer, but rather breaking that paradigm and creating experiences of presence. We wanted to bring the viewer inside of an actual experience where things don’t come through your eyes, through the cognitive. It would be a form in which experiences just reach your senses and this would somehow give you the raw emotions. We didn’t think that cinema could provide that. So we started doing video installations, large scale projection mapping projects, multimedia environments … We were exploring different ways to articulate presence in media.

We experimented with 3D cinema and holography, but once we heard about Oculus releasing the DK1 virtual reality headset, we completely flipped out and said, “This is exactly what we’ve been trying to do all of those years using different tools.” We created a technology to be able to do reality capture, basically to film in 360 degrees, and a system that allows us to record 360 degree possibilities of sound from one point of view. We started using that technology to explore virtual reality as a storytelling and experimental medium.

Beyza: Have you seen Godard’s 3D film? It seems like he is doing exactly the opposite of what you’re doing by manipulating and distorting the 3D technology on purpose. The film is called Goodbye to Language, he is breaking the 3D language to make his point.

Felix: Yes, It’s a totally different approach. Godard is focused on the language of filmmaking, and the craft is the subject in a way. The manipulation, the storytelling modes, the formal choice are all statements that he does. What we’re trying to do is erase the sense of form, erase the sense of manipulation. We’re trying to get rid of anything that feels like manipulation or that feels like a proper articulation of a language. We’re trying to make the experiences just feel like sort of a natural flow of reality.

We just want people to experience things. We’re trying to remove the sense of the conceptual as much as possible, and it doesn’t mean that it’s not a conceptual piece, because it requires a lot of thinking to get to non-conceptual, but it is sort of the constant preoccupation that people don’t get caught away in intellectual … sort of trying to understand the pieces from an intellectual standpoint.

We don’t want that, basically. We want them to emotionally surrender on an experiential level. Just let go and surrender to the moment or to the experience. We think that by doing that, it stays in you. The experience comes in without any sense of resistance, in a way, and it stays somehow. It becomes something that is kind of part of you. We wish we could achieve that. Maybe we will learn to go further in that process over the years.

Beyza: This was the first time I’ve used Oculus Rift, and I think it came very natural to me. I wasn’t thrown off by this new medium. It felt really comfortable. I think it has a lot to do with what you’re saying. Your pieces are very observational and…

Felix: … and calm.

Beyza: Yes. What do you think the emotional impact of these pieces on the audience is?

Felix: Surprisingly enough, I think that the register of emotional impact that I’m referring to is in a zone of empathy and intimacy. So for example, The Strangers with Patrick Watson, I’ve showed it to maybe five thousand people over the last years, and I very often hear people saying things like, “I felt guilty turning my gaze away from the artist. It felt kind of bad to do that. I didn’t want to do this because I was afraid to offend him, and I was self aware that this was a virtual reality experience and I was self aware that this was kind of ridiculous or silly because he’s not really there, but still, my mind was telling me to keep looking at the guy, and when I experienced turning around, I really did feel guilty.”

It’s not purposefully designed to create that sense of guilt, but I’m just taking that as an example of how we can identify with characters and human beings in virtual reality. It is very powerful and very different from cinema.

For example, take Wild. You watched the film and the emotional journey you have is by identification to the characters of Reese Witherspoon, Laura Dern and others. In VR, the notion of identification is very different. The actress comes and she sits next to you and she looks at you, and you feel engaged, and in a more kind of direct and visceral way, you can actually, in a way, feel that person’s presence before she even starts talking, before there’s even an attempt at storytelling. Or narrative construction. The person sits next to you and you feel the presence of that person and you react emotionally to that.

There’s a sense of empathy, a sense of intimacy that, in my mind, was not possible with filmmaking in cinema, or it was possible only if you were 66 percent into the film and you’re very much engaged into the story and very much engaged into the acting, and the craft of the film is perfect, then you might start feeling these things. In VR, you feel them the moment you sit, in a way. For me, that’s a very interesting emotional landscape to explore.

Beyza: I think in a conventional film, you put yourself in the shoes of the character, but here, because you are you in the film, you experience it with the characters, next to them. I thought an important difference between Wild, Strangers and Herders is that there was no eye contact in Herders. You feel a little more comfortable looking away from them. In the other two, it feels like those characters are performing to you.

Felix: Yes, exactly. What you said about the main difference being the notion of the viewer’s presence, the notion that you are there instead of being sort of a distraction looking at a screen, that is very important to shaping any form of contact with VR interview. When we did the Mongolian piece, we didn’t want any eye contact. We didn’t want that. We didn’t want the viewer to feel like an intruder. We wanted the viewer to just observe and contemplate. We thought there is virtue in that.

There is virtue in just letting go and just taking it in without any sense of direct emotional or psychological challenge. Someone looking at you is pretty extraordinary. Pretty powerful, but you don’t want to use it all the time. It needs to be used for a meaning in the emotional journey of the viewer. So we did that in Wild and we did that in Strangers, but we didn’t want to do that in Herders. It was a conscious choice.

Beyza: Will you keep on working on similar contemplative, observational pieces?

Felix: We’re not so much interested in the spectacle aspect of virtual reality, the hardcore adrenaline register of emotions. Scaring people or freaking them out, this is probably interesting but it’s not our cup of tea. I think we’re focusing on building a sense of a narrative language gradually, instead of forcing it or trying to retrofit existing ways into VR and realizing it doesn’t work. Each experience is a way for us to explore a new aspect of viewers’ engagement in virtual reality.

The truth is, we don’t really know if it’s going to work or not. It’s a matter of gut feeling in this one, because you can’t really turn and say, “Hey, man…” It’s been 20 years of that kind of practice, and I’ve seen twenty five hundred pieces like that, and I know that this works. At this point, it’s such a new medium that you need to commit to experimentation to a certain extent. Otherwise, you are not progressing.

Interview with PAUL

Beyza: Can you talk a bit about the technology behind your work? I know that you use your own gear and software but I could not find any information about them, not even what your camera looks like.

Paul: Felix and I have always been developing technology in sort of parallel with our work. Usually, we created content that was a mixture of technology and art. I think I’ve always had a hard time with traditional media because I feel like there’s so much influence from all the work that’s already been done. I know there are sort of two schools of thought on that one. You get inspiration from all that work and the other is, you feel kind of in a web of preexisting content. So, my trick since I was a kid was always to create the kind of things that were offshoots of mediums or different things. For instance, we did a lot of multimedia installations. When we were doing 3D cinema we also built our own rigs. It was at a time where I think Cameron was doing Avatar and the only way to shoot in 3D was with a 2 million dollar rig and software only he had. We started adapting our 3D hardware and software for virtual reality the second we saw the first Oculus Rift. We immediately realized that’s what we were trying to do with 3D and, in a way, that was kind of what 3D was trying to be but couldn’t emancipate itself from the screen the way VR has.

Beyza: Do you have a design philosophy in your projects?

Paul: The philosophy so far has been, ‘really harvest that feeling of presence.’ That the camera is not a camera, it’s a viewer, it’s the person trying on virtual reality experience. Cinematic principles and conventions do not apply. If they do, then they need to be heavily adapted to the medium.

Beyza: Are there any cinematic influences in your VR work, any films you looked at?

Paul: I think there’s a lot of Asian cinema, a lot of European cinema and on the fiction side, has been a big influence. I think it’s hard to have a really huge influence on cinema with this field. Last year, was getting more and more into VR and coming from a film background, I was trying to deconstruct all that cinematic language in my head. I had a very hard time watching cinema because even though the medium hasn’t changed, I’m hyper-aware of all the codes and the things that are happening. This happened to me once when I started film school but at a much more modest degree and now, it’s like I’m lucky if I can get through a film at all.

It kind of breaks my heart but I find it harder and harder to get inspiration from cinema in VR. However, I find myself getting more inspiration from real moments in my life or a dream I might have had or an immersive theater or going to a show. A performance of some kind.

Beyza: What do you think the language of this medium is?

Paul: It’s a lot slower. It’s a lot closer to how we live life and in the real world, our life does not cut every three seconds. So, it doesn’t mean there’s not a form of the art that will allow you to get to that speed of- that intensity. There probably will but it’s going to need to evolve. I think that projects that go too far off in doing these things right now, it’s like you disconnect from the core. You lose that sense of presence. You lose that sense of feeling like you actually have freedom. One of the beauties of virtual reality is the sense of, ‘yes, there’s an artist showing me something’ but I also have a role in this and that role feeds into the presence, which feeds into the emotional response that you can have. It’s like the dynamic range of what you’re looking at is at a different scale.

In VR, you could have someone just sitting next to you and looking at you and you feel something. Whereas in cinema, you can’t do that for too long. You could perhaps do that more in photography than cinema. You could have a beautiful picture of just a portrait and that could be something you keep on your wall for years. Cinema is like a larger magnifying glass, which means you can extract more out of subtly but it also means that higher intensity hits harder.

You could do a lot less in VR and get that same feeling. So, the range you’re working in is different. If you want to elicit a certain emotion or reaction in the viewer, you need to be operating at a different register.

Beyza: When you place the camera in a scene, you, the filmmakers, can’t be around it, right?

Paul: You can but then you have to erase yourself out. So, sometimes we’re not in the scene. Sometimes we are but we hide ourselves. It depends on the concept or the shot. For example, in the exterior shots, we hid behind the hills or behind trees.

Going back to how we shoot the scenes. We actually place the camera in the position where they usually seat their guests. It’s also true with Patrick Watson when we shot in his studio. We asked him, ‘where does your friend or your wife sit when it’s just you and him or her and you’re playing music?’ And put the camera exactly at the position on the bench where they would be and there is this quality that emanates from the people when you do that with a camera.

Beyza: How about the actors or subjects? How does the experience of shooting with VR cameras effect them?

Paul: It’s about making them really feel like this camera is a person. It’s in the scene, whether you address that person or not. It’s just that there’s someone there. It’s like the way you act when you’re alone versus the way you act when someone’s there. The direction might be to ignore that person completely but so far, everything we’ve done has been more about, ‘no, there is someone there, acknowledge that presence.’ I think the reason we’re leaning towards that, for now at least, is because it’s the first time we can do it.

Right? Even in film, if you look at the camera, there are a lot of great moments in cinema and less good ones where the character looks at the lens and he’s supposed to look at the viewer but it doesn’t feel anything like if that happens in VR. Even if when you’re not looking directly at the viewer, just behaving as if there was someone there makes you feel so much more present. I don’t know if it’s going to end up being the thing that, at the beginnings of VR, everyone was just looking for. I don’t know and eventually it’s going to dissolve but I feel like it’s the first time you can do that and it’s such a strong thing and, to me at least, for now, it’s a make or break.

If you get that feeling of you really feel present in a scene because of how you’ve been positioned in the scene, because of what’s happening, because of how the subjects are relating to you, even if they’re not relating directly, that is a big factor. So, we always make that a part of the direction.

Beyza: I think in Herders, the characters ignored the camera the most.

Paul: They do. Herders could have almost been called ‘Strangers’ because we were strangers in this family. They don’t speak the same language and we had a translator to speak to them. The moments we spent with them, having soup, eating or drinking tea, we couldn’t talk but we were kind of in each other’s presence. They’re mostly talking to each other. They’re a little ill-at-ease and as you’re not part of their culture, family or someone who spends time with them everyday, that’s how it would feel.

It’s kind of a coincidence because an actual guest might get that experience. A weird looking VR camera would also elicit that same kind of reaction.

Beyza: Yes, that makes sense. Although did you tell them not to look at the camera?

Paul: We did because that’s also sort of a gut instinct. There’s for example, you might not have seen it but, in Strangers, Patrick looks at you for one moment. You might miss it.

Beyza: Yes, I saw that.

Paul: There’s another piece we did for the introduction to VR for Samsung. There’s a Cirque de Soleil piece and there are two acrobats performing, you’re on the stage while the other performers are sitting next to you and they’re looking at you. It works in that case because its kind of over the top and it worked for that scene but in ‘Herders,’ it felt like we’d be going just past the threshold of putting the viewer too much on the spot.

Patrick does it once and that’s enough. We had told him to look very rarely and in the end, you see it and you’re like, ‘it doesn’t feel right.’

Beyza: How about Wild then?

Paul: Yes, that’s the other extreme.

Beyza: How did the relationship start with Fox?

Paul: They have an innovation lab at Fox and we met with them this summer. They had heard about what we were doing, at the time we had only really done ‘Strangers,’ and we showed them that. They told us they wanted to make a companion piece for Wild. Coincidentally we are from Montreal and so was the director of the film. They said they wanted to do a perspective, they felt like it would be a good match for the medium. They showed us a work in progress of the film and they asked us, ‘what do you think?’

What we found interesting and this is kind of a side effect of my difficulties with cinema these days, is that although the loneliness is a topic in the film, you don’t feel that loneliness. You understand that she is going through that and you can empathize with that but you can’t feel it yourself. Why don’t we try to amplify that? That was the inspiration for the piece. I’d have to let other people tell me if that worked but the impetus was really about taking the characters and the story, the themes of the film and doing something in VR that we just couldn’t do in film.

You know piano has a certain type of range and tones and a saxophone has different tones. They overlap a little but they go in different places and they compliment each other.

Beyza: What’s your next project?

Paul: Our next project is actually a second project with Cirque de Soleil and Samsung. It’s going to be a full hour. It’s a more ambitious Cirque de Soleil piece and we’ve got a few other companion pieces to films. There was something interesting about doing a companion piece because it takes some of the pressure off telling a story in VR and building the characters. I think Wild stands alone, if you haven’t seen the film. It’s certainly a richer experience if you have seen the film.

We also want to explore wholly original fiction and we have other projects like Herders in the works too.

Beyza: Where do you see this technology going?

Paul: The easy answer is it’s going to get higher quality, lighter, more transparent and I think it’s going to get more and more widespread in the next few years. Microsoft just announced an augmented reality device. We know Sony and Facebook have VR. Google is doing cardboard and is bound to evolve. There’s faith, from the major tech companies, that this is going to be the next major computing platform, whether it’s VR, AR or both.