Overview

The Westwood Experience was a location-based experience created and operated by Nokia Research Center Hollywood for several performances on December 2009 and January 2010. Each performance lasted a little over an hour, depending on the speed of the participants. Small groups of 5-8 people participated in each performance. The Westwood Experience was built as a research experiment to investigate and evaluate a variety of techniques to connect a narrative to unique and powerful real world locations, and as a showcase for the grand opening celebration of our new laboratory.

We believe the contribution of this experience (compared against previous work) comes from the combination of three things:

The map below provides an overview of the locations in The Westwood Experience and shows the route that the participants walk. This is an area south of the UCLA campus. The numbers in the green circles match the sections listed below with more detailed explanations of each part of the experience.

Route

1) Crest Theater

The experience starts at the Westwood Crest theatre, which is one of the many wonderful movie theaters in Los Angeles, featuring black light artworks of many LA landmarks. The participants are seated in the theater, the curtains open, and they see a short video of the (honorary) Mayor of Westwood practicing his entrance. The Mayor leaves the movie screen by walking off the right edge of the screen, and then he appears in person from the right side of the theater. He introduces himself and describes the experience they are about to go through:

“I do believe our story is better told the way we’re going to tell it. A new sort of story, the first ‘location based, mobile device driven, interactive, narrative experience.’ Creating it has been as exciting for us as making movies must have been for those involved in the early days of film making. We couldn’t help think too of the first movies ever shown, of trains, which seemed to come right out of the screen, and sent audiences scrambling to get out of the way. The story there was, ‘Look, a train!’ We won’t terrify you that way, and our story is a little fuller, we hope. This is only a beginning, but if, when it’s over, you feel a tiny part of what those first film audiences felt diving out of the way of that train, we’ll have succeeded.”

The participants now each receive a Nokia N900 device. Before they can go on the experience, they play a game together to practice using the interface on the N900. In this game, the N900’s touchscreen acts as a game controller, where each participant can move a colored cursor. Each participant has a unique color. The game takes place on the theater screen, not on the mobile device. This is our first example of moving the experience away from the mobile device and inserting it into the environment around the user. We are in a movie theater; why not use the large movie screen for a collaborative game? In this game, colored shapes move rapidly outward on the screen. The participants move their cursors to touch the shapes that match their personal colors. When they do, that shape is captured and becomes part of a figure in the middle of the screen. Eventually this figure is completed and becomes an image of the Mayor himself.

Game in theater

 

After they complete the game, the Mayor tells the participants to stand up and he leads them out of the theater and into the lobby. This experience starts in a home of traditional media (stage theater and movies) but it will not take place there. As he leads the participants out, the Mayor comments on the LA landmarks painted on the walls, including ones that they will soon see in person.

Crest theater

2) Map effect

Once in the lobby, the Mayor informs the participants that they will be going through the experience on their own. He will not accompany them in person, but they will hear his narration on the N900. Before they leave, the Mayor helps them by drawing a simple map of the town, covering the route they will travel. He hand draws this map on a sheet of paper, in full view of the participants. This map now exists as a real object. How, then, to get this into the participants’ N900 devices? The Mayor asks each person to take a picture of the map, from any angle. Then a custom computer vision algorithm that we wrote for this recognizes the map, warps the image so that it becomes upright, and then animates the route that the participants will be taking. This image then becomes the background for the help screen for the rest of the experience.

Map effect

 

This effect is not faked. The participants really do snap a picture of the hand-drawn map, and it really is recognized and warped to become the help screen. To emphasize that this is the actual sketch that they just saw drawn, and not some pre-stored image, we asked the actor to put his index finger on top of the hand-drawn map, pointing to the first stop on the route, and his finger is captured as part of the photo the participant takes. You can download a video documenting the map effect and also download a second video that includes the UI, or play the two embedded videos below:

We created this effect as something that participants might find delightful and magical. This is the second technique for mixing real and virtual together, where a real object, created in front of your eyes, is captured and becomes a key part of the user interface.

After participants successfully capture a map, they each get a pair of headphones and start using them with the N900 so they can hear the Mayor's narration on the device. The participants are also reminded to be careful while walking around town. They are wearing headphones and looking down at a device, which is potentially dangerous while walking through intersections. The experience was designed to have users walk to a point, then stand still while experiencing content, and then to walk again. However there was nothing physically preventing users from walking while distracted by the content, if they chose to do so.

The participants also learn that the Mayor’s name is Pete. They are escorted as a group by a staff member to the spot of the Peet’s effect.

3) Peet’s effect

Participants stand in front of the Peet’s Coffee building, which is a historic building in Westwood. In the past, it used to be a Ralph’s grocery store and was a local landmark. Participants are asked to take a picture of that building with their N900. If the effect works (and it usually did), our computer vision algorithm would recognize the building and its location and then perform a “static AR” effect where that captured image is processed so the Peet’s logo is covered up, a Ralph’s logo is superimposed, and other elements like an old car are put into the scene. Then as Mayor Pete tells you the story of this building (as a voice heard via the headphones), the image on the screen smoothly dissolves to the purely virtual illustration of what the building looked like in 1949. Then after Pete is done talking, the image dissolves back to the real world view from the N900’s camera.

The effect occasionally does not work if the lighting conditions are too different from the template we compare against. If our computer vision algorithm cannot find a match, the N900 then asks the user to line up the view of the Peet’s building with a virtual outline (similar to the Yamato outline shown in Section 4), and then the captured image dissolves directly to the purely virtual illustration.

You can download a video showing this effect, or play the embedded video below:

Peets effect

 

Notes on navigation

Participants navigated through the experience through sequences of image clues. They were presented with an image of an object that they had to find and then go to. After they reached the designated object, they pressed a button and were presented with another image clue (or an effect or movie, if there was content at that particular spot). Sequences of these images led the participants point-to-point along the intended route. These images were simple puzzles, which were intended to be easy to solve, although some participants treated this as a game and competed to see if they could advance more quickly than the other participants.

Example navigation clues

Why not use GPS to track user positions? GPS was not accurate in Westwood because there were many high-rise buildings surrounding the streets along our route. Therefore we resorted to this low tech, but perhaps more engaging and interactive method of navigation

If participants got lost, they could also bring up the map through a help button or ask fellow participants for help.

4) Yamato effect

Participants stand in front of a striking domed building that is another landmark in Westwood. Today it is the Yamato restaurant, but in 1949 it was a Bank of America building that had a tall tower on top of the dome. The N900 draws a virtual outline of that building on top of the real world view, and the participant is asked to line that up with the Yamato building, as shown below. Once that happens, the view dissolves to an illustration that shows the building as it was in 1949. This is part of a 360 degree virtual panoramic image shown below. As the participant pans and tilts his or her N900, the device selects a different part of the panoramic image corresponding to the current direction of the device. The end result is that the N900 turns into a “magic lens” that looks back in time, where the participant can see the real world directly with his or her own eyes, but on the N900 the participant sees the same environment as it was in 1949. The participant can spin around in place 360 degrees for an immersive view of the environment at that location.

Since the N900 only has accelerometers and no compass or gyros, we added an external sensor that contained a compass and rate gyros in order to implement this effect.

Most participants considered this their favorite moment in the experience (along with the ending, described in Section 9) and several said that the entire experience should have been based on this effect.

Why not use computer vision to identify the Yamato building? We tried initially, but the detection rarely worked. That building is surrounded by tall palm trees that cast a complex, ever-changing pattern of shadows onto the building as the sun moves. These shadows are detected as features in the computer vision algorithm, making it difficult to match against a template. Also, participants actually found it more fun to line up the outline.

Yamato panorama

 

5) Norma

Pete now begins to tell you his story. Across the street from the Yamato building used to be a cafe. Pete used to be a sailor and while in that cafe, he met a strikingly beautiful woman named Norma. They left together and she took Pete to a building that today is the Westwood Brewery.

While Pete tells a story at a location, participants see one or more static illustrations on the N900, like the two examples shown below. The illustrations are animated through Ken Burns effect animations. The illustrations attempt to show locations near the participants as they were in 1949 (in our fictional story).

Cafe

 

6) Westwood Brewery

In our story, the Westwood Brewery used to be a theater arts school, where Norma had access to a rehearsal space. As the participants reach the Westwood Brewery, they are met by one of our staff members who gathers the entire group and takes them upstairs to a room. Inside, the participants find that the room is set up as that rehearsal space in 1949, as shown in the images below. We rented the room from the restaurant and had a set director design and fill the space with authentic pieces from that period, even including the perfume that Norma wore and the cigarettes she smoked.

Inside this room, the experience is reversed from what the participants have experienced so far. Up to this point, they have heard Pete’s narration through headphones and seen virtual depictions of 1949 on the screen of the N900. Inside this room, the experience takes place in the real space rather than on the N900. The representation of 1949 is in the form of real objects, not virtual images. In this room, participants only hear Norma talking, not Pete. Her voice is heard through spatialized audio (several speakers hidden in different parts of the room). Norma describes how she and Pete spend one magical evening together, but the next day she tries to convince Pete that she can’t be the person who spends her life with him. Pete doesn’t listen and goes outside. Norma then calls a friend and arranges to have him pick her up in his car later that day. Then our staff member gathers the group, leads them outside the Westwood Brewery and has them continue on their journey, following what Pete did.

Set 1

Set 2

Set 3

Set 4

 

7) Losing Norma

The participants experience several story elements at different locations as they walk south on Glendon Avenue, finishing Pete’s story. They stop at a jewelry store where they learn that Pete impetuously bought a ring to prove his love to Norma. He sees Norma leaving and catches up to her and forces her to take the ring. Norma tells him that she can’t be with him because she is trying to protect him. She knows the path she is going to take in life and doesn’t want him to be hurt by the tragedy yet to come. Norma distracts Pete and is able to slip into the car that she had arranged to come pick her up. As the car drives away, she waves goodbye. That is the last time that Pete sees her in person. Much later, Pete meets a different woman and that person becomes his actual wife.

Losing Norma

 

8) Meeting Norma in person

In the Westwood Experience, we take the participants back to the year 1949 via various effects, and then tell a simple story of a striking woman that Pete met, loved, and lost. Now comes the payoff of the experience. Pete tells you that you will meet Norma in person. And there is a powerful moment when the participants realize how this is going to happen: by entering a cemetery, which is hidden away behind the high-rise buildings along Wilshire Blvd.

You are not going to enter a Hollywood set, or see Norma as some image or virtual model that appears on your mobile device, or meet her by encountering an actress portraying her.

You are actually going to meet her, the real person, in a real cemetery, in her actual resting place. The story that has led you here is fictional, but Norma is real.

The moment you enter a cemetery, you think differently and you act differently. You are reminded of your own mortality, the end that will come to each one of us, someday.

“She became what she’d said she would, a movie star... In the end, Norma came back to Westwood too. She’s between engagements now, ‘resting’ as actors sometimes say. She’s not alone, but among many others, some who lived their lives as publicly as she did, many of their names once as familiar as hers. I come to see her every so often, as many others do. I’m taking you to her now. I know you’ll be mindful of the customs proper to the place we find her, a place of real people and real endings.”

The Gates

9) Crypt

The last set of image clues leads the participants to Norma’s final resting place. The name on her crypt is never directly mentioned, but Pete provides enough details so that participants can unambiguously determine which crypt is hers. There is a bench nearby with her name on it, and there are always flowers on her crypt.

The participants discover that Norma is Norma Jeane Baker, better known to the world by her other name: Marilyn Monroe.

When a participant finds and stands in front of the crypt, a final movie plays on the N900. It shows footage of her funeral service. In some parts of the movie, the participant can clearly see a 1-1 correspondence between the movie footage and his/her direct view of the crypt. This is not AR, but it is a technique that Steve Feiner’s group at Columbia University called “Situated Documentaries.” When combined with our original musical score composed for this location, most participants reported this was a powerful and poignant moment. You can download the video that actually played on the N900 in this spot, or play the embedded video below. One participant stated that he had a feeling of discovering the crypt on his own, and when he found it, the emotions he felt were far different than he would have felt if a tourist guidebook had led him there.

Crypt

 

This location is the reason the Westwood Experience was created. It is a compelling example of the Power of Places, and how one might leverage that power in a respectful manner through new forms of mobile experiences. The experience ends with an invitation to explore the cemetery further, as there are many other famous people buried there. Participants hand their equipment to a waiting staff member and are then free to explore and contemplate on their own. This cemetery is a “hidden gem” of Los Angeles. While not a secret, many people who work and commute nearby remain unaware of its existence and importance. We now invite you to also explore Westwood Village Memorial Park Cemetery, through this list, this tour, and this map, or to visit in person if you are able to.