Bob Yarwood

I am British, 73, and for most of my career I was a Chartered Engineer working in the oil industry. I specialised in the design of control systems for oil refining and chemical processing.

Best full house remodeling in Los Angeles DomumRemodeling

This essay has been written to publicise the idea of having movies and video games which are watched with a head-mounted display helmet, so that instead of having a picture projected onto a screen, viewers would appear to be inside the scene with the action going on all round them. As far as they could see and hear, they would be in an entirely different place - that is, wherever the camera was which shot the scene. They would be able to look up, and turn right round, and the scene would be all round them and above them. They would not be able to see or hear anything of their real surroundings, although of course they could still feel the chair they were sitting in.

Before I go into the details of the system I will introduce myself and give some details of my background. I am British, 73, and for most of my career I was a Chartered Engineer working in the oil industry. I specialised in the design of control systems for oil refining and chemical processing. My academic qualifications are a B.Sc. in Physics and an M.Sc. in Control Systems. Both these subjects are general, in the sense that Physics is the basis of all the natural sciences and Control Systems appear in many branches of engineering – mechanical, electrical, electronic, optical and aerodynamic. This background has given me a basic understanding and ‘feel’ for what is possible now in science and technology, and what will be possible in the near future.

I think this would be a major revolution in the movie industry, similar in many ways to the introduction of sound movies (talkies) in the 1920s. It would also revolutionize video games, because instead of watching characters on a screen viewers would be able to take part in the game in person, actually going into the dungeon or wherever, and having the animated characters all round them.

The technology exists right now to make a good start on this, but so far no-one has taken the idea seriously. To me, it would be the logical and obvious next step in movies and video games.

I have provisionally invented the term “Virtual Space” for this technique, because I think it best describes the fact that you can be in two spaces at once. Your body can be in real space, but your eyes and ears, and more importantly your mind, can be in another space altogether - a fictional or 'virtual' space. I like to think of the head-mounted display (HMD) as a magic helmet which instantly transports the wearer to another place. Anyone who has seen (and understood) the movie “The Matrix” will see at once what I mean. I don’t think there is any chance that things will ever be like they are in that movie, but it does show the basic idea. Some people have suggested the term “virtual reality” for this, but I think that phrase has already been used to mean too many different things already. It usually refers to computer-generated images which are as close as possible to photographs or films of real subjects, and which we have already seen in ordinary movies and on television.

Leaving the technicalities aside for the moment, I think you can see that this would be an awesome experience. Sitting at home in your living room, you would put on the helmet and immediately be in another place. We get most of our information about the world through our eyes and ears, so that if your eyes and ears tell you that you are standing on top of Mount Everest, or deep underwater, or floating in space outside the International Space Station, it would be a very powerful and exciting feeling. As I watched the opening of the Beijing Olympics on television, with all that magnificent display, I was frustrated by the limited view of the television camera and screen. With just one virtual-space (VS) camera set up in the middle of the field, everyone on Earth with access to the right equipment could have been standing right there with an all-round view exactly as if they were there in person. Although there has not yet been a human landing on Mars, VS cameras could be mounted on the next robotic Mars explorers. This would mean that we could go to Mars, in a sense, before any of the astronauts and we could travel round the planet seeing exactly what we would see if we were actually there. Other probes could take us to the moons of Jupiter, and we could see the rings of Saturn filling the sky.

As we often watch movies in company with friends or relatives, two or more people could watch a VS movie together. There could be voice communication between the helmets, so that the viewers could talk to each other and discuss the action. Everyone would be watching the same scene, but not necessarily looking in the same direction at the same time. In computer games, two or more people would even be able to play in the same game, with each person seeing the others as animated characters – “avatars” as I believe the term is.

The components of the VS system are a helmet with a screen inside and earphones for the sound, some means of detecting the movement of the wearer’s head, and a computer which adjusts the scene shown in the screen to correspond with it. This is called “head tracking”. The viewer thus gets the impression that he is inside the scene, with the action going on all round, and gets a very powerful sense of “being there” in person. The view in every direction must be available in the computer so that it can select the right view to correspond with the direction the viewer is facing. For filmed movies this means that the camera must have an all-round view – 360 degrees in the horizontal plane and 180 degrees in the vertical. For animated movies there would be no need for a camera as the whole scene would be written in the software. For maximum realism, the optics in the helmet must be designed to give the same field of view as the human eye – approximately 180 degrees.

For filmed movies, there would have to be big changes in the methods of filming. There could only be one camera in each scene, because if there were more than one, they could see each other. Likewise, the director and crew would have to be concealed from sight – probably in another place altogether, watching the scene through their own HMDs.

The first VS movies would probably be documentaries made specially to demonstrate the possibilities of the method, like the early Cinemascope and IMAX movies. At first, even to mount a VS camera on top of a truck and drive through some attractive mountain scenery would produce a movie which would be an exciting novelty. The viewers would feel that they were right there on the truck, and there could be a guide and commentator talking to camera in the normal way. Each viewer would feel that he alone was there and that the commentator was talking to him, but of course the conversation would have to be one-sided!

As for what subjects would be covered in VS movies, they would be exactly the same subjects as we have already in ordinary movies. Scenes of the kind that have been shown for decades in ordinary movies will acquire a new interest as audiences experience the personal involvement, the sense that they are going to these places in person instead of looking at a picture made by someone else. Anywhere on Earth, in fact anywhere in the Solar System where a space probe could go, anyone with the right equipment could be there in person.. And they would have none of the difficulties that they would have if they were there in reality. Sitting in your living room, safe and warm, you could be climbing up Everest or a vertical rockface with no danger, wing-walking on an aircraft doing aerobatics, swimming underwater with no breathing difficulties, or floating in space outside the space station with no space suit. Using something like the Google Earth software you could fly to any place on Earth, across the actual landscape, like Superman. In fact, like Superman we could fly to Mars and back in a few hours. With animated movies we could walk through buildings, or cities, that have disappeared long ago, or have yet to be built, as long as we have the drawings or sufficient other information about them.

One example of this could be the Crystal Palace in London, at the Great Exhibition of 1851. The building was destroyed by fire but the original drawings are available, so that we could walk through the building in company with crowds of Victorian characters which had been programmed in using the latest virtual reality software.

Everyone could have a personal planetarium with all the information about the stars which normal planetariums have, and travel round the Solar System or the Galaxy faster than light. Looking further ahead, as soon as there is a return to the Moon, or an expedition to Mars, everyone could be travelling along with the astronauts and living with them when they arrived. I think NASA would appreciate that way of generating public interest and support for the exploration of space!

Apart from the entertainment industry, there will be other uses for VS techniques. . Architects can walk their clients, room by room and floor by floor, through buildings which exist only as drawings, so that they can see exactly what they would see in the finished building. Likewise they could drive round a proposed building complex. We could see what the 2012 Olympic stadium will look like from the inside. or travel round the whole site before it is finished in reality – and we could choose where we want to go in the site instead of following a set route. Travel agents can show their customers exactly what a particular place would look like if they were there. Simulators of various kinds would be easier and cheaper to build – for example they could simulate driving a car or flying a small plane. Hazardous areas, such as the inside of nuclear reactors or the craters of active volcanoes, could be inspected by a robot with a VS camera.

To see whether all this could be done with today’s technology, we need only consider what we can already do. We can send a robot to Mars and have it travel around taking photographs and doing chemical testing of the soil. We have commercial airliners with hundreds of passengers doing landings in thick fog entirely under the control of a computer. We have aircraft like the B2 bomber or the Typhoon Eurofighter which are aerodynamically unstable and have to be tightly controlled by a computer just to stay in the air. We are beginning to see robots that move uncannily like human beings. Compared with all these, VS movies and video games as I have described them would be perfectly possible and even a lot easier.

My final reason for thinking that it would work is that I have already seen a working model. It was in a seaside amusement arcade several years ago. I stepped onto a platform put on a helmet, and immediately I was in a different place. It was the most ridiculously simple static scene you could imagine, just a boxing ring with the ropes and corner posts, and that was all. The rest of the hall was, conveniently, too dark to see the audience! When I turned round, sure enough the ropes and corner posts were all round me. Unfortunately, when I moved there was a noticeable time lag before the scene changed, which did not help the realism. Still, it showed the idea could work, and it was a fascinating novelty for me. On my next visit I tried to find out where it came from but it had disappeared and I didn’t think to follow it up at the time. I have heard of similar exhibits somewhere in Florida, but I don’t know the details.

From what I know of what is happening in science and technology, I see no reason why a VS system could not be put together right now, as all the component parts are already in existence. It would probably be quite small-scale at first and would be improved and extended in time, just as ordinary movies went from black and white to color and from silent to talkies. VS movies would almost certainly be for home viewing only, as there would be no point in going to a movie theater. You would buy or rent some kind of disc or memory stick and play it in a special-purpose player - or perhaps by that time your ordinary home computer could do the job The motion detectors could be external - that is, installed around where the viewer is sitting, or they could be inside the helmet.

Probably the biggest problem will be in the design of the optics inside the helmet. I envisage a solid state screen of some kind, either LCD or plasma, curved into a hemispherical shape so as to fill the viewer's field of view. This should not be a big problem, as we already have screens of this type in flat form, but the screen will be very near to the eyes. To avoid eyestrain there will need to be a diverging lens system so that the image appears to be far away. This is standard practice with optical instruments, but they only have a very small field of view. How it would work with a hemispherical screen I don't know, and I think this would be the really tricky part of the design Still, I am only a physicist and not an optical systems design engineer, so I will have to leave that to others more qualified than myself.

As a first step, we could use something like the video glasses which are becoming available. These are worn like spectacles but instead of lenses there are two very small screens. The viewer gets what looks like a large movie screen which can be watched in a very small space. They are mainly for use with personal media players like iPod which have very small screens and are hardly suitable for watching full-length movies. They will be ideal for watching movies in a seat in a car or plane, and I think the airlines will one day supply them to their passengers for watching in-flight movies. Everyone would get a perfect view, and no-one would be able to walk in front of the screen! Some of the manufacturers are Myvu, eDimensional, Zetronics and Vuzix, and all can be found with Google.

While these glasses are a long way from the Virtual Space helmets as I have described them, they are nevertheless an important step in the right direction and may well be used in the first-generation VS sets. The scene shown on the screen would change according to the position of the viewer's head so that the viewer would get an all-round view, but only by moving his head around. The effect would be like wearing blinkers which restrict the view to straight ahead. In time, after more research and development, the screen would get bigger and would show more of the all-round view.

So why hasn’t it happened yet – and more importantly, why has there not been a single mention of it in any book or magazine article that I have read? Well, I don’t know, but history tells us that things do not always happen in a logical order at the logical time. Consider the introduction of sound movies in 1928, which incidentally has many parallels to the introduction of VS movies now. In 1920, radio broadcasting was started. Does that mean that after radio broadcasting started, another 8 years of research and development were needed for sound movies to become a reality? No, certainly not! Radio broadcasting was much more complex and sophisticated than sound movies. Because a modulated carrier system had to be used, they needed radio-frequency circuits, and transmitting and receiving antennas. Sound movies needed much less sophisticated equipment, and could have been developed before sound broadcasting.

One reason might have been that the biggest and most successful companies in the movie business just did not want sound movies. They were doing well as things were, and saw no reason to risk investing in a radically new technology with all the difficulties and problems that they would have. It might not have worked commercially, and even if it did, the same companies might not have been the most successful in the new medium. I think that might true today of VS movies – after all, I can hardly be the only person in the world who has thought of them!

I want to emphasise that I consider VS movies an entirely natural and obvious development of present technology, and one day we will look back and be amazed that it took so long for the idea to be accepted.

I have not dealt in this essay with the camera and computer control parts of the VS system, because I think they are well within present-day technology. At the very least, a VS camera could be made from a number of small cameras on a framework, spaced so that their fields of view overlap. Likewise, the most awesome things can be done in computing already so that to set up computer control of the VS set linked to motion detectors would be a fairly straightforward matter

October 2009

I have recently become aware of the Nintendo Wii Remote, which uses motion sensing as part of a video game. The details can be seen in the Wikipedia articles titled "Wii Remote" and "Wii Motion Plus". It seems to be a very sophisticated system which has been under development since 2001. This is very encouraging, and confirms my belief that all the component parts of a VS system for computer gaming are now in existence. All it needs now is for someone to put the whole system together, and it looks as though the best companies to do this would be Sony or Nintendo.

I haven't used any pictures in this essay because they would be pictures on a screen, which is what I am trying to get away from. In any case, it's easy to understand what a VS movie would look like, because it wouldn't look like a movie at all - it would be like being somewhere for real.


Russian version