news.

Non-visual 3D Environments

For most people, the title might seam to contradict itself. 3D, at least if you’re not a mathematician, immediately brings to mind 3D computer games, 3D movies, and other visual media. In this article however, I will not focus on the visual aspect associated with 3D. In fact, I want to describe a 3D experience without any graphical rendering whatsoever. You see, I believe although it is the most common, a visual rendering is not the only method of portraying 3D scenes.

To explain this from another angle, most readers would, at some point, have played a computer game with the sound turned off. This is possible, because unlike graphics, sound can be considered to be a secondary medium. Now imagine playing a game with the screen turned off… This is not something people usually attempt, and certainly not when the game is played online! This is, however, what I will be focusing my research on, not only for gaming, but for 3D virtual environments in general. 3D environments are being applied to all kinds of scenarios these days, including traditional 3D gaming, education, and communication. A non-visual 3D rendering is needed as a graphical display might not always be available. This is especially true for accessibility, i.e. making 3D environments accessible to blind people, but also for something like mobile gaming. Imagine for instance, playing a 3D computer game on a digital audio player.

One necessary element for non-visual 3D rendering is realistic audio. When considering the audio rendering of current computer games, there is a lot of realism still to be added. Most game events will have some sound attached to it, and most creatures or equipment as well. There are objects, however, for which no sound is played. This includes most inanimate objects like the walls, trees and even the floor. But wait! I hear you exclaim, those things don’t make any sound, not even in real life! Actually, they do. Every sound in our environment is not only heard by our ears, but also reflected by every object nearby. The sound is not only reflected, however, it is also changed depending on the material a sound is made of. To test this, cover your eyes (and only your eyes) so that you can’t see a thing. Now move some object e.g. your hand past your face. You should be able to know when the object is moving passed your face. Your ears are able to hear the sound reflections coming off the object and thus provide you with it’s approximate location. This ability may be further developed when practiced, and is recognized as one of the main navigational aids available to the blind (from there one of the reasons for the “tap-tap” of a white cane). For a non-visual 3D rendering to be successful, this “echolocation” must also be simulated by the auditory medium.

The second medium for providing location information in a non-visual 3D environment is through textual descriptions. This might be similar to the descriptions provided by text based games like Zork or Adventure, “You are standing at the end of a path in front of a building. A stream flows nearby. etc.”, but adapted to a 3D environment. For the old Zorg and Adventure games, simple compass directions like north and northeast were enough. For a 3D environment, a more detailed direction and distance system might be needed. This can either be done by giving the location in the description, “You see a dwarf at 1 o’clock at 3 meters” or the description of the object might be spoken at its actual location in 3D space using spacial audio.

For the reader wanting to experience true eyes-free gaming, I will give some examples of audio games produced by the blind gaming community. Shades of Doom available from GMAGames (www.gmagames.com) involves the player exploring an abandoned military installation. The game uses sound queues and voice prompts to inform the user about the environment. Another audio game called Terraformers (www.terraformers.nu) lets the user explore his/her location by using a virtual sonar. Some more games can be found on the website audiogames.net. Note that most audio games are not 3D games, however, simpler side-scrollers and board games seem to dominate.

The examples I gave above demonstrate some methods of non-visual virtual environment rendering, however I feel that more could be done. None of them simulates echolocation for example. The 3D environment is also not utilized to the fullest — most objects are on the same vertical level as the user. This all just means that I have quite a number of interesting things to explore in my research. I might even explore some alternative output methods like using the vibrator (found on most mobile phones) as a way to communicate texture information.

No comments yet.

Leave a comment

Leave a Reply

(required)