Augmented reality (AR) refers to computer displays that add virtual information to a user's sensory perceptions. Most AR research focuses on "see-through" devices, usually worn on the head that overlay graphics and text on the user's view of his or her surroundings. AR systems track the position and orientation of the user's head so that the overlaid material can be aligned with the user's view of the world.
Consider what AR could make routinely possible. A repairperson viewing a broken piece of equipment could see instructions highlighting the parts that need to be inspected. A surgeon could get the equivalent of x-ray vision by observing live ultrasound scans of internal organs that are overlaid on the patient's body. Soldiers could see the positions of enemy snipers who had been spotted by unmanned reconnaissance planes.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what makes augmented reality different is how the information is presented: not on a separate display but integrated with the user's perceptions. In augmented reality, the user's view of the world and the computer interface literally become one.
Video games have been entertaining us for nearly 30 years. Computer graphics have become much more sophisticated since then, and soon, game graphics will seem all too real. In the next decade, researchers plan to pull graphics out of your television screen or computer display and integrate them into real-world environments. This new technology, called augmented reality, will further blur the line between what's real and what's computer-generated by enhancing what we see, hear, feel and smell. The basic idea of augmented reality is to superimpose graphics, audio and other sense enhancements over a real-world environment in real-time. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information.
Walk down the street, look at the world. This is reality. Now repeat, but wearing an odd-looking, bulky pair of glasses that place into your line of vision selective, relevant bits of data about the world or informative graphics and produce sound which will coincide with whatever you see.. This is augmented reality. An AR system, can superimpose computer generated text, graghics,3-D animation, sound, or any other digitised data on the real world. The augmented reality systems employ a see-through head-worn display that overlays graphics and sound on a person's naturally occurring sight and hearing. By tracking users and objects in space, these systems provide users with visual information that is tied to the physical environment. It not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective.
Augmented-reality displays will overlay
computer-generated graphics onto the real world.
On the spectrum between virtual reality, which creates immersible, computer-generated environments, and the real world, augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world as it exists. You can expect video games to drive the development of augmented reality, but this technology will have countless applications. Everyone from tourists to military troops will benefit from the ability to place computer-generated graphics in their field of vision.
Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with whatever you see. These enhancements will be refreshed continually to reflect the movements of your head. In this article, we will take a look at this future technology, its components and how it will be used.
2. COMPARISON WITH VIRTUAL REALITY
COMPARISON WITH VIRTUAL REALITY
Augmented reality is very much close to virtual reality. Virtual reality is a technology that encompasses a broad spectrum of ideas. The term was defined as "a computer generated, interactive, three-dimensional environment in which a person is immersed." Virtual reality creates immersible, computer generated environments which replaces real world .Here the head mounted displays block out all the external world from the viewer and present a view that is under the complete control of the computer.
A very visible difference between the two types of systems is the immersiveness of the system. Virtual reality strives for a totally immersive environment. The visual, and in some systems aural and proprioceptive, senses are under control of the system. In contrast, an augmented reality system is augmenting the real world scene necessitating that the user maintains a sense of presence in that world. The virtual images are merged with the real view to create the augmented display. There must be a mechanism to combine the real and virtual that is not present in other virtual reality work. Developing the technology for merging the real and virtual image streams is an active research topic
Augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world, as it exists. Thus it augments the real world scene in such a way that the user can maintain a sense of presence in that world. That is, the user can interact with the real world , and at the same time can see, both the real and virtual world co-existing. For the same reason it has a large number of applications in the day to day life as compared to virtual reality.
The computer generated virtual objects must be accurately registered with the real world in all dimensions. Errors in this registration will prevent the user from seeing the real and virtual images as fused. The correct registration must also be maintained while the user moves about within the real environment. Discrepancies or changes in the apparent registration will range from distracting which makes working with the augmented view more difficult, to physically disturbing for the user making the system completely unusable. An immersive virtual reality system must maintain registration so that changes in the rendered scene match with the perceptions of the user. Any errors here are conflicts between the visual system and the kinesthetic or proprioceptive systems.
Milgram (Milgram and Kishino 1994; Milgram, Takemura et al. 1994) describes a taxonomy that identifies how augmented reality and virtual reality work are related. He defines the Reality-Virtuality continuum shown as Figure 2.
Figure 2 - Milgram's Reality-Virtuality Continuum
The real world and a totally virtual environment are at the two ends of this continuum with the middle region called Mixed Reality. Augmented reality lies near the real world end of the line with the predominate perception being the real world augmented by computer generated data. Augmented virtuality is a term created by Milgram to identify systems which are mostly synthetic with some real world imagery added such as texture mapping video onto virtual objects. This is a distinction that will fade as the technology improves and the virtual elements in the scene become less distinguishable from the real ones.
3. EARLY DEVELOPMENT
The first AR system was developed in the 1960s by computer graphics pioneer Ivan Sutherland and his students at Harvard University and the University of Utah. In the 1970s and 1980s a small number of researchers studied augmented reality at institutions such as the U.S. Air Force's Armstrong Laboratory, the NASA Ames Research Center and the University of North Carolina at Chapel Hill. It wasn't until the early 1990s that the term "augmented reality" was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
The past decade has seen a flowering of AR research as hardware costs have fallen enough to make the necessary lab equipment affordable. Scientists have gathered at yearly AR conferences since 1998. Eventually, possibly by the end of this decade, we will see the first mass-marketed augmented-reality system, which one researcher calls "the Walkman of the 21st century."
4.COMPONENTS OF AUGMENTED
COMPONENTS OF AUGMENTED REALITY SYSTEM
What augmented reality attempts to do is not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Here are the three components needed to make an augmented-reality system work :
Â¢ Head mounted displays
Â¢ Tracking and orientation system
Â¢ Mobile computing power
Photo courtesy Columbia University Computer Graphics and User Interfaces Lab
Early prototype of a mobile augmented-reality system
The goal of augmented-reality developers is to incorporate these three components into one unit, housed in a belt-worn device that wirelessly relays information to a display that resembles an ordinary pair of eyeglasses. Let's take a look at each of the components of this system.
4.1 HEAD MOUNTED DISPLAYS
Just as monitors allow us to see text and graphics generated by computers, head-mounted displays (HMDs) will enable us to view graphics and text created by augmented-reality systems. So far, there haven't been many HMDs created specifically with augmented reality in mind. These forms one of the main components of an augmented reality system. They are used to merge the virtual world and real world in front of the user in such a way that he feels he is looking at a single real scene . They resemble some type of skiing goggles.
There are two basic types of HMDS:
Â¢ video see-through
Â¢ optical see-through
VIDEO SEE- THROUGH
Video see-through displays block out the wearer's surrounding environment, using small video cameras attached to the outside of the goggles to capture images. On the inside of the display, the video image is played in real-time and the graphics are superimposed on the video. One problem with the use of video cameras is that there is more lag, meaning that there is a delay in image-adjustment when the viewer moves his or her head.
Optical see-through displays is not fully realized yet. It is supposed to consist of a ordinary-looking pair of glasses that will have a light source on the side to project images on to the retina.
There are advantages and disadvantages to each of these types of displays. With both of the displays that use a video camera to view the real world there is a forced delay of up to one frame time to perform the video merging operation. At standard frame rates that will be potentially a 33.33 millisecond delay in the view seen by the user. Since everything the user sees is under system control compensation for this delay could be made by correctly timing the other paths in the system. Or, alternatively, if other paths are slower then the video of the real scene could be delayed. With an optical see-through display the view of the real world is instantaneous so it is not possible to compensate for system delays in other areas. On the other hand, with monitor based and video see-through displays a video camera is viewing the real scene. An advantage of this is that the image generated by the camera is available to the system to provide tracking information.
The optical see-through display does not have this additional information. The only position information available with that display is what can be provided by position sensors on the head mounted display itself.
4.2 TRACKING AND ORIENTATION SYSTEMS
The biggest challenge facing developers of augmented reality is the need to know where the user is located in reference to his or her surroundings.
In order to combine real and virtual worlds seamlessly so that the virtual objects align well with the real ones, an AR system needs to know two things precisely:
1) where the user is located, and
2) where he is looking.
A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given moment. Currently, both video see-through and optical see-through displays typically have lag in the overlaid material due to the tracking technologies currently available. For augmented reality to reach its full potential, it must be usable both outdoors and indoors.
There are ways to increase tracking accuracy
Small area tracking and orientation systems
Tracking is easier in small spaces than in large spaces. Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 square feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
Â¢ six user-mounted, optical sensors
Â¢ infrared-light-emitting diodes (LEDs) embedded in special ceiling panels
Photo courtesy Tracking Project at UNC-Chapel Hill
The Hiball Tracking System uses an optical sensing device and LED-embedded ceiling tiles to track movements over a short range.
The system uses the known location of the LEDs, the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user's position and orientation. The system resolves linear motion of less than .2 millimetres, and angular motions less than .03 degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one millisecond
Large area tracking and orientation systems
For instance, the military uses multiple GPS (Global Positioning System) signals. There is also differential GPS, which involves using an area that has already been surveyed. Then the system would use a GPS receiver with an antenna that's location is known very precisely to track your location within that area. This will allow users to know exactly how inaccurate their GPS receivers are, and can adjust an augmented-reality system accordingly. Differential GPS allows for sub meter accuracy. A more accurate system being developed, known as real-time kinematic GPS, can achieve centimetre-level accuracy.
In case of out door application where the movement of user will be comparatively larger, his location with respect to his environments is tracked with the help of GPS RECEIVERS which works in coordination with the GPS satellites and the direction of vision of the user is calculated down to few degrees by INERTIAL/MAGNETIC TRACKER.
Tracking using GPS
The Global Positioning System (GPS) is actually a constellation of 27 Earth-orbiting satellites (24 in operation and three extras in case one fails). The U.S. military developed and implemented this satellite network as a military navigation system, but soon opened it up to everybody else.
Each of these 3,000- to 4,000-pound solar-powered satellites circles the globe at about 12,000 miles (19,300 km), making two complete rotations every day. The orbits are arranged so that at any time, anywhere on Earth, there are at least four satellites "visible" in the sky.
A GPS receiver's job is to locate four or more of these satellites, figure out the distance to each, and use this information to deduce its own location. This operation is based on a simple mathematical principle called trilateration
Positioning by 3-D trilateration
If we know we are 10 miles from satellite A in the sky, we could be anywhere on the surface of a huge, imaginary sphere with a 10-mile radius. If we also know we are 15 miles from satellite B, we can overlap the first sphere with another, larger sphere. The spheres intersect in a perfect circle. If we know the distance to a third satellite, we get a third sphere, which intersects with this circle at two points.
The Earth itself can act as a fourth sphere -- only one of the two possible points will actually be on the surface of the planet, so you can eliminate the one in space. Receivers generally look to four or more satellites, however, to improve accuracy and provide precise altitude information.
A GPS receiver calculates the distance to GPS satellites by timing a signal's journey from satellite to receiver. As it turns out, this is a fairly elaborate process. At a particular time (let's say midnight), the satellite begins transmitting a long, digital pattern called a pseudo-random code. The receiver begins running the same digital pattern also exactly at midnight. When the satellite's signal reaches the receiver, its transmission of the pattern will lag a bit behind the receiver's playing of the pattern. The length of the delay is equal to the signal's travel time. The receiver multiplies this time by the speed of light to determine how far the signal travelled. Assuming the signal travelled in a straight line, this is the distance from receiver to satellite. In order to make this measurement, the receiver and satellite both need clocks that can be synchronized down to the nanosecond. Every satellite contains an expensive atomic clock, but the receiver itself uses an ordinary quartz clock, which it constantly resets. In a nutshell, the receiver looks at incoming signals from four or more satellites and gauges its own inaccuracy.
When we measure the distance to four located satellites, we can draw four spheres that all intersect at one point. Three spheres will intersect even if our numbers are way off, but four spheres will not intersect at one point if we have measured incorrectly. Since the receiver makes all its distance measurements using its own built-in clock, the distances will all be proportionally incorrect. The receiver can easily calculate the necessary adjustment that will cause the four spheres to intersect at one point. Based on this, it resets its clock to be in sync with the satellite's atomic clock. The receiver does this constantly whenever it's on, which means it is nearly as accurate as the expensive atomic clocks in the satellites. In order for the distance information to be of any use, the receiver also has to know where the satellites actually are. This isn't particularly difficult because the satellites travel in very high and predictable orbits. The GPS receiver simply stores an almanac that tells it where every satellite should be at any given time. Things like the pull of the moon and the sun do change the satellites' orbits very slightly, but the Department of Defence constantly monitors their exact positions and transmits any adjustments to all GPS receivers as part of the satellites' signals
This system works pretty well, but inaccuracies do pop up. For one thing, this method assumes the radio signals will make their way through the atmosphere at a consistent speed (the speed of light). In fact, the Earth's atmosphere slows the electromagnetic energy down somewhat, particularly as it goes through the ionosphere and troposphere. The delay varies depending on where you are on Earth, which means it's difficult to accurately factor this into the distance calculations. Problems can also occur when radio signals bounce off large objects, such as skyscrapers, giving a receiver the impression that a satellite is farther away than it actually is. On top of all that, satellites sometimes just send out bad almanac data, misreporting their own position.
Differential GPS (DGPS) helps correct these errors. The basic idea is to gauge GPS inaccuracy at a stationary receiver station with a known location. Since the DGPS hardware at the station already knows its own position, it can easily calculate its receiver's inaccuracy. The station then broadcasts a radio signal to all DGPS-equipped receivers in the area, providing signal correction information for that area. In general, access to this correction information makes DGPS receivers much more accurate than ordinary receivers.
Thus the most essential function of a GPS receiver is to pick up the transmissions of at least four satellites and combine the information in those transmissions with information in an electronic almanac, all in order to figure out the receiver's position on Earth. Once the receiver makes this calculation, it can tell us the latitude, longitude and altitude (or some similar measurement) of its current position. To make the navigation more user-friendly, most receivers plug this raw data into map files stored in memory. We can use maps stored in the receiver's memory, connect the receiver to a computer that can hold more detailed maps in its memory. A standard GPS receiver will not only place us on a map at any particular location, but will also trace our path across a map as you move. If we leave our receiver on, it can stay in constant communication with GPS satellites to see how our location is changing.
For orientation, an inertial/magnetic tracker rides on a headband above the AR glasses. This device is a combination of miniature gyroscopes and accelerometers that detect head movements along with an electronic compass that establishes the direction of the viewer's gaze in relation to Earth's magnetic field.
4.3 MOBILE COMPUTING POWER
Mobile computing can be accomplished with the help of a wearable computer. A wearable computer is a battery-powered computer system worn on the user's body (on a belt, backpack or vest). It is designed for mobile and predominantly hands-free operations, often incorporating head-mounted displays and speech input.
The wearable computer is more than just a wristwatch or regular eyeglasses: it has the full functionality of a computer system but in addition to being a fully featured computer, it is also inextricably intertwined with the wearer. This is what sets the wearable computer apart from other wearable devices such as wristwatches, regular eyeglasses, wearable radios, etc
Three important features of wearable computers are
The computer runs continuously, and is always ready'' to interact with the user. Unlike a hand-held device, laptop computer, or PDA, it does not need to be opened up and turned on prior to use. The signal flow from human to computer, and computer to human runs continuously to provide a constant user--interface.
Traditional computing paradigms are based on the notion that computing is the primary task. Wearable computing, however, is based on the notion that computing is NOT the primary task. The assumption of wearable computing is that the user will be doing something else at the same time as doing the computing. Thus the computer should serve to augment the intellect, or augment the senses[attachment=2406][attachment=2407][attachment=2406]