Tele-Immersion will enable users at geographically distributed sites to collaborate in real time in a shared, simulated environment as if they were in the same physical room. The idea of Tele-immersion was proposed by Jaron Lanier, the head of Advanced Networks and Services in 1997. National Tele-Immersion Initiative (NTII) was formed to implement this. ? In May 2000, researchers at University of North Carolina (UNC), the University of Pennsylvania and Advanced Networks and Services developed this technology. ? A user sitting in office at UNC was able to see lifelike 3D images of his colleagues hundreds of mile away. ? ? ? ? ? Bank of digital cameras capture participants from a variety of angles. There are about 7-15 camera clusters each contains 4 cameras out of which 3 are black and white and one is color. B&W cameras are for stereo construction. The color camera is for extracting the appearance. Three Camera Views Each camera is synchronized to other cameras in the same bus. Each camera unit is connected to a workstation. Machines in the workstation are dual boot with Linux and Windows 2000. The scene is illuminated by imperceptibly structured light. 4 microphones and 4 speakers are used for sound. Each set of images taken at a given instant is sorted into subsets of overlapping trios of images. From each trio of images a disparity map is calculated, reflecting the degree of variation among the images at all points in the visual field. ? Disparities are analyzed to yield depth. ? All depth maps are combined into a single view point sculptural model of the scene at a particular moment ? This Reconstruction uses Trinocular Stereo Algorithm. ? After constructing the disparity map, apply median filtering to remove a few outliers in the disparity map. ? Then the median filtered disparity map is combined with a registered color texture to reconstruct a 3D point cloud. ? Each pixel have 5 bytes out of which 2 bytes for depth information and 3 for color of the pixel. Why?... If the background is stationary, no need to reconstruct at each time frame. Reconstruct the background scene in advance of the session and transmit it once to the remote sites. System, which generates 640Ãƒâ€”480 pixel depth color maps from 10 camera clusters in a rate of 15 frames per second (fps), requires a 220mbps bandwidth. Around 1.2gbps is necessary for high quality effects!! ? 3D point cloud is skeletonized ? Skeletonized data and the residuals are send to the destination. and decompression are done using Modern Driven compression algorithm. ? Compression (a) (b) © (d) (e) Reconstructions from the compressed data and their differences with the uncompressed data. (a) Uncompressed data. (b) Compressed with 50% residuals. © Difference between (a) and (b). (d) Compressed with 100% residuals. (e) Difference between (a) and (d). Color Reduction scheme 3 byte color information is reduced to 1 byte. A color map is send along with the data. 1 byte is for representing color index. ? JPEG scheme Change video into JPEG form. Motion JPEG uses JPEG still image compression on each frame separately. Sacrifice quality ? Visual quality before color compression ? Visual quality after color compression Telecubicle : Consists of two stereo immersive wall surfaces and a stereo immersive desk surface. These three display surfaces join to form a corner desk unit. Walls appear as windows to other usersâ„¢ environment and desk forms a virtual conference table. CAVE 10 foot-cubed room that is projected with stereoscopic images creating illusion that objects appear to co-exist with the user in the room ImmersaDesk Small, drafting table like system capable of projecting stereoscopic images. ? The software for implementing Trinocular Stereo Algorithm and Modern Driven Compression algorithm is CAVERNsoft. ? CAVERNsoft is a C++ toolkit. ? Work across SGI, Linux, Solaris and Win32 platforms. ? Next Generation Internet initiative to dramatically advance the speed and quality of the Internetâ„¢s infrastructure and thus the ability to evolve new applications. ? Provide high bandwidth. ? A mark of around 8.8 gigabits per seconds was set on April 26, 2006 . ? Remote medical diagnosis and therapy. ? Distributed decision making. ? Remote and distributed education. ? Distributed artistic performance and interaction. ? Virtual travel and tourism. ? Automobile Engineering. ? Application continuesÂ¦ No need to travel long distance in order to conduct business meetings. A geographically distanced surgeon could be teleimmersed into an operation theater. This could be lifesaving! Scene from original surgery Captured scene rendered from reconstructed 3D geometry with annotations and trainee's hands mimicing the surgeon Automobile designers from different continents could design new generation vehicles. ? Dancers from different countries could train together, dance together. ? Dance with yourself! Quality of transmitting multimedia and TeleImmersion data streams over the Internet is affected by high packet loss rates. Solution: FEC/UDP. Expensive. High bandwidth requirement. Tele-Immersion provides natural interaction to the remote world. In the tele-immersive environment, computers recognize the presence and movements of individuals and objects and then permit them to be projected in realistic, multiple, geographically distributed immersive environments where individuals can interact with each other and with computer generated models.