Augmented reality

Not to be confused with Virtual reality or Alternate reality.

Virtual Fixtures – first AR system, U.S. Air Force, Wright-Patterson Air Force Base (1992)

Augmented muney reality (AR) is an interactive experience that combines the real world and

See also: Automotive navigation system

As AR technology has evolved and second and third generation AR devices come to market, the impact of AR in enterprise continues to flourish. In the Harvard Business Review, Magid Abraham and Marco Annunziata discuss how AR devices are now being used to "boost workers' productivity on an array of tasks the first time they're used, even without prior training". They contend that "these technologies increase productivity by making workers more skilled and efficient, and thus have the potential to yield both more economic growth and better jobs".

Broadcast and live events

Weather visualizations were the first application of augmented reality in television. It has now become common in weather casting to display full motion video of images captured in real-time from multiple cameras and other imaging devices. Coupled with 3D graphics symbols and mapped to a common virtual geospatial model, these animated visualizations constitute the first true application of AR to TV.

AR has become common in sports telecasting. Sports and entertainment venues are provided with see-through and overlay augmentation through tracked camera feeds for enhanced viewing by the audience. Examples include the yellow "first down" line seen in television broadcasts of American football games showing the line the offensive team must cross to receive a first down. AR is also used in association with football and other sporting events to show commercial advertisements overlaid onto the view of the playing area. Sections of rugby fields and cricket pitches also display sponsored images. Swimming telecasts often add a line across the lanes to indicate the position of the current record holder as a race proceeds to allow viewers to compare the current race to the best performance. Other examples include hockey puck tracking and annotations of racing car performance and snooker ball trajectories.

AR has been used to enhance concert and theater performances. For example, artists allow listeners to augment their listening experience by adding their performance to that of other bands/groups of users.

Tourism and sightseeing

Travelers may use AR to access real-time informational displays regarding a location, its features, and comments or content provided by previous visitors. Advanced AR applications include simulations of historical events, places, and objects rendered into the landscape.

AR applications linked to geographic locations present location information by audio, announcing features of interest at a particular site as they become visible to the user.

Translation

AR systems such as Word Lens can interpret the foreign text on signs and menus and, in a user's augmented view, re-display the text in the user's language. Spoken words of a foreign language can be translated and displayed in a user's view as printed subtitles.

Music

It has been suggested that augmented reality may be used in new methods of music production, mixing, control and visualization.

In a proof-of-concept project Ian Sterling, an interaction design student at California College of the Arts, and software engineer Swaroop Pal demonstrated a HoloLens app whose primary purpose is to provide a 3D spatial UI for cross-platform devices—the Android Music Player app and Arduino-controlled Fan and Light—and also allow interaction using gaze and gesture control.

Research by members of the CRIStAL at the University of Lille makes use of augmented reality to enrich musical performance. The ControllAR project allows musicians to augment their MIDI control surfaces with the remixed graphical user interfaces of music software. The Rouages project proposes to augment digital musical instruments to reveal their mechanisms to the audience and thus improve the perceived liveness. Reflets is a novel augmented reality display dedicated to musical performances where the audience acts as a 3D display by revealing virtual content on stage, which can also be used for 3D musical interaction and collaboration.

Snapchat

Snapchat users have access to augmented reality in the app through use of camera filters. In September 2017, Snapchat updated its app to include a camera filter that allowed users to render an animated, cartoon version of themselves called "Bitmoji". These animated avatars would be projected in the real world through the camera, and can be photographed or video recorded. In the same month, Snapchat also announced a new feature called "Sky Filters" that will be available on its app. This new feature makes use of augmented reality to alter the look of a picture taken of the sky, much like how users can apply the app's filters to other pictures. Users can choose from sky filters such as starry night, stormy clouds, beautiful sunsets, and rainbow.

Concerns

Reality modifications

In a paper titled "Death by Pokémon GO", researchers at Purdue University's Krannert School of Management claim the game caused "a disproportionate increase in vehicular crashes and associated vehicular damage, personal injuries, and fatalities in the vicinity of locations, called PokéStops, where users can play the game while driving." Using data from one municipality, the paper extrapolates what that might mean nationwide and concluded "the increase in crashes attributable to the introduction of Pokémon GO is 145,632 with an associated increase in the number of injuries of 29,370 and an associated increase in the number of fatalities of 256 over the period of 6 July 2016, through 30 November 2016." The authors extrapolated the cost of those crashes and fatalities at between $2bn and $7.3 billion for the same period. Furthermore, more than one in three surveyed advanced Internet users would like to edit out disturbing elements around them, such as garbage or graffiti. They would like to even modify their surroundings by erasing street signs, billboard ads, and uninteresting shopping windows. So it seems that AR is as much a threat to companies as it is an opportunity. Although, this could be a nightmare to numerous brands that do not manage to capture consumer imaginations it also creates the risk that the wearers of augmented reality glasses may become unaware of surrounding dangers. Consumers want to use augmented reality glasses to change their surroundings into something that reflects their own personal opinions. Around two in five want to change the way their surroundings look and even how people appear to them.

Next, to the possible privacy issues that are described below, overload and over-reliance issues are the biggest danger of AR. For the development of new AR-related products, this implies that the user-interface should follow certain guidelines as not to overload the user with information while also preventing the user from over-relying on the AR system such that important cues from the environment are missed. This is called the virtually-augmented key. Once the key is ignored, people might not desire the real world anymore.

Privacy concerns

The concept of modern augmented reality depends on the ability of the device to record and analyze the environment in real time. Because of this, there are potential legal concerns over privacy. While the First Amendment to the United States Constitution allows for such recording in the name of public interest, the constant recording of an AR device makes it difficult to do so without also recording outside of the public domain. Legal complications would be found in areas where a right to a certain amount of privacy is expected or where copyrighted media are displayed.

In terms of individual privacy, there exists the ease of access to information that one should not readily possess about a given person. This is accomplished through facial recognition technology. Assuming that AR automatically passes information about persons that the user sees, there could be anything seen from social media, criminal record, and marital status.

The Code of Ethics on Human Augmentation, which was originally introduced by Steve Mann in 2004 and further refined with Ray Kurzweil and Marvin Minsky in 2013, was ultimately ratified at the virtual reality Toronto conference on 25 June 2017.

Property law

The interaction of location-bound augmented reality with property law is largely undefined. Several models have been analysed for how this interaction may be resolved in a common law context: an extension of real property rights to also cover augmentations on or near the property with a strong notion of trespassing, forbidding augmentations unless allowed by the owner; an 'open range' system, where augmentations are allowed unless forbidden by the owner; and a 'freedom to roam' system, where real property owners have no control over non-disruptive augmentations.

One issue experienced during the Pokémon Go craze was the game's players disturbing owners of private property while visiting nearby location-bound augmentations, which may have been on the properties or the properties may have been en route. The terms of service of Pokémon Go explicitly disclaim responsibility for players' actions, which may limit (but may not totally extinguish) the liability of its producer, Niantic, in the event of a player trespassing while playing the game: by Niantic's argument, the player is the one committing the trespass, while Niantic has merely engaged in permissible free speech. A theory advanced in lawsuits brought against Niantic is that their placement of game elements in places that will lead to trespass or an exceptionally large flux of visitors can constitute nuisance, despite each individual trespass or visit only being tenuously caused by Niantic.

Another claim raised against Niantic is that the placement of profitable game elements on land without permission of the land's owners is unjust enrichment. More hypothetically, a property may be augmented with advertising or disagreeable content against its owner's wishes. Under American law, these situations are unlikely to be seen as a violation of real property rights by courts without an expansion of those rights to include augmented reality (similarly to how English common law came to recognise air rights).

An article in the Michigan Telecommunications and Technology Law Review argues that there are three bases for this extension, starting with various understanding of property. The personality theory of property, outlined by Margaret Radin, is claimed to support extending property rights due to the intimate connection between personhood and ownership of property; however, her viewpoint is not universally shared by legal theorists. Under the utilitarian theory of property, the benefits from avoiding the harms to real property owners caused by augmentations and the tragedy of the commons, and the reduction in transaction costs by making discovery of ownership easy, were assessed as justifying recognising real property rights as covering location-bound augmentations, though there does remain the possibility of a tragedy of the anticommons from having to negotiate with property owners slowing innovation. Finally, following the 'property as the law of things' identification as supported by Thomas Merrill and Henry E Smith, location-based augmentation is naturally identified as a 'thing', and, while the non-rivalrous and ephemeral nature of digital objects presents difficulties to the excludeability prong of the definition, the article argues that this is not insurmountable.

Some attempts at legislative regulation have been made in the United States. Milwaukee County, Wisconsin attempted to regulate augmented reality games played in its parks, requiring prior issuance of a permit, but this was criticised on free speech grounds by a federal judge; and Illinois considered mandating a notice and take down procedure for location-bound augmentations.

An article for the Iowa Law Review observed that dealing with many local permitting processes would be arduous for a large-scale service, and, while the proposed Illinois mechanism could be made workable, it was reactive and required property owners to potentially continually deal with new augmented reality services; instead, a national-level geofencing registry, analogous to a do-not-call list, was proposed as the most desirable form of regulation to efficiently balance the interests of both providers of augmented reality services and real property owners. An article in the Vanderbilt Journal of Entertainment and Technology Law, however, analyses a monolithic do-not-locate registry as an insufficiently flexible tool, either permitting unwanted augmentations or foreclosing useful applications of augmented reality. Instead, it argues that an 'open range' model, where augmentations are permitted by default but property owners may restrict them on a case-by-case basis (and with noncompliance treated as a form of trespass), will produce the socially-best outcome.

Notable researchers

• Ivan Sutherland invented the first VR head-mounted display at Harvard University.
• Steve Mann formulated an earlier concept of mediated reality in the 1970s and 1980s, using cameras, processors, and display systems to modify visual reality to help people see better (dynamic range management), building computerized welding helmets, as well as "augmediated reality" vision systems for use in everyday life. He is also an adviser to Meta.
• Ronald Azuma is a scientist and author of works on AR.
• Dieter Schmalstieg and Daniel Wagner developed a marker tracking systems for mobile phones and PDAs in 2009.
• Jeri Ellsworth headed a research effort for Valve on augmented reality (AR), later taking that research to her own start-up CastAR. The company, founded in 2013, eventually shuttered. Later, she created another start-up based on the same technology called Tilt Five; another AR start-up formed by her with the purpose of creating a device for digital board games.

In media

The futuristic short film Sight features contact lens-like augmented reality devices.

See also

• ARTag – Fiduciary markerPages displaying wikidata descriptions as a fallback
• Browser extension, also known as Augmented browsing – Program that extends the functionality of a web browser
• Augmented reality-based testing – type of testingPages displaying wikidata descriptions as a fallback
• Augmented web – Web technologyPages displaying short descriptions of redirect targets
• Automotive head-up display – Advanced driver assistance system
• Bionic contact lens – Proposed device to display information
• Computer-mediated reality – Ability to manipulate one's perception of reality through the use of a computer
• Cyborg – Being with both organic and biomechatronic body parts
• Head-mounted display – Type of display device
• Holography – Recording to reproduce a three-dimensional light field
• List of augmented reality software
• Location-based service – General class of computer program-level services that use location data to control features
• Mixed reality – Merging of real and virtual worlds to produce new environments
• Optical head-mounted display – Type of wearable device
• Simulated reality – Concept of a false version of reality
• Smartglasses – Wearable computers glasses
• Virtual reality – Computer-simulated experience
• Visuo-haptic mixed reality
• Wearable computer – Small computing device worn on the body

Sources

• Conroy, Declan T (2017). "Property Rights in Augmented Reality". Michigan Telecommunications and Technology Law Review. 24 (1).
• McClure, William (1 November 2017). "When the Virtual and Real Worlds Collide: Beginning to Address the Clash Between Real Property Rights and Augmented Reality Location-Based Technologies Through a Federal Do-Not-Locate Registry". Iowa Law Review. 103 (1). SSRN 3906369.
• Mallick, Samuel (2020). "Augmenting Property Law: Applying the Right to Exclude in the Augmented Reality Universe". Vanderbilt Journal of Entertainment and Technology Law. 19 (4).

External links

Media related to Augmented reality at Wikimedia Commons