Avi Bar-Zeev

An optical see-through head-mounted display device includes a see-through lens which combines an augmented reality image with light from a real-world scene, while an opacity filter is used to selectively block portions of the real-world scene so that the augmented reality image appears more distinctly. The opacity filter can be a see-through LCD panel, for instance, where each pixel of the LCD panel can be selectively controlled to be transmissive or opaque, based on a size, shape and position… Show more

An optical see-through head-mounted display device includes a see-through lens which combines an augmented reality image with light from a real-world scene, while an opacity filter is used to selectively block portions of the real-world scene so that the augmented reality image appears more distinctly. The opacity filter can be a see-through LCD panel, for instance, where each pixel of the LCD panel can be selectively controlled to be transmissive or opaque, based on a size, shape and position of the augmented reality image. Eye tracking can be used to adjust the position of the augmented reality image and the opaque pixels. Peripheral regions of the opacity filter, which are not behind the augmented reality image, can be activated to provide a peripheral cue or a representation of the augmented reality image. In another aspect, opaque pixels are provided at a time when an augmented reality image is not present. Show less

Described herein is a telepresence system where a real-time a virtual hologram of a user is displayed at a remote display screen and is rendered from a vantage point that is different than the vantage point from which images of the user are captured via a video camera. The virtual hologram is based at least in part upon data acquired from a sensor unit at the location of the user. The sensor unit includes a color video camera that captures 2-D images of the user including surface features of… Show more

Described herein is a telepresence system where a real-time a virtual hologram of a user is displayed at a remote display screen and is rendered from a vantage point that is different than the vantage point from which images of the user are captured via a video camera. The virtual hologram is based at least in part upon data acquired from a sensor unit at the location of the user. The sensor unit includes a color video camera that captures 2-D images of the user including surface features of the user. The sensor unit also includes a depth sensor that captures 3-D geometry data indicative of the relative position of surfaces on the user in 3-D space. The virtual hologram is rendered to orientate the gaze of the eyes of the virtual hologram towards the eyes of a second user viewing the remote display screen. Show less

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The processing unit and hub may collaboratively determine a map of the mixed reality environment. Further, state data may be extrapolated to predict a field of view for a user in the future at a time when the mixed reality is to be… Show more

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The processing unit and hub may collaboratively determine a map of the mixed reality environment. Further, state data may be extrapolated to predict a field of view for a user in the future at a time when the mixed reality is to be displayed to the user. This extrapolation can remove latency from the system.
Show less

A see-through head-mounted display (HMD) device, e.g., in the form of augmented reality glasses, allows a user to view a video display device and an associated augmented reality image. In one approach, the augmented reality image is aligned with edges of the video display device to provide a larger, augmented viewing region. The HMD can include a camera which identifies the edges. The augmented reality image can be synchronized in time with content of the video display device. In another… Show more

A see-through head-mounted display (HMD) device, e.g., in the form of augmented reality glasses, allows a user to view a video display device and an associated augmented reality image. In one approach, the augmented reality image is aligned with edges of the video display device to provide a larger, augmented viewing region. The HMD can include a camera which identifies the edges. The augmented reality image can be synchronized in time with content of the video display device. In another approach, the augmented reality image video provides a virtual audience which accompanies a user in watching the video display device. In another approach, the augmented reality image includes a 3-D which appears to emerge from the video display device, and which is rendered from a perspective of a user's location. In another approach, the augmented reality image can be rendered on a vertical or horizontal surface in a static location.
Show less

The claimed subject matter relates to an architecture that can employ vision-monitoring techniques to enhance an experience associated with elements of a local environment. In particular, the architecture can establish gaze- or eye-tracking attributes in connection with a user. In addition, a location and a head or face-based perspective of the user can also be obtained. By aggregating this information, the architecture can identify a current field of view of the user, and then map that field… Show more

The claimed subject matter relates to an architecture that can employ vision-monitoring techniques to enhance an experience associated with elements of a local environment. In particular, the architecture can establish gaze- or eye-tracking attributes in connection with a user. In addition, a location and a head or face-based perspective of the user can also be obtained. By aggregating this information, the architecture can identify a current field of view of the user, and then map that field of view to a modeled view in connection with a geospatial model of the environment. In addition, the architecture can select additional content that relates to an entity in the view or a modeled entity in the modeled view, and further present the additional content to the user. Show less

Technology is provided for synchronization of clock information between networked devices. One or more of the devices may include one or more applications needed access to data and a common time reference between devices. In one embodiment, the devices have applications utilizing data shared in a network environment with other devices, as well as having a reference to a local clock signal on each device. A device may have a layer of code between the operating system and software applications… Show more

Technology is provided for synchronization of clock information between networked devices. One or more of the devices may include one or more applications needed access to data and a common time reference between devices. In one embodiment, the devices have applications utilizing data shared in a network environment with other devices, as well as having a reference to a local clock signal on each device. A device may have a layer of code between the operating system and software applications that processes the data and maintains a remote clock reference for one or more of the other devices on the network. Show less

Aspects relate to a cognitive agent that performs functions associated with a desired result. The functions performed by cognitive agent supplement other activities performed at a same time. In such a manner, the cognitive agent can function as a surrogate for a user. A performed activity can trigger implementation of another activity that is an extension of the performed activity. Cognitive agent can perform functions that can be represented as an avatar. Further, cognitive agent can be… Show more

Aspects relate to a cognitive agent that performs functions associated with a desired result. The functions performed by cognitive agent supplement other activities performed at a same time. In such a manner, the cognitive agent can function as a surrogate for a user. A performed activity can trigger implementation of another activity that is an extension of the performed activity. Cognitive agent can perform functions that can be represented as an avatar. Further, cognitive agent can be associated with a diagnostics component that evaluates an operating condition. Based on the operating condition cognitive agent can implement automatic actions associated with mitigating failures and/or prolonging the life of machinery. Show less

Aggregated data associated with a geographic area may be obtained. The aggregated data may be dynamically distilled to obtain a presentation set of data based on a dynamically changing geographic location associated with the geographic area. A transmission of a stream of personalized audio data that is based on the presentation set of data may be initiated, via a device processor, to a receiving device associated with a user that is associated with the dynamically changing geographic location.

A system and method for analyzing, summarizing, and transmitting life experiences captured using a life recorder is described. A life recorder is a recording device that continuously captures life experiences, including unanticipated life experiences, in video and/or audio recordings. In some embodiments, the video and/or audio recordings generated by a life recorder are automatically summarized, indexed, and stored for future use. By indexing and storing life recordings, a life recorder may… Show more

A system and method for analyzing, summarizing, and transmitting life experiences captured using a life recorder is described. A life recorder is a recording device that continuously captures life experiences, including unanticipated life experiences, in video and/or audio recordings. In some embodiments, the video and/or audio recordings generated by a life recorder are automatically summarized, indexed, and stored for future use. By indexing and storing life recordings, a life recorder may search for and acquire life recordings generated by itself or another life recorder, thereby allowing life experiences to be shared minutes or even years later. In some embodiments, recordings generated by a life recorder may be analyzed in real-time and automatically pushed to one or more target devices. The ability to automatically and instantaneously push life recordings as live feeds to one or more target devices allows friends and family to experience one's life experience in real-time. Show less

Techniques are provided for synchronization of sensor signals between devices. One or more of the devices may collect sensor data. The device may create a sensor signal from the sensor data, which it may make available to other devices upon a publisher/subscriber model. The other devices may subscribe to sensor signals they choose. A device could be a provider or a consumer of the sensor signals. A device may have a layer of code between an operating system and software applications that… Show more

Techniques are provided for synchronization of sensor signals between devices. One or more of the devices may collect sensor data. The device may create a sensor signal from the sensor data, which it may make available to other devices upon a publisher/subscriber model. The other devices may subscribe to sensor signals they choose. A device could be a provider or a consumer of the sensor signals. A device may have a layer of code between an operating system and software applications that processes the data for the applications. The processing may include such actions as synchronizing the data in a sensor signal to a local time clock, predicting future values for data in a sensor signal, and providing data samples for a sensor signal at a frequency that an application requests, among other actions. Show less

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. The user's intent to interact with one or more objects in the scene is determined. An optimized image is generated for the user based on the user's intent. The optimized image is displayed to the user, via the see-through display device. The optimized image visually enhances the appearance of objects that the user intends… Show more

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. The user's intent to interact with one or more objects in the scene is determined. An optimized image is generated for the user based on the user's intent. The optimized image is displayed to the user, via the see-through display device. The optimized image visually enhances the appearance of objects that the user intends to interact with in the scene and diminishes the appearance of objects that the user does not intend to interact with in the scene. The optimized image can visually enhance the appearance of the objects that increase the user's comprehension. The optimized image is displayed to the user, via the see-through display device.
Show less

An augmented reality system provides improved focus of real and virtual objects. A see-through display device includes a variable focus lens a user looks through. A focal region adjustment unit automatically focuses the variable focus lens in a current user focal region. A microdisplay assembly attached to the see-through display device generates a virtual object for display in the user's current focal region by adjusting its focal region. The variable focus lens may also be adjusted to provide… Show more

An augmented reality system provides improved focus of real and virtual objects. A see-through display device includes a variable focus lens a user looks through. A focal region adjustment unit automatically focuses the variable focus lens in a current user focal region. A microdisplay assembly attached to the see-through display device generates a virtual object for display in the user's current focal region by adjusting its focal region. The variable focus lens may also be adjusted to provide one or more zoom features. Visual enhancement of an object may also be provided to improve a user's perception of an object.
Show less

The technology provides an augmented reality display system for displaying a virtual object to be in focus when viewed by a user. In one embodiment, the focal region of the user is tracked, and a virtual object within the user focal region is displayed to appear in the focal region. As the user changes focus between virtual objects, they appear to naturally move in and out of focus as real objects in a physical environment would. The change of focus for the virtual object images is caused by… Show more

The technology provides an augmented reality display system for displaying a virtual object to be in focus when viewed by a user. In one embodiment, the focal region of the user is tracked, and a virtual object within the user focal region is displayed to appear in the focal region. As the user changes focus between virtual objects, they appear to naturally move in and out of focus as real objects in a physical environment would. The change of focus for the virtual object images is caused by changing a focal region of light processing elements in an optical path of a microdisplay assembly of the augmented reality display system. In some embodiments, a range of focal regions are swept through at a sweep rate by adjusting the elements in the optical path of the microdisplay assembly. Show less

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The system creates a volumetric model of a space, segments the model into objects, identifies one or more of the objects including a first object, and displays a virtual image over the first object on a display (of the head mounted display)… Show more

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The system creates a volumetric model of a space, segments the model into objects, identifies one or more of the objects including a first object, and displays a virtual image over the first object on a display (of the head mounted display) that allows actual direct viewing of at least a portion of the space through the display. Show less

Techniques are provided for providing 3D audio, which may be used in augmented reality. A 3D audio signal may be generated based on sensor data collected from the actual room in which the listener is located and the actual position of the listener in the room. The 3D audio signal may include a number of components that are determined based on the collected sensor data and the listener's location. For example, a number of (virtual) sound paths between a virtual sound source and the listener may… Show more

Techniques are provided for providing 3D audio, which may be used in augmented reality. A 3D audio signal may be generated based on sensor data collected from the actual room in which the listener is located and the actual position of the listener in the room. The 3D audio signal may include a number of components that are determined based on the collected sensor data and the listener's location. For example, a number of (virtual) sound paths between a virtual sound source and the listener may be determined The sensor data may be used to estimate materials in the room, such that the affect that those materials would have on sound as it travels along the paths can be determined In some embodiments, sensor data may be used to collect physical characteristics of the listener such that a suitable HRTF may be determined from a library of HRTFs.
Show less

This disclosure describes an unmanned aerial vehicle (“UAV”) configured to autonomously deliver items of inventory to various destinations. The UAV may receive inventory information and a destination location and autonomously retrieve the inventory from a location within a materials handling facility, compute a route from the materials handling facility to a destination and travel to the destination to deliver the inventory.

Aggregated data associated with a geographic area may be obtained. The aggregated data may be dynamically distilled to obtain a presentation set of data based on a dynamically changing geographic location associated with the geographic area. A transmission of a stream of personalized audio data that is based on the presentation set of data may be initiated, via a device processor, to a receiving device associated with a user that is associated with the dynamically changing geographic location.

Accordingly, the present discussion is directed in one respect; to a method of associating metadata with a digital content item and then controlling the presentation of that item in accordance with metadata-associated controls/constraints. The method may include determining a time-specific portion of a digital content item with which a user desires to associate a metadata item; associating the metadata item with the time-specific portion of the digital content item. During subsequent… Show more

Accordingly, the present discussion is directed in one respect; to a method of associating metadata with a digital content item and then controlling the presentation of that item in accordance with metadata-associated controls/constraints. The method may include determining a time-specific portion of a digital content item with which a user desires to associate a metadata item; associating the metadata item with the time-specific portion of the digital content item. During subsequent consumption of the digital content item by the user or another, the metadata item is presented synchronously in co lection with the time-specific portion of the digital content item, where such presentation is constrained or controlled in response to user-controlled filters implemented through a social network. Show less

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. An optimized image for display relative to the a field of view of a user in a scene is created. The user's head and eye position and movement are tracked to determine a focal region for the user. A portion of the optimized image is coupled to the user's focal region in the current position of the eyes, a next position of… Show more

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. An optimized image for display relative to the a field of view of a user in a scene is created. The user's head and eye position and movement are tracked to determine a focal region for the user. A portion of the optimized image is coupled to the user's focal region in the current position of the eyes, a next position of the head and eyes predicted, and a portion of the optimized image coupled to the user's focal region in the next position.
Show less

A method and system for generating time synchronized data streams based on a viewer's interaction with a multimedia content stream is provided. A viewer's interactions with a multimedia content stream being viewed by the viewer are recorded. The viewer's interactions include comments provided by the viewer, while viewing the multimedia content stream. Comments include text messages, audio messages, video feeds, gestures or facial expressions provided by the viewer. A time synchronized commented… Show more

A method and system for generating time synchronized data streams based on a viewer's interaction with a multimedia content stream is provided. A viewer's interactions with a multimedia content stream being viewed by the viewer are recorded. The viewer's interactions include comments provided by the viewer, while viewing the multimedia content stream. Comments include text messages, audio messages, video feeds, gestures or facial expressions provided by the viewer. A time synchronized commented data stream is generated based on the viewer's interactions. The time synchronized commented data stream includes the viewer's interactions time stamped relative to a virtual start time at which the multimedia content stream is rendered to the viewer. One or more time synchronized data streams are rendered to the viewer, via an audiovisual device, while the viewer views a multimedia content stream. Show less

The claimed subject matter relates to an architecture that can provide for a second-person avatar. The second-person avatar can rely upon a second-person-based perspective such that the avatar is displayed to appear to encompass all or portions of a target user. Accordingly, actions or a configuration of the avatar can serve as a model or demonstration for the user in order to aid the user in accomplishing a particular task. Updates to avatar activity or configuration can be provided by a… Show more

The claimed subject matter relates to an architecture that can provide for a second-person avatar. The second-person avatar can rely upon a second-person-based perspective such that the avatar is displayed to appear to encompass all or portions of a target user. Accordingly, actions or a configuration of the avatar can serve as a model or demonstration for the user in order to aid the user in accomplishing a particular task. Updates to avatar activity or configuration can be provided by a dynamic virtual handbook. The virtual handbook can be constructed based upon a set of instruction associated with accomplishing the desired task and further based upon features or aspects of the user as well as those of the local environment. Show less

Aspects relate to determining an appropriate time to render markers to a user. Markers includes any type of information, including messages, photos, advertisements, and so forth. The appropriate time can include internal factors that relate to the user (e.g., mood, preferences, and so on), personal external factors (e.g., scheduled events, location of the user), and/or non-personal external factors (general information about the surroundings, information about local entities). Non-personal… Show more

Aspects relate to determining an appropriate time to render markers to a user. Markers includes any type of information, including messages, photos, advertisements, and so forth. The appropriate time can include internal factors that relate to the user (e.g., mood, preferences, and so on), personal external factors (e.g., scheduled events, location of the user), and/or non-personal external factors (general information about the surroundings, information about local entities). Non-personal external factors can also include the content of the marker. One or more of the contexts are evaluated to ascertain whether a particular marker should be rendered to the user or whether it would be more appropriate not the render the marker (e.g., user would not be receptive to the marker). If the user is not receptive to the marker, rendering of the marker is disabled and/or delayed until a more appropriate time.
Show less

The claimed subject matter provides a system and/or a method that facilitates displaying virtual content within a virtual environment. A virtual environment can enable at least one user to connect in order to interact with a portion of virtual content. A collection of virtual content can be viewed from a user connected to and being present within the virtual environment. A filter component that can automatically adjust a displayable feature of a portion of the collection of virtual content… Show more

The claimed subject matter provides a system and/or a method that facilitates displaying virtual content within a virtual environment. A virtual environment can enable at least one user to connect in order to interact with a portion of virtual content. A collection of virtual content can be viewed from a user connected to and being present within the virtual environment. A filter component that can automatically adjust a displayable feature of a portion of the collection of virtual content based upon a relevancy to an index, wherein the adjustment of the displayable features provides at least one of an emphasis of the portion of the collection of virtual content or a de-emphasis of the portion of the collection of the virtual content. Show less

An immersive virtual store can be presented to a user that allows the user to navigate through the store in a manner similar to a physical store. The presented virtual store can allow the user to travel along aisles, stop and browse items at a table of a vendor, as well as perform searches for different items. Items of the store can be arranged in a manner that facilitates purchase and feedback of the arrangement can be collected and used in other arrangements.

Various entities might desire to leave markers at various locations in a mapping application for themselves and others. These markers can be provided by an entity physically located near the location associated with the marker and/or at a different location (e.g., entity is at a first location and the marker is associated with a second location). Further, different entities can provide markers associated with a similar geographic area. A user visiting the location (virtually or physically) can… Show more

Various entities might desire to leave markers at various locations in a mapping application for themselves and others. These markers can be provided by an entity physically located near the location associated with the marker and/or at a different location (e.g., entity is at a first location and the marker is associated with a second location). Further, different entities can provide markers associated with a similar geographic area. A user visiting the location (virtually or physically) can review the various markers left by others. In such a manner, the user visiting the location can have further details associated with the location. Different categories of markers can be combined and rendered to the user as a hybrid of markers. Show less

The claimed subject matter relates to an architecture that can encourage ad hoc or impromptu engagements between entities as well as simplify or facilitate planning engagements between those entities, all potentially based upon projected routes or paths of the entities. In particular, the architecture can receive location information associated with an entity and can further employ the location information for constructing or updating a path tree for the entity, with each branch of the path… Show more

The claimed subject matter relates to an architecture that can encourage ad hoc or impromptu engagements between entities as well as simplify or facilitate planning engagements between those entities, all potentially based upon projected routes or paths of the entities. In particular, the architecture can receive location information associated with an entity and can further employ the location information for constructing or updating a path tree for the entity, with each branch of the path tree indicative of a possible or likely future path. Additionally, the architecture can compare the path tree for the entity with a path tree for a disparate entity in order to identify a possible or likely intersection. Moreover, the architecture can generate an intersection notification with details relating to the possible or likely intersection, and provide the intersection notification to one or more entities.
Show less

Aspects relate to providing markers related to a mapping application, wherein the provided markers are selectively modified as a function of the relevancy of the markers to a person that has requested the markers. The relevancy is determined based on one or more indicators included in the marker. The indicators answer questions related to who, what, when, where, and why. Markers can be ranked based on the association between the person that requested the marker and the person that provided the… Show more

Aspects relate to providing markers related to a mapping application, wherein the provided markers are selectively modified as a function of the relevancy of the markers to a person that has requested the markers. The relevancy is determined based on one or more indicators included in the marker. The indicators answer questions related to who, what, when, where, and why. Markers can be ranked based on the association between the person that requested the marker and the person that provided the marker. Further, a copy of an original marker can be modified to conform with the relevancy of the marker to the requestor. Show less

Techniques are provided for synchronization of sensor signals between devices. One or more of the devices may collect sensor data. The device may create a sensor signal from the sensor data, which it may make available to other devices upon a publisher/subscriber model. The other devices may subscribe to sensor signals they choose. A device could be a provider or a consumer of the sensor signals. A device may have a layer of code between an operating system and software applications that… Show more

Techniques are provided for synchronization of sensor signals between devices. One or more of the devices may collect sensor data. The device may create a sensor signal from the sensor data, which it may make available to other devices upon a publisher/subscriber model. The other devices may subscribe to sensor signals they choose. A device could be a provider or a consumer of the sensor signals. A device may have a layer of code between an operating system and software applications that processes the data for the applications. The processing may include such actions as synchronizing the data in a sensor signal to a local time clock, predicting future values for data in a sensor signal, and providing data samples for a sensor signal at a frequency that an application requests, among other actions. Show less