David Swapp

Social VR (SVR) systems are VR systems with a common subset of features facilitating unstructured social interaction. In the real world, social situations have many purposes, each with a different set of requirements, and roles its participants take - creator, moderator, performer, visitor, etc. Yet, common SVR systems typically offer only a single client to users. Even if there are versions for different platforms, there is a one-size-fits-all approach to the user experience. Consequently users need to employ workarounds or build their own functionality to support specific roles, where this is possible at all. We argue that platforms need to develop more open frameworks that support different processes and user interactions. One way to do this is through using appropriate web standards and an open messaging system in order to allow distributed clients that can leverage the strongest features of heterogeneous computing platforms. Supporting asymmetrical capabilities greatly increases the scope of supported virtual social interactions and potential use cases of SVR. We take a qualitative experimental approach to exploring cross platform support in this way, from a designers perspective. We use the open-source SDK Ubiq, and create a library that allows building Ubiq Peers using web standards and thus clients that can operate solely in a web browser or certain Javascript environments. We validate our approach by demonstrating six proof of concept demonstrators that would be difficult or impossible to achieve in most other SVR systems, and report on what we encountered for the benefit of other SVR designers.

Image-warping , a per-pixel deformation of one image into another, is an essential component in immersive visual experiences such as virtual reality or augmented reality. The primary issue with image warping is disocclusions, where occluded (and hence unknown) parts of the input image would be required to compose the output image. We introduce a new image warping method, Metameric image inpainting - an approach for hole-filling in real-time with foundations in human visual perception. Our method estimates image feature statistics of disoccluded regions from their neighbours. These statistics are inpainted and used to synthesise visuals in real-time that are less noticeable to study participants, particularly in peripheral vision. Our method offers speed improvements over the standard structured image inpainting methods while improving realism over colour-based inpainting such as push-pull. Hence, our work paves the way towards future applications such as depth image-based rendering, 6-DoF 360 rendering, and remote render-streaming.

Developing mixed-reality (MR) experiments is a challenge as there is a wide variety of functionality to support. This challenge is exacerbated if the MR experiment is multi-user or if the experiment needs to be run out of the lab. We present Ubiq-Exp - a set of tools that provide a variety of functionality to facilitate distributed and remote MR experiments. We motivate our design and tools from recent practice in the field and a desire to build experiments that are easier to reproduce. Key features are the ability to support supervised and unsupervised experiments, and a variety of tools for the experimenter to facilitate operation and documentation of the experimental sessions. We illustrate the potential of the tools through three small-scale pilot experiments. Our tools and pilot experiments are released under a permissive open-source license to enable developers to appropriate and develop them further for their own needs.

Computer-Generated Holography (CGH) offers the potential for genuine, high-quality three-dimensional visuals. However, fulfilling this potential remains a practical challenge due to computational complexity and visual quality issues. We propose a new CGH method that exploits gaze-contingency and perceptual graphics to accelerate the development of practical holographic display systems. Firstly, our method infers the user's focal depth and generates images only at their focus plane without using any moving parts. Second, the images displayed are metamers; in the user's peripheral vision, they need only be statistically correct and blend with the fovea seamlessly. Unlike previous methods, our method prioritises and improves foveal visual quality without causing perceptually visible distortions at the periphery. To enable our method, we introduce a novel metameric loss function that robustly compares the statistics of two given images for a known gaze location. In parallel, we implement a model representing the relation between holograms and their image reconstructions. We couple our differentiable loss function and model to metameric varifocal holograms using a stochastic gradient descent solver. We evaluate our method with an actual proof-of-concept holographic display, and we show that our CGH method leads to practical and perceptually three-dimensional image reconstructions.

Virtual Reality (VR) is maturing as a technology. Now that mainstream head-mounted displays (HMDs) are consumer-affordable, the space of application development has begun in earnest. Some of this development transitions existing applications (e.g. computer games) to work with a 3D tracked interface while others explore completely novel and innovative uses of VR. The idea of using VR in architectural practice has a long history. As a tool with the potential to allow 3D visualisation at 1:1scale, the use-case for architectural visualisation has seemed natural and obvious since the early days of the technology. However, the realisation of this idea was not initially straightforward. In 2000 UCL built a CAVE-like VR projection theatre – this is a 3m x 3m room where three of the four walls and the floor are stereo displays, viewed through tracked stereo glasses allowing perspective-correct stereo views. This was driven by a state-of-the-art SGI computer, many times more powerful than any standard PC (and about 20 times the size). However, despite this vast graphics processing power, most architectural models, could not easily be adapted to this new technology. These models had been designed for accurate renderings of detailed geometry. Twenty minutes of processing with standard computer graphics applications on a desktop PC could produce a beautiful rendering of a view into this model, but VR demands real-time frame rates (ideally at least 60 frames per second) and the models were simply too large and detailed for this. These tensions between designs for single viewpoint renderings and designs for real-time rendering are now better understood, and advances in both graphics hardware and software have improved this situation. However recent trends in consumer VR towards standalone headsets mean that simulations are now driven by the same graphics processors that drive the mobile devices in our pockets. Aside from these technical hurdles, the cost has been the main contributing factor to the relatively slow uptake of VR as a tool for exploring design, but now that we have affordable devices available, what are the factors that still hinder progress?

Distributed virtual environments (DVEs) are challenging to create as the goals of consistency and responsiveness become contradictory under increasing latency. DVEs have been considered as both distributed transactional databases and force-reflection systems. Both are good approaches, but they do have drawbacks. Transactional systems do not support Level 3 (L3) collaboration: manipulating the same degree-of-freedom at the same time. Force-reflection requires a client-server architecture and stabilisation techniques. With Consensus Based Networking (CBN), we suggest DVEs be considered as a distributed data-fusion problem. Many simulations run in parallel and exchange their states, with remote states integrated with continous authority. Over time the exchanges average out local differences, performing a distribued-average of a consistent, shared state. CBN aims to build simulations that are highly responsive, but consistent enough for use cases such as the piano-movers problem. CBN’s support for heterogeneous nodes can transparently couple different input methods, avoid the requirement of determinism, and provide more options for personal control over the shared experience. Our work is early, however we demonstrate many successes, including L3 collaboration in room-scale VR, 1000’s of interacting objects, complex configurations such as stacking, and transparent coupling of haptic devices. These have been shown before, but each with a different technique; CBN supports them all within a single, unified system.

The Dexmo glove is a haptic exoskeleton that provides kinesthetic feedback in virtual reality. Unlike many other gloves based on string-pulleys, the Dexmo uses a free-hinged link-bar to transfer forces from a crank to the fingertips. It also uses an admittance-based controller parameterized by position, as opposed to an impedance-based controller parameterized by force. When setting the controller's target position, developers must use its native angular coordinate system. The Dexmo has a number of uninstrumented degrees of freedom. Mature forward models can reliably predict the hand pose, even with these unknowns. When it comes to computing angular controller parameters from a target pose in Cartesian space however, things become more difficult. Complex models that provide attractive visuals from a small number of sensors can be non-trivial or even impossible to invert. In this letter, we suggest side-stepping this issue. We sample the forward model in order to build a lookup table. This is embedded in three-dimensional space as a curve, on which traditional queries against world geometry can be performed. Controller parameters are stored as attributes of the sample points. To compute the driver parameters for a target position, the application constrains the position to the geometry, and interpolates them. This technique is generalizable, stable, simple, and fast. We validate our approach by implementing it in Unity 2017.3 and integrating it with a Dexmo glove.

Subcortical reflexive motor responses are under continuous cortical control to produce the most effective behaviour. For example, the excitability of brainstem circuitry subserving the defensive hand-blink reflex (HBR), a response elicited by intense somatosensory stimuli to the wrist, depends on a number of properties of the eliciting stimulus. These include face-hand proximity, which has allowed the description of an HBR response field around the face (commonly referred to as a defensive peripersonal space, DPPS), as well as stimulus movement and probability of stimulus occurrence. However, the effect of stimulus-independent movements of objects in the environment has not been explored. Here we used virtual reality to test whether and how the HBR-derived DPPS is affected by the presence and movement of threatening objects in the environment. In two experiments conducted on 40 healthy volunteers, we observed that threatening arrows flying towards the participant result in DPPS expansion, an effect directionally-tuned towards the source of the arrows. These results indicate that the excitability of brainstem circuitry subserving the HBR is continuously adjusted, taking into account the movement of environmental objects. Such adjustments fit in a framework where the relevance of defensive actions is continually evaluated, to maximise their survival value.

In Milgram’s seminal obedience studies, participants’ behaviour has traditionally been explained as a demonstration of people’s tendency to enter into an ‘agentic state’ when in the presence of an authority figure: they attend only to the demands of that authority and are insensitive to the plight of their victims. There have been many criticisms of this view, but most rely on either indirect or anecdotal evidence. In this study, participants (n = 40) are taken through a Virtual Reality simulation of the Milgram paradigm. Compared to control participants (n = 20) who are not taken through the simulation, those in the experimental conditions are found to attempt to help the Learner more by putting greater emphasis on the correct word over the incorrect words. We also manipulate the extent to which participants identify with the science of the study and show that high identifiers both give more help, are less stressed, and are more hesitant to press the shock button than low identifiers. We conclude that these findings constitute a refutation of the ‘agentic state’ approach to obedience. Instead, we discuss implications for the alternative approaches such as ‘engaged followership’ which suggests that obedience is a function of relative identification with the science and with the victim in the study. Finally, we discuss the value of Virtual Reality as a technique for investigating hard-to-study psychological phenomena.

The art of picking up signs that a child may be suffering from abuse at home is one of those skills that cannot easily be taught, given its dependence on a range of non-cognitive abilities. It is also difficult to study, given the number of factors that may interfere with this skill in a real-life, professional setting. An immersive virtual reality environment provides a way round these difficulties. In this study, we recruited 64 general practitioners, with different levels of experience. Would this level of experience have any impact on General Practitioners’ (GPs) ability to pick up child-safeguarding concerns? Would more experienced GPs find it easier to pick up subtle (rather than obvious) signs of child-safeguarding concerns? Our main measurement was the quality of the note left by the GP at the end of the virtual consultation: we had a panel of 10 (all experienced in safeguarding) rate the note according to the extent to which they were able to identify and take the necessary steps required in relation to the child safeguarding concerns. While the level of professional experience was not shown to make any difference to a GP’s ability to pick up those concerns, the parent’s level of aggressive behavior towards the child did. We also manipulated the level of cognitive load (reflected in a complex presentation of the patient’s medical condition): while cognitive load did have some impact upon GPs in the `obvious cue’ condition (parent behaving particularly aggressively), this effect fell short of significance. Furthermore, our results also suggest that GPs who are less stressed, less neurotic, more agreeable and extroverted tend to be better at raising potential child abuse issues in their notes. These results not only point at the considerable potential of virtual reality as a training tool. They also highlight fruitful avenues for further research, as well as potential strategies to support GP’s in their dealing with highly sensitive, emotionally charged situations.