A combination of computer vision and projector-based illumination opens the possibility for a new type of augmented reality:  selectively illuminating the scene to improve or manipulate how the reality itself, rather than its display, appears to a human. One such example is the Smart Headlight being developed at Carnegie Mellon University’s Robotics Institute. The project team has been working on a new set of capabilities for the headlight, such as making rain drops and snowflakes disappear, allowing for the high beams to always be on without glare, and enhancing the appearance of objects of interest. This talk will present the idea, approach, and current status of the Smart Headlight Project.

Many inter-cardiac interventions are performed either via open-heart surgery, or using minimally invasive approaches, where instrumentation is introduced into the cardiac chambers via the vascular system or heart wall. While many of the latter procedures are often employed under x-ray guidance, for some of these x-ray imaging is not appropriate, and ultrasound is the preferred intra-operative imaging modality. Two such procedures involves the repair of a mitral-valve leaflet, and the replacement of aortic valves. Both employ instruments introduced into the heart via the apex. For the mitral procedure, the standard of care for this procedure employs a 3D Trans-esophageal echo (TEE) probe as guidance, but using primarily its bi-plane mode, with full 3D only being used sporadically. In spite of the clinical success of this procedure, many problems are encountered during the navigation of the instrument to the site of the therapy. To overcome these difficulties, we have developed a guidance platform that tracks the US probe and instrument, and augments the US mages with virtual   elements representing the instrument and target, to optimise the navigation process. Results of using this approach on animal studies have demonstrated increased performance in multiple metrics, including total tool distance from ideal pathway, total navigation time, and total tool path lengths, by factors of 3,4,and 5 respectively, as well as a 40 fold reduction in the number of times an instrument intruded into potentially unsafe zones in the heart. 

The aortic valve procedure primarily uses X-ray fluoroscopy guidance, but this suffers from the problem of high radiations dose, poor target visibility and potential kidney damage as a result of x-ray contrast administration. To overcome these limitations, we have adopted similar technology to that used for the mitral valve problem, to develop an ultrasound-only solution, again augmented with virtual models of instruments and key targets to guide aortic valve replacement procedures. Preliminary results of this approach on cardiac phantoms indicate that the US-only approach may be as accurate as the standard fluoroscopy-guided technique.

As an original pioneer of Virtual Reality, Tom Furness, founder of the international network of HIT Labs, presents both the legacy and future of Mixed and Augmented Reality through innovations that unlock and link minds.  Inventing some of the original inspirations that led to the Google Glass and Oculus Rift, his keynote will inspire making creative leaps to realize the promise of research.  Since the 1960’s, his career has been about how to achieve radical innovation through helping others to “see anew.”  In inspiring a paradigm shift in thinking about the virtuality continuum, his work and global network of labs has shifted the thinking from being about function to purpose; perception to performance; from capabilities to possibilities.

“The virtuality continuum for me has been really about really bridging between the wonders of the real world and the infinite possibilities of the human imagination."

As Mixed and Augmented Reality innovation begins to transform everyday life, Dr. Furness presents the future role of transdisciplinary research and curriculum in melting the boundaries between Science, Technology, Media, Art, Humanities and Design to teach the next generation of innovators.  He shares his future venture of the Virtual World Society that is creating a collaborative initiative of experiential learning, using virtual, mixed and augmented reality as a platform to empower youth to envision and create a better world for tomorrow.

INTERESTING QUOTE:

The real voyage of discovery consists not in seeking new landscapes but in having new eyes. - Marcel Proust (1871-1922)