The Brazilian-born inventor gets recognition for his groundbreaking work with the Microsoft Hololens.
The European Patent Office (EPO) announced that it has nominated U.S.-based inventor Alex Kipman as a finalist for the European Inventor Award 2018.
“Alex Kipman’s invention is a vision for the future of computing with the potential to change how we connect and collaborate,” said EPO President Benoît Battistelli. “His ideas are enhancing the real world with digital information that helps people achieve higher levels of efficiency, and creating new opportunities in fields such as engineering, communications, and healthcare.”
Kipman started his professional career at Microsoft’s headquarters in Redmond, Washington, right after graduating from the Rochester Institute of Technology in 2001. In a career spanning more than 15 years as a developer for the company, he has been named as the primary inventor on more than 150 U.S. and Asian patents, as well as two granted European patents and more than a dozen European patent applications. His current position as Technical Fellow of the Windows and Devices Group enables him to maintain a hands-on approach to building the HoloLens and related Windows Mixed Reality software platform.
The HoloLens incorporates several patent-protected Microsoft inventions, including Kipman’s motion sensors initially brought to market for the Xbox Kinect video game controller. The software for the HoloLens is developed in the Windows 10 operating system in the Windows Mixed Reality software environment. Current applications of the device include telecommunications (e.g. for enhanced video calls), a 3D virtual tourism application, and a suite of mixed-reality games, as well as industrial uses for metering and quality control, and interactive digital human anatomy. A host of promising new applications are being developed. In 2017, a Spanish team of surgeons used the mixed-reality headset successfully while operating on a patient with a malignant muscular tumour. During the procedure, the surgeons were able to view live diagnostic imaging from MRI and radiography data feeds, without taking their eyes off the patient.
Thanks to built-in machine learning and AI-algorithms, the HoloLens is contextually aware of its surroundings and able to interpret user commands such as “move that chair.” It observes its surroundings via an inertial measurement unit (a device that measures linear and angular motion), a depth-sensing camera, a 2.4-megapixel video camera, four microphones and an ambient light sensor — so it “knows” which chair the wearer is currently seeing. It collects several terabytes of data every second, piped in via sensors and then computed.
“Current-generation mixed-reality glasses include “smart” ski goggles that project data such as altitude, temperature and pulse into the wearer’s field of vision. But these devices display only two-dimensional text and graphics, while requiring a connection to a smartphone for computing power,” explains Battistelli. “The HoloLens is a stand-alone device, independent from external PCs, tablets or smartphones. It contains all necessary computing power in its custom-built processors. These include a CPU (central processing unit), GPU (graphics processing unit) and custom HPU (holographic processing unit), an industry first.”
These processors collect several terabytes of data every second as the HoloLens reads its environment. Data arrives through an inertial measurement unit (IMU), a depth-sensing camera, a 2.4-megapixel video camera, four microphones and an ambient light sensor.
As the name suggests, the HoloLens owes its innovative edge to the waveguide lenses that “trick” the human brain into interpreting photons as solid, three-dimensional things — hence holograms. They are computed on the GPU and displayed through the device’s “light engines”, two tiny liquid-crystal-on-silicon projectors which beam light onto the two lenses.
The image is adjusted between the two lenses for clarity and depth, then blended with the real world. As a result, holographic images appear to exist in front of the wearer — from a few centimeters to many meters away — with true depth of field.
Once ready for mass adoption, the HoloLens could lead to a fundamental shift, as mixed-reality devices replace today’s smartphones as the main communications devices. According to experts at market researcher Gartner, 30% of web browsing sessions could be done without a screen by 2020. At that point, 50% of all search queries could originate, not from keyboards or touchscreens, but voice commands as computing moves into an era beyond screens.
Consulting firm Digi-Capital estimated the world market for augmented- and virtual-reality devices at EUR 3.3 billion in 2016. The firm projects that, driven by next-generation mobile augmented-reality hardware, the market will reach EUR 85 billion by 2021. Once Microsoft has built an attractive mixed-reality library, products such as the three-dimensional teleconferencing app Holoportation could well join bestsellers like the Office 365 commercial suite with over 100 million users.
“Kipman’s HoloLens can make holographic likenesses appear in thin air with new levels of sophistication — for instance in teleconferencing calls. Because the mixed-reality headset requires no computer screen or keyboard, it could be part of a revolution that would replace smartphones as the main communications devices,” adds Battistelli.
“It’s an experience where content is no longer tied to a screen but exists in both time and space laid over the real world,” says Kipman, who has predicted the “death” of the smartphone and a new revolution in personal computing build around the concept of Mixed Reality. Current industry trends appear to support his statement — such as the shift towards voice search with digital assistants like Cortana from Microsoft, Siri from Apple and Alexa from Amazon. Consumers may one day be talking and gesturing to holographic “ghosts”, while the smartphone becomes a spectre of the past.
Kipman has been awarded the Popular Mechanics Breakthrough Award (2009), Microsoft Outstanding Technical Achievement Award (2012), and IPO Education Foundation’s Inventor of the Year Award (2012). He also received the honor of being named one of TIME magazine’s Top 25 “Nerds of the Year” (2010), an epithet he wears proudly.
To find out how to leverage VR/AR/MR in your enterprise, Tech Trends offers bespoke Virtual Reality Consultancy support
Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.
The sexily named patent is viewable in full here: EP3028121
Originally published at Tech Trends.