When flat-panel TVs became popular, home decorators lamented the big black rectangles on walls distracting from the room’s décor. In response, they developed innovative ways to camouflage the TVs when turned off. Now, as display screens proliferate in automobiles, a similar perspective is taking root among car designers and tech suppliers, who are embracing the idea of hiding displays and other user interface (UI) or user experience (UX) elements when not in use whether on the interior or exterior of the car.
To be sure, this is not a new concept. But at CES 2022, it became clear that smart surface technology could be in cars in five years. The benefits also may extend beyond UI and UX, to improvements in vehicle safety systems. But there are hurdles to overcome.
Minimalism Meets Personalization
“The big design trend has always been towards minimalism to make the inside of a vehicle look as sleek and barren as possible,” says Chris Schreiner, director for UX research at Strategy Analytics in Perth, Scotland. “Smart surfaces help with that, because now instead of a knob or a physical screen, the interior surface of the vehicle can be the controls and displays. Part of the appeal is to reach that design aesthetic.”
Derek Jenkins, senior vice president of design and brand at automaker Lucid Group in Newark, CA, says, “Everybody is looking for opportunities to reduce switch gear inside the car because it is expensive, and often we have to hold those components and not change them” between model lines. Retooling, electronics changes and redesign drives costs up. Smart surfaces present “a multitude of opportunities” to offer different switch inputs or selective functionality through various materials and graphics, as a substitute for switch gear. He says, “The technology is finally being industrialized to the point where it’s applicable at a reasonable cost.”
For an automaker like Lucid that produces an electric vehicle (EV), “the battery is such an expensive component inside the car, it ends up accounting for half the cost of build of materials.” Jenkins adds, “We’re looking for opportunities to save cost, but more importantly have design flexibility.” Smart surfaces provide “the opportunity to unlock that.”
Still, these technologies are fundamentally not yet automotive grade, “to be where it is strong enough to pass automotive tests and last through the lifetime of a car,” contends Strategy Analytics’ Schreiner. “The other challenge has to do with the user experience.”
Schreiner adds, “The shape and style of the interface communicates to the user how it’s supposed to be used and you lose that a bit with smart surfaces. There will be some usability challenges with implementing them.” For instance, “When you see controls embedded in a smart surface, one, they can always change, and two, it’s hard to know exactly where the boundary of that control is. If you see a physical button, you can feel and see where the edge is and you know you have to press somewhere in there.”
He sees smart surfaces included in the next generation of cars. Schreiner says, automakers will face hurdles including organizational challenges, because vehicle interior designers and those who implement controls and displays “don’t necessarily talk to each other very well.”
Smart surface innovations could appear on vehicle exteriors, too. At CES 2022, BMW unveiled the iX Flow concept car, covered with a digital paper wrap that integrated the same display technology used in dedicated e-book readers like Amazon’s Kindle Oasis. Made by E INK Holdings Inc., it is “bistable,” meaning it uses power only when changing appearance, not to maintain it and also boasts ultra-low power consumption explains Timothy O’Malley, executive vice president of operations at E Ink, based in Billerica, MA. The result is a car that can change its color along a grayscale from black to white “in a dynamic flow,” at the push of a button.
“This concept car with BMW is an amazing display of how surfaces of the future will transform to personalize, customize and provide information,” O’Malley says. “What it means to be a car is in the midst of a huge change,” as the transition to EVs “means thinking about how power is managed” and self-driving cars make “being in a vehicle a completely changed experience.”
For example, changing the color of the car with seasonal weather can help reduce the need for power-draining heating or air-conditioning in an EV. White on hot summer days can increase the vehicle’s reflectivity, keeping the interior cooler. While black helps absorb heat from the sun on cold winter days, retaining cabin warmth.
However, wrapping an entire car with an E Ink display was an engineering feat because of the vehicle’s myriad contours, angles and other exterior surface details, O’Malley says. “The first things you’ll see will be smaller components [that use the technology] and it will probably be years before we get to a full car,” he predicts. On the iX Flow, the wheels also were wrapped in E Ink’s color-changing digital paper. “We think about surfaces, and [enabling them] to either be design elements or functional elements,” he says.
In fact, the intent was to use E INK technology inside a car, says Stella Clarke, head of the iX Flow project and a research engineer at BMW in Munich, Germany. “We’re always looking for innovative ways to improve our interiors, and that’s where the first use cases were, and then it grew from there.”
The implementation is best described as a “disappearance effect” instead of as a smart surface, because “there’s no intelligent algorithm behind it,” she clarifies. The usual deployment is extremely energy intensive, involving a very bright light source deployed behind a masking surface, she adds. “This is one of the reasons I love E INK. It’s so low-energy, in and of itself,” plus doesn’t require masking. “It also has a practical advantage of sunlight readability.”
Inside the cabin, “there’s more surface than just the dedicated button areas or the dedicated display areas” that could incorporate an E INK surface. Outside, one use case is information conveyance on a car-share vehicle, Clarke suggests. Exterior color could indicate the cleanliness status of the interior, or identify a particular user’s vehicle at a glance.
BMW’s iX Flow began as a hobby project for Clarke, with her first implementation of an exterior application created at a local Maker Space, and shown within the company in 2020. Exterior applications began in 2021, evolving to the concept car. The body wrap is composed of 60 segments linked to seven microcontrollers. Each wheel wrap is paired with a microcontroller, as well. Right now, the automaker has no plans to put E INK in a production vehicle. “We’re still in the process of doing the robustness tests,” Clarke says.
Displays Are Act One
At CES 2022, auto supplier Marelli showcased smart surface technology built into a central console, a full-stack unit that flows from the dashboard to a center armrest. Up top, it features a touchscreen display optically bonded to unique curved glass with embedded illumination and switchless buttons. The base features a wireless charging pad visible through the transparent display above. The armrest contains a circular touchpad made of smart surface wood that controls the display and provides light feedback. The heart of the system is a Marelli-developed multi-function layer technology called MFL that allows for icons, switchless buttons, histograms and other symbols to be displayed on a transparent surface, says François Nivelle, head of innovation and advanced engineering for the Electronics division of Marelli in Châtellerault, France.
While MFL generates light in the transparent material, smart surface technology generates light behind the surface using a local dimming scheme, turning it on only where needed and leaving it black where not, Nivelle explains. But instead of a traditional TFT display, the surface can be anything from real or artificial leather, wood, metal or stone — modified to let the light shine through. The addition of haptic technology could alter the feel of the surface from smooth to rough, he adds.
Some executions are more difficult than others, Nivelle says. Leather and wood versions could be available in production vehicles in two years. “We are at the beginning of the story,” he says. “There is a lot of technology coming and usage that we have not yet imagined.” Some automakers could leverage smart surface technology as features purchased for a vehicle through over-the-air software updates, he muses.
Meanwhile, Continental Automotive won CES 2022 Innovation Awards in the In-Vehicle Entertainment and Safety categories for its ShyTech Displays, which include a video-capable display that can be hidden behind a surface, and a display that hides and reveals buttons, lights and switches showing only when needed.
As displays proliferate in vehicles, the amount of information displayed on them can increase driver distraction, says Jens Brandt, vice president for user experience, North America, based in Auburn Hills, MI. ShyTech displays mitigate distraction as well as help automakers distinguish themselves stylistically, he says. “You can achieve a minimalistic design.”
Of course, the challenge is to bond a normal-looking transparent material on top of a display without reducing brightness, sharpness and contrast, Brandt says, adding that the target is 800 to 1,000 nits of brightness. The answer is a sophisticated backlight matrix with many LEDs coupled with a thermal management system, which Continental created entirely in-house. Continental’s ShyTech displays should be in a production vehicle as soon as 2025, Brandt says.
“What matters is what people see. What you measure is not important,” says Tara Akhavan, founder and general manager of Faurecia IRYStec Inc., based in Montréal, QC, Canada. “A 20-year-old sees details on a screen very differently from an 80-year-old driver. We can’t expect them to perceive and then react the same way.”
So, IRYStec has developed what Akhavan describes as a perceptual processing and evaluation system called PDP Vision (perceptual display platform), which measures how a screen image is perceived by the user, also taking into account ambient variables such as sunlight. The technology is agnostic to display type; it works with both disappearing smart surface displays and traditional varieties, she says. PDP Vision has been in production vehicles from Mercedes-Benz for the past two years, and other automakers are set to adopt it, she says.
Further, IRYStec’s tech roadmap involves ways to personalize the display for physiological requirements, such as age-related vision conditions or colorblindness.
Not all smart surfaces have to be relegated to controls and displays. Toyota Research Institute of North America (TRINA) in conjunction with Toyota IP Solutions, launched last August after debuting at CES in 2020, and has developed a technology called electroactive artificial muscles — that can be licensed for applications as simple as a cell phone holder or as involved as a seat massage function.
“Each one is about half a millimeter thick. They’re flexible polypropylene envelopes that have electrodes in them and when we apply a voltage, they flex,” explains Michael Rowe, who is both the program manager for advanced development at Toyota IP Solutions and a principal scientist at TRINA in Ann Arbor, MI.
They also are ultralight and very powerful, Rowe says, noting that a single one can move a 500-gram weight. “If you want to lift more weight, you’d put more than one stack next to each other and have them work together.” Very little power is needed to run them; power from a USB is sufficient and “these are actually a form of flexible capacitor, so after they’re initially charged, they no longer use energy” beyond a trickle charge.
Assembled in a structure that resembles a Gothic spire, the muscles can be distributed throughout a seat and used as a massaging device — a function normally reliant on heavy electric motors, pumps and gears. In a three-point triangle design, they can create a non-slip surface, like a floor mat, on-demand. Or, in another form, the muscles could create a suction-cup-like structure that emerges from a dashboard to grip a cell phone. Or a two-part cantilevered structure that resembles a leaf could open and close to contain sunglasses, and muscles inside can grab or release them on-demand.
Outside, the muscles could be applied to “roughen the surface” of the vehicle on-demand and “create turbulence” that combats crosswinds when needed, improving vehicle stability through actively altering aerodynamics, he says.
Eventually, Rowe envisions using electroactive artificial muscles to create “reconfigurable interiors that no one could ever hope to do.” For example, to make foot stools or seats that pop up from the car floor and can support up to 50 times their own weight. “You literally build a seat up like Lego blocks,” he says. “We’re looking at structures that would turn this 90 degrees” to form a seat back.
“If we want to lead and not follow, then we don’t go out and do what everyone else did,” Rowe declares.