The practice of UX design has historically been centered on digital interfaces—screens, workflows, navigation systems, and information architectures. However, the next generation of products is increasingly defined by the convergence of physical systems, embedded software, connected devices, and human interaction. From medical devices and surgical technologies to mobility systems and wearable health platforms, many of today’s most impactful products operate across both physical and digital domains, requiring experiences that extend well beyond traditional screen-based interactions.
By:
Gregor Mittersinker
June 29, 2026
Designing for complex ecosystems demands a broader and more integrated approach to user experience. The user journey is no longer confined to a digital interface; it is shaped by industrial design, hardware constraints, sensor inputs, software behavior, environmental context, and human factors. As physical and digital systems become increasingly interconnected, human-centered design must evolve to address the full continuum of interaction, ensuring that products are not only functional and usable but also intuitive, safe, and effective within the real-world environments in which they are deployed.
The experience is not located on the screen. It is distributed across a system of physical and digital touchpoints that the user moves through as one continuous perception.
For most of the history of digital product design, the screen was the assumed site of the experience. Websites, applications, dashboards, and mobile interfaces: the discipline of UX grew up designing interactions that took place on flat, rectangular displays. Within that constraint, the field developed extraordinary depth: information architecture, interaction design, visual hierarchy, accessibility, usability testing, and service blueprinting. But the world has moved on. The Internet of Things has placed computational intelligence inside physical objects that have no screen at all. Wearable devices collect biometric data and communicate it through haptic feedback, ambient light, or companion applications. Industrial equipment now carries embedded sensors, remote monitoring interfaces, and machine learning diagnostics alongside its physical controls. Medical devices must be operable in high-stress clinical environments where a clinician cannot afford to look down at a display. In each of these contexts, the experience is not located on the screen. The experience is distributed across a system of physical and digital touchpoints that the user moves through, sometimes sequentially, sometimes simultaneously, always as one continuous perception.
Physical-digital product ecosystems have evolved beyond the boundaries of any single product or service. Today, user experiences are distributed across hardware, software, connected services, and the environments in which they are used. The quality of that experience depends not on the performance of individual components, but on how seamlessly those components work together. Designing for these ecosystems requires a shift from component-level thinking to system-level thinking. The ecosystem itself is not defined by a device, interface, or touchpoint in isolation; rather, it is a network of interdependent elements whose value emerges from their interactions.
Consider a continuous glucose monitoring (CGM) system used by a person managing diabetes. The experience extends far beyond the wearable sensor attached to the body. It includes the mobile application that displays glucose trends and delivers alerts, cloud services that synchronize data across devices, clinician dashboards used to review patient information, and the real-world contexts in which individuals make decisions about meals, exercise, medication, and sleep. From a technical perspective, these are distinct systems with different architectures, interfaces, and stakeholders. From the user's perspective, however, they form a single experience. A delayed alert, a confusing data visualization, a synchronization failure, or a disconnect between patient and clinician tools can erode trust and disrupt decision-making, regardless of how well any individual component performs. It is this unified human experience, not the boundaries between systems, that UX must design for. As products increasingly operate across interconnected physical and digital domains, the central challenge for UX becomes creating coherent, reliable, and human-centered experiences across entire ecosystems rather than optimizing isolated interfaces.
Most organizations do not design physical and digital ecosystems as unified experiences. Instead, they develop them as parallel workstreams.
Industrial design teams focus on the physical product: its form, materials, ergonomics, and manufacturability. Software teams focus on applications, interfaces, and backend integration. Engineering teams focus on technical performance, including reliability, power management, connectivity, and regulatory compliance. Each discipline optimizes for its own objectives.
The result is often a product whose individual components are well designed in isolation but whose transitions feel fragmented. A user picks up a connected device and finds the physical controls intuitive. They then open the companion app and encounter a different visual language, interaction model, and conceptual framework. The boundary between the physical and digital worlds is rarely designed intentionally; it often emerges where one team’s responsibility ends and another’s begins. Yet that boundary is exactly where users experience the greatest friction.
This fragmentation is not the result of negligence. It is the predictable output of organizational structures that were built for a world where physical and digital products were genuinely separate things. The organizational structure has not yet caught up to the product reality.
Designing for physical-digital ecosystems requires a set of principles that extend beyond screen-based UX without abandoning its foundations. Human-centered thinking remains essential. Usability, accessibility, and empathy remain core values. But the methods, the artifacts, and the collaboration models must evolve.
In a physical-digital ecosystem, users constantly move between modalities. They interact with a physical control, glance at a display, check a mobile notification, and return to the device. Each transition introduces cognitive load, requiring users to reorient themselves to a different interface while maintaining their understanding of the task at hand. Interaction continuity is the practice of making these transitions feel like movements within a single experience rather than context switches between separate products. Information presented on a device and in its companion application should be consistent, complementary, and synchronized. Physical controls and digital interfaces should share a common logic, with interaction patterns and terminology that remain recognizable across both domains.
Achieving this continuity is not primarily a visual design challenge. It is a systems design challenge that requires close coordination between industrial design, interaction design, and engineering, along with a shared understanding of the users’ workflow across all three disciplines.
Embedded UX has developed advanced techniques for understanding user behavior, including task analysis, user flows, and scenario mapping. These methods focus on identifying users’ goals and designing interfaces that help them achieve those goals effectively and efficiently. Physical-digital design introduces another critical dimension: context. Context includes the physical setting in which a task takes place, whether on a noisy factory floor, in a sterile operating room, inside a moving vehicle, or in a patient’s home. It also encompasses the user’s physical and cognitive state, such as wearing gloves, working under significant mental workload, operating in low-light conditions, or juggling multiple responsibilities at once. Beyond these factors, it extends to the social and organizational environment, including whether users work independently or collaboratively, whether errors have immediate consequences, and whether devices are shared among multiple individuals. These contextual factors play a central role in determining what makes an interaction effective. An interface that works well in a quiet office may become cumbersome, or even hazardous, in a fast-paced clinical environment. Designing for context requires studying users in the settings where products are actually used and understanding how environmental conditions influence behavior, decision-making, and performance. What feels intuitive to an experienced technician may be challenging for a patient using a home infusion device for the first time.
Designing for context therefore requires research methods that extend beyond traditional usability labs.
In digital design, the interface is what appears on a screen. In physical-digital design, the interface begins with the object itself. Its weight, texture, size, and form communicate meaning before a user touches a control or opens an application. A device that feels substantial and precise can convey reliability and accuracy. One that feels lightweight and flexible may suggest portability and adaptability. Conversely, a device that is awkward to orient or difficult to grip can create confusion, regardless of how refined its digital interface may be. Physical form is not solely the domain of industrial design; it is also a UX concern. The way users perceive and interpret a product’s physical characteristics shapes their mental model of how it works, what it can do, and whether it can be trusted. By contributing human factors research and user insights to form-development decisions, UX practitioners help ensure that the physical interface is as intentional and meaningful as the digital one.
Physical-digital products often support user journeys that span time, environments, and modes of interaction. A connected insulin pump, for example, may be configured by a clinician in a hospital, managed by a patient at home, monitored remotely by a care team, and serviced by a field technician. Although each user interacts with the same product ecosystem, they do so in different contexts, with different goals, levels of expertise, and consequences for success or failure. Designing for the entire journey requires understanding the ecosystem at multiple levels: individual interactions, sessions, daily workflows, and the long-term relationship between users and the product. It also requires identifying where different users’ journeys intersect and where those intersections can create coordination challenges, communication gaps, or breakdowns in the overall experience.
Journey mapping at the ecosystem level is more complex than traditional user journey mapping, but it delivers the same core insight: users do not experience products as isolated screens or discrete interactions. They experience them as continuous flows of activity through which their interaction modes are either enabled or obstructed.
The principles of physical-digital design require practical methods to implement. Several established UX tools extend naturally to ecosystem contexts, while others require new approaches developed specifically for hardware-software integration.
An effective ecosystem journey map captures how users interact across modalities, access information, transition between physical and digital channels, and encounter points of friction throughout the experience.
Ecosystem journey maps extend the service design perspective by representing the experiences of multiple stakeholders operating within a shared product ecosystem. Rather than focusing on a single user, they capture how different actors interact with interconnected physical and digital touchpoints across their respective journeys. An effective ecosystem journey map captures how users interact across modalities, access information, transition between physical and digital channels, and encounter points of friction throughout the experience. By making these interdependencies and points of failure visible, ecosystem journey maps expose forms of fragmentation that are often hidden by organizational silos, disciplinary boundaries, and disconnected development processes. When developed collaboratively by engineering, industrial design, and product management teams, ecosystem journey maps function as a shared service blueprint that establishes a common understanding of the end-to-end user experience. As a cross-functional artifact, they enable teams to align perspectives, identify systemic challenges, and translate strategic design principles into coordinated and actionable product decisions.
Usability testing in controlled laboratory settings remains valuable for evaluating specific interactions and interface behaviors. However, physical-digital products are rarely used under controlled conditions. They operate in hospitals, factories, vehicles, and homes, environments shaped by interruptions, time pressure, environmental constraints, and the unpredictable demands of everyday use. Contextual inquiry, conducted in the environments where products are actually used, reveals critical aspects of the user experience that laboratory studies often overlook. It exposes how a nurse interacts with a device while simultaneously communicating with patients, how a field technician operates equipment in poor lighting or adverse weather conditions, and how a patient navigates a device originally designed for clinical settings.
For physical-digital products, contextual research is not merely beneficial—it is essential. Because the physical environment directly influences product use, it must be understood as an integral part of the design problem. Gaining that understanding requires observing users in the context of real-world practice.
Testing exposes breakdowns at physical-digital handoffs, mismatches in user mental models, and design assumptions that fail under real-world conditions.
Modern UX prototyping tools such as Figma, Lovable, and Claude Code are effective for exploring screen-based interactions and validating user experience goals. Physical-digital products, however, require integrated prototyping approaches that evaluate hardware and software interactions together, often through rapid prototypes, simulations, or digital twins. These prototypes can range from low-fidelity combinations of physical mockups and interface prototypes to higher-fidelity systems that pair functional hardware with interactive software. The objective is not complete realism but sufficient fidelity to validate the interaction model and ecosystem experience before full development.
Testing integrated prototypes with users in realistic contexts reveals insights that component-level testing cannot. It exposes breakdowns at physical-digital handoffs, mismatches in user mental models, and design assumptions that fail under real-world conditions.
Human factors engineering, which focuses on designing systems for safe and effective human use, with particular emphasis on error prevention, provides essential methods for physical-digital product design, particularly in high-stakes domains such as medical devices, industrial equipment, and mobility systems. Methods such as task analysis, error analysis, and formative usability testing under human factors protocols introduce rigor to the evaluation of physical-digital interactions that conventional UX approaches may not fully capture. In regulated industries, human factors documentation is a formal requirement. Beyond compliance, however, it serves as a critical framework for making the consequences of design decisions explicit, reinforcing that interaction design is not merely an aesthetic concern but a safety-critical engineering responsibility.
The technical and methodological challenges of physical-digital design are real. But the organizational challenge is often harder. Designing coherent physical-digital experiences requires collaboration between disciplines that have historically operated independently, with different professional vocabularies, different success metrics, and different organizational reporting structures.
Establishing a shared design language through common artifacts, vocabularies, and frameworks is a prerequisite for effective cross-disciplinary collaboration. Ecosystem journey maps, shared principles, and co-located design reviews act as mediating structures, creating a common reference point that enables dialogue across disciplines and supports coordinated decision-making in complex product ecosystems.
UX input must arrive much earlier in hardware development than it typically does in software development. Most design decisions are locked before usability testing even begins.
Hardware and software development operate on fundamentally different timelines. Physical components require specification, tooling, and manufacturing—processes that are costly and difficult to reverse—so UX input must occur much earlier than in software development. In many organizations, UX is only introduced during late-stage usability testing, when key design decisions are already fixed. At that point, feedback is largely retrospective, informing future iterations rather than the current product. To influence hardware design effectively, UX practitioners must engage at the outset of development, during industrial design exploration, human factors definition, and early trade-off discussions. This requires a sufficient understanding of hardware processes to contribute credibly, ensuring human-centered considerations shape decisions before they become locked in.
A well-developed Visual Brand Language (VBL) is a key mechanism for creating coherence across physical and digital touchpoints. It defines a consistent set of visual principles that express a brand’s identity across both physical products and digital interfaces. Beyond ensuring visual consistency, a strong VBL establishes a shared perceptual framework that signals a single unified system, even as users move between modalities. Consistency in color, typography, iconography, and interaction cues enables seamless transitions between devices and applications, while inconsistency exposes perceptual “seams” in the experience.
Developing a cross-modal VBL requires close collaboration between industrial design and digital UX disciplines, each of which traditionally operates in isolation. A unified system depends on shared principles that can translate across physical form and digital interface.
The issue is a mismatch in mental models, differences in how information is structured, labeled, and prioritized across modalities.
The gap between physical and digital design often becomes visible only after problems emerge. Usability testing can reveal issues that neither industrial design nor UX teams identify in isolation. Physical devices are often optimized for ergonomic, task-based interaction, while companion applications follow established digital UX patterns for clarity and information hierarchy. Each may perform well independently, but when combined, users can experience friction when moving between the two. The issue is typically not local usability but a mismatch in mental models, differences in how information is structured, labeled, and prioritized across modalities.
Addressing this requires more than refining individual interfaces. It depends on establishing a shared information architecture that serves as a common foundation across physical and digital systems. This alignment enables cognitive continuity, allowing users to transition between modalities without having to reconstruct their understanding of the task.
Inclusive design becomes more complex and more critical in physical-digital product ecosystems. While digital accessibility is supported by mature standards and tools, and physical accessibility is addressed through separate human factors disciplines, the transition between the two is often overlooked. In connected systems, users may face barriers across modalities: physical controls that are difficult for users with motor impairments, digital interfaces that are inaccessible without assistive technologies, and cognitive load introduced by navigating inconsistent physical and digital mental models. These are not edge cases but common realities, particularly in domains such as connected health where many users have diverse abilities.
Effective physical-digital accessibility requires integrating human factors, digital accessibility standards, and inclusive design practices, with users involved in research and testing from the earliest stages of development.
Modern products increasingly embed automation, AI, and connected intelligence, shifting UX from simple interaction design to shaping relationships with responsive, adaptive objects. Smart devices that learn preferences, medical systems that generate algorithmic alerts, and wearables that accumulate long-term biometric data all introduce new forms of trust, interpretation, and dependency. These capabilities are neither inherently beneficial nor harmful, but they demand intentional design. The key question is whether such intelligence is transparent, aligned with user goals, and accessible across diverse users. For UX practitioners, this carries real stakes: trust in medical devices, safety in clinical workflows, and accessibility in everyday use all depend on how these systems behave and are understood.
Human-centered design across physical and digital contexts is therefore not aspirational but essential. Without it, optimized systems risk becoming efficient for machines but opaque or alienating for people.
The future of UX is not confined to screens but lies in the transitions between them—between physical interaction and digital interface, device and application, and discrete touchpoints and continuous journeys. These moments of transition are where the most consequential design decisions are made, and where user experience is most often shaped or disrupted. The boundary between physical and digital is not a gap to be bridged through visual consistency or owned by a single discipline. It is a design space in its own right, requiring the integration of human factors, interaction design, industrial design, systems thinking, and contextual understanding of real-world use. Practitioners who develop fluency in this space—who think in ecosystems rather than screens, contexts rather than tasks, and journeys rather than sessions—will shape the next generation of meaningful products. Not because physical-digital design is a trend, but because it reflects where human experience increasingly occurs.
Design has always been about people. The challenge now is to follow experience wherever it unfolds, across the full physical-digital reality they inhabit.
Austrian-born Gregor is in his element while dissecting most complex business & technology challenges and creating their next level business outcomes. Prior to starting Loft, Gregor led a Strategy & Design team at Accenture Interactive, where he helped launch new multi-billion dollar businesses for global fortune 500 companies. He also led creative teams at Rollerblade, InMusic & Cross.He has worked in the US, Europe & Asia over the past 30 years has earned numerous design awards as well as holds well over 100 patents for product innovations around the globe.
Outside of business hours he teaches Service Design & UX at RISD, and hosts a weekly think tank with global business & political leaders around the world.
A natural motivator, leader, collaborator, and innovator, the only thing that takes Gregor’s eyes off of design for long is his love for winter sports, kitesurfing and DJing in local clubs. Many have tried to keep up with Gregor, few have succeeded.
Next level inspiration … Japanese wood craft and joinery, minimalist forms that are functional and proportioned.
Previous
Next
AI
Innovation
Education
Ecosystem Design
Strategy
Digital Ecosystems
.png)
