If a system only works when people compensate, mask, or absorb failure, it is not actually functional. It might look fine. It might even be celebrated. But it is not safe.
We’ve been designing this way for a long time, usually without meaning to. We design for the median. We design for what we can measure. We design for what moves fastest. And then we’re surprised when people are harmed by systems that technically work, but only if the human inside them bends, stretches, or breaks.
We already know this problem exists. Accessibility and inclusive design proved it. Once we stopped treating disability as an edge case and started seeing it as part of the human spectrum, everything shifted. Humans vary. Conditions vary. Capacity varies. So we design for limitation, not perfection.
This is not that different from how we handle color.
There are more colors than we can name or standardize, so we created primaries, secondaries, contrast rules, and guardrails. We didn’t disrespect the colors we couldn’t name. We built systems that work across variation, and often something beautiful emerges.
People are like that too.
We will never understand every human situation, but we don’t need to. We can standardize positively the things that reduce harm and amplify joy. Rest. Clarity. Margin. Safety. Recovery. Dignity. Systems that still work when someone is tired, grieving, overwhelmed, sick, injured, or simply human.
Other industries already do this.
In film, bridge sets are engineered to look dangerous while being structurally safe. Anyone responsible for safety can stop the shot. That veto isn’t a failure of creativity. It’s what makes the performance possible.
When we ignore this in the real world, we see what happens. The London Millennium Bridge looked safe, but failed when load accumulated unexpectedly. The fix wasn’t behavioral. It was structural. Capacity, load, limits.
This is what product design is missing.
Not empathy. Not intention. Governance.
Until governance exists, we still have a mandate. Designers, PMs, and engineers can start running safety checks now. Ask where load concentrates. Ask where humans are compensating. Ask where the system only works if someone absorbs the failure quietly.
AI is about to amplify whatever is already present. Fragility or stability. Harm or goodness. That choice happens before scale, not after.
If you want to contribute, I’m not looking for personal stories of harm. I’m looking for ideas for governance.
What would a real safety check look like in product practice.
What policies or permissions would make pausing or vetoing possible.
What models from other industries could translate here.
If we’re serious about AI amplification, this is the moment to design guardrails before scale, not after harm.
That’s the conversation I want to have.
7
1 Comment