"Human In The Loop" Is Not Enough

Welcome to data uncollected, a newsletter designed to enable nonprofits to listen, think, reflect, and talk about data we missed and are yet to collect. In this newsletter, we will talk about everything the raw data is capable of – from simple strategies of building equity into data+AI processes to how we can make a better community through purpose-driven analysis.

**** A love note on rest and care****

It has been almost a month and a half since I last wrote here, and I have felt that absence. So before drafting another edition for you and me, I wanted to take a moment to reach out — with a note of love — to honor this season’s tenderness and complexity.

I hope that as you feel summer in small and big ways, in and around you, there is care, joy, and enough love to protect you and those you hold dear.

Let us start with an acknowledgement - this is not an easy time – and I want to name it (or at least try).

So many of us are navigating exhaustion layered with grief and anger, resisting harm in many forms: from imperial violence to creeping authoritarianism, from racism and transphobia to attacks on bodily autonomy, from surveillance to the entanglement of AI in unchecked power. It is a lot to carry, and many of us carry it every day.

In the midst of this, I want to send a simple reminder: your rest matters. Care is not something we earn when the work is done — it is how we sustain our humanity in the face of all that would threaten it. Rest is not retreat. It is an act of preservation and sustainability.

My own pause over these past weeks has been part of that practice — allowing space to step outside the rush of constant doing. I know that some of you, too, may be feeling stretched thin right now. If so, I hope you can give yourself permission to slow down, step back, and replenish, however that looks for you.

This note is a recognition of our shared need for tenderness and care.

I am thinking of you — of the invisible and visible ways you are holding on, showing up, resisting, creating, dreaming. If, in the process of finding rest and building dreams, you need a friend to cheer for you, know that I am here.

We don’t have to know each other for years or have shared past or experiences. We don’t even need to have shared meals. The mere reason that you choose to show up in this newsletter space, in the “you” that I like to use so often here, is more than enough to keep me in your corner, always.

Yes, I see you.

***********************************

“Don’t worry, we’ll have a human in the loop.” It is a phrase that shows up in most of my AI conversations too easily, perhaps intended to reassure us around the table that humans will retain control even as artificial intelligence becomes more deeply embedded in decision-making.

I suppose it has become shorthand for “let’s trust the human we can get involved, this won’t go off the rails.” But lately, this phrase makes me uncomfortable. In many cases, it is somewhat of a shallow promise that papers over deeper concerns about whether a system is truly designed with human interests at its core.

At one of my recent calls on AI governance, someone suggested that a group of lawyers would review and approve AI tools that staff members propose using. Most people around the table labeled this as “human-centric governance” — us putting humans acting as responsible gatekeepers. But this version of the “human in the loop” approach risks curbing innovation and experimentation. It reinforces hierarchy and control rather than empowering staff to explore, learn, and responsibly test new tools in their day-to-day work. This approach isn’t entirely human in the loop; it’s human in the way.

The phrase “human in the loop” is too often treated as an ethical badge — a box to check that signals accountability — but its application often fails to center actual human needs, voices, or agency. The presence of a person somewhere in the workflow doesn’t automatically make a system more humane, ethical, or equitable. Who is the human in the loop? At what point in the process do they engage? Are they simply there to rubber-stamp decisions, or do they have the genuine authority to challenge whether a system should even exist?

In many scenarios, the human is inserted at the end, asked to approve or disapprove something a machine has already decided, with limited context or understanding. Think of the content moderator scrolling through hundreds of flagged items per hour, or the caseworker forced to “validate” algorithmic recommendations on benefits eligibility. In other cases, the human in the loop is upstream, shaping training data or design, but often without community input or reflection on bias and impact. And even when feedback is solicited, the so-called “loop” often lacks mechanisms for those insights to flow back meaningfully and lead to change.

This is why I am bringing this lens to explore with you – that we need to define this “human in the loop”.

There is and will always be a deeper question with AI: What does it mean to center humans in AI-driven environments truly?

Real inclusion and innovation demand much more than occasional oversight. They require cultures that enable experimentation, shared power, and reflection. Let’s return to my example above. Imagine if instead of a monthly panel deciding what staff can try, the organization created an AI sandbox — a safe space where employees could experiment with tools in low-risk contexts, share learning with colleagues, and document both successes and failures. Different titles and departments could still be involved, but as advisors and guides rather than gatekeepers. This would build trust, promote responsible use, and encourage curiosity — key ingredients for meaningful innovation.

I want us to understand something clearly - humans in the loop doesn’t guarantee human-centeredness.

The real work involves designing processes and cultures where human judgment is trusted (and challenged in healthy ways), where learning is encouraged, and where those most impacted by AI have a seat at the table from the very beginning. Otherwise, we are just pretending that inserting a human at a control point will make up for a fundamentally exclusionary or opaque system.

Here are some examples of what it could look like:

• Community Feedback Loops: A nonprofit developing a grant scoring algorithm holds quarterly community (public/invited) town halls with grantees to review how the tool is functioning. If certain organizations are consistently deprioritized, the tool is paused and retrained. Here, “human in the loop” means lived experience is the loop.
• Peer AI Labs: A mid-size organization launches an internal AI playground where different teams can experiment with generative AI tools and log discoveries and concerns. A rotating, cross-functional committee facilitates learning and ensures values alignment. No monthly approvals — just collective stewardship.
• Red Teaming for Equity, Not Just Security: Before deploying a donor engagement chatbot, an organization invites staff from all backgrounds (to build real diversity) to “red team” the tool — not just for security risks but for harmful framing, biases, and accessibility issues. This is human against the loop, if needed.
• Data Consent Dialogues: Instead of a standard “terms of use” checkbox, an organization offers participants a short video and guided reflection on what it means to share data in an AI-powered system. People can opt in to some uses and not others. Consent becomes a conversation, not a transaction.

In all of these examples, the key difference is that people closest to the impacts — whether staff, communities, or clients — are not simply “in” the loop; they are defining, shaping, and questioning the loop itself.

Let us put some definition to this “human in the loop”.

This should mean

• not just approval after the fact, but collective clarity upfront on why the AI exists and who it serves.
• including people from historically marginalized communities, not as test subjects, but as co-creators.
• Building transparent feedback systems where insights, errors, and improvements flow both ways — from people to machines and back.
• Encouraging distributed agency - so that anyone affected by the system has the power to ask questions, suggest changes, or even halt the automation.

It is insufficient for the complexities AI can and will create that we continue to use “human in the loop” loosely. Especially if it’s meant to be a proxy for lack of organizational accountability. We need to move beyond thinking of human oversight as a checkbox and toward a richer, more participatory vision where humans aren’t just auditing machines (so it can be called “human-centric”) but co-shaping the entire ecosystem around them.

Here is an easy trick: ask “does the table building AI systems have voices from all identities, shapes, histories, context, to  influence, question, and reimagine AI?”

Until the answer is “yes,” we haven’t built human-centric AI — we have built AI with occasional human interrogations.

***********************************

*** So, what do I want from you today (my readers)?

• Share with us: how can we hold organizations responsible for their actions with AI, while going beyond “human in the loop”?