Image: Archive

Most organisations that attempt to run innovation pilots hit the same wall: real operational problems are messy, the people who understand them are busy, and employing solutions without deeply understanding the problem first is how budget disappears with nothing to show for it.

This case study is about one engagement that looked at that wall and decided to approach it differently: a UK charity that chose to name its hardest operational challenges openly, and experiment with building a repeatable, defensible way to bring external innovation in to solve organisation-wide problems.

The brief

The client is a large UK charity delivering multiple public services nationwide. This project set out to test whether startup ventures could help the charity deliver some of its own services better.

I was brought in as an external innovation consultant with a three-part brief:

1. Understand the context and the roots behind the operational problems, including how much they are costing the organisation in staff time, budget, and quality of the outcomes of the services delivered
   
2. Translate the learnings into a repeatable methodology for connecting this organisation with private tech ventures for targeted pilots, and
   
3. Test the methodology for the first time across three of the charity’s live service areas

What’s interesting to note is that none of the challenges in this project are unique to this charity. They run across the third sector, but most organisations have learned to live with them in one way or another.

What is unusual here is that the charity chose not only to name these challenges, but to take a controlled risk in attempting to solve them, as part of a wider, stated ambition to lead on innovation in public service delivery.

But why bring in external tech startups to solve organisational problems, rather than building internally or procuring custom software?

When introduced to this problem, my first questions were:

Wouldn’t it be easier if we discovered the problem and worked with a software partner to build a custom software solution to it?

How can we guarantee that an already existing tech venture will solve a problem we haven’t even defined yet? Isn’t this working backwards?

Those are all reasonable questions. But turning to existing tech ventures is actually the cleverer route into this specific context, for reasons I only came to fully understand after immersing myself in it.

1. Mutual interest and problem transferability

Public services and tech ventures often want to find each other, but speak different languages.

Ventures, especially early-stage ones, are looking for first customers and design partners: real environments that turn their ideas into real-world case studies and gather the evidence they need for funding. Many public sector organisations are looking for solutions to deeply ingrained operational problems, but are stuck in cycles of long procurement and low trust in new tech.

This project set out to test whether that gap can be bridged in a structured, repeatable way. If it can, the same bridge is useful to many other public sector organisations across the UK.

2. Custom software has a maintenance problem that charities cannot afford

A custom build does not end at launch: it needs hosting, security patching, feature development, bug fixes, and someone to own it… indefinitely.

That demands a product team, and charities don’t usually have product teams, nor the budget to hire external agencies to maintain them.

The sector is dominated by incumbent or commissioned tools that worked on day one and were abandoned within months because nobody was resourced to maintain them. A venture, by contrast, is adaptable and maintains its product because its survival depends on it. Plus, every other customer they win funds improvements the charity gets for free.

3. Speed to learning

A custom build takes months to be ready, and rests over a fixed brief before anyone learns anything. An existing-but-imperfect product can be in practitioners’ hands within weeks of selection.

For a problem that has resisted well-intentioned fixes across the sector, the fastest route to understanding what actually works is to put working tools in front of the people with the problem and see what happens, rather than writing specifications for a problem that is not yet fully understood.

4. The discovery work de-risks the match

The risk of shopping for an existing solution is buying something that doesn’t fit, and only finding it out when too much money has already been spent. The in-depth problem-discovery phase exists precisely to reduce that risk: by the time the venture callout opened, the problem was defined sharply enough (step by step, service by service) that applications could be judged against a real picture of the work.

The methodology doesn’t hope a venture happens to fit, it proactively defines the problem first, then asks ventures to demonstrate how they will fit it.

That isn’t to say that custom software is not the right solution to similar problems in different types of organisations. My point is that, in this case, it wasn’t.

Why this matters

While charities are non-profits, they are not outside the economy. They run on grants, contracts, and commissioned funding, and they compete for all three to stay running. When a bid is written, the organisation’s evidence of impact is what wins it: concrete stories of what a service did for the people in it, backed by data.

That is what makes this problem commercial, not just operational. Every service well delivered produces a measurable outcome. Every outcome that gets captured becomes a case study. Every strong case study strengthens the next bid. And every bid won brings income that funds better services.

Improving how services support their users is therefore not just a nice-to-have for a charity, but feeds the very engine that keeps the organisation funded and the services running. Every contract won translates into money that keeps coming in, services that keep running, and people who keep being supported.

The problem

The problem chosen for the pilot was born from a specific concern in one specific service, but my discovery efforts showed that it exists across all the services investigated, and therefore assumed to be, in some shape or form, universal across the whole organisation.

The charity’s practitioners spend a large share of their working time connecting service users (members of the public) to outside support the charity itself does not provide, such as mental health services, housing, training, employment opportunities and community organisations.

Today that work is almost wholly manual. It runs on individual practitioners’ knowledge, personal contacts, and search engines that often show inaccurate or outdated results. Staff have to spend significant time chasing outdated information and putting effort into making personal connections with external service providers.

When staff leave, their knowledge, contacts and processes often leave with them. Attempts to build shared databases haven’t stuck, because they relied on already stretched practitioners maintaining them manually.

The three services involved in the pilot could hardly be more different: one works with people in the criminal justice system, one supports young people outside mainstream education, and one supports young people at moments of acute crisis. What they share is that connecting people to outside provision (housing, mental health, employment, community support) is central to all three.

Phase 1: Discovery

This is the most important part of the entire process, and the one most often skipped by companies trying to run innovation work.

Solutions employed to half-understood problems do something worse than fail loudly: they fail vaguely, and later nobody can articulate why. This leads to lost effort and budget, and it’s why so many innovation initiatives die after one bad pilot, or fail to be adopted after a good one.

We went the other way instead: spent a lot of time understanding and articulating the problem in all its nuance first, documenting it heavily, and naming the assumptions we were making as we went. That way, if something failed later, we would know exactly why, instead of being blindsided by it.

What we did

The discovery phase ran one session per service area over roughly six weeks, each designed to build a grounded picture of how the connection problem plays out in that service today.

The first service examined in depth received a full-day in-person workshop with a small group, including frontline practitioners and senior leadership. The day mapped the service user’s connection journey stage by stage, surfaced the constraints that shape every decision (data sharing, risk, reporting), and closed with the team’s own view of what good would look like.

As the baseline for everything that followed, its outputs were a service blueprint and a question: which parts of this picture are transferable to the other services, and which are unique to it? That question would follow me through the rest of discovery.

The remaining services each had a focused online session with their service leads, covering their delivery model, referral pathways, what had been tried before and why it didn’t stick, and concrete examples of where the current process loses information. A fourth service took part in discovery but did not proceed to the pilot stage.

why the difference in format FOR EACH SERVICE?

Because that is the process working as intended: the method stayed the same, but the format flexed around the discovery needs and each team’s capacity.

Whatever the format, every session was recorded, transcribed, synthesised into a written record, analysed both individually and against the others, and shared back to participants, so the people who contributed could see what came of their input.

What we found

The connection problem shows up uniquely in each service, but has the same shape in all four investigated, which consists of five main steps.

In a nutshell: Finding providers is manual and slow. Knowledge lives in individuals’ heads, not in systems. And, as it happens in most organisations doing this work, visibility often drops once a connection between community service and service user is made: whether the person engaged, or whether the support helped is largely unknown.

Closing that feedback loop is one of the core aims of the pilot, because gathering impact data and translating it into measurable outcomes will have a direct impact in the organisation’s ability to win more bids.

Phase 2: The ventures callout

Process and method

To choose the ventures who would come into each service, we opened a competitive process and callout for ventures to apply.

As per the public problem framing, rather than narrowing the pilot to a single problem, we decided to present founders with the whole journey and let them self-select which parts their product can solve. We hypothesised some ventures would offer to solve one step, while some would offer more. Both could get in, and what matters is their credibility and the quality of what they already deliver, rather than the breadth of their solution.

Applications were scored on a weighted marking sheet that objectively assessed:

• Fit with the problem
• Ability to run the pilot
• Quality of the proposed measurement plan
• Ability to work within the organisation’s constraints (with information security as a hard pass-or-fail gate), and
• Price

Two principles shaped the design:

• The process is screening for products that already exist, not for funding development. Founders may “wizard-of-oz” unbuilt features, but only if the feature is already on a credible product roadmap and they are honest about it.
• Measurement quality carries the same scoring weight as product fit, because it is the strongest available signal of whether a founder genuinely understands the problem.

Phase 3: Pilot design and evaluation

One venture was selected per service, which means there are three active pilots within the charity, each running six months from the go-live date.

Before launch, I gathered with each service a baseline metrics document, showing the numbers around how their version of the connection work runs today, from time spent finding provision to how often anyone learns what happened after a connection.

These documents did double duty as the venture’s brief, and they are what the pilots will be measured against halfway through and at the end. That way, success is defined as movement against a service’s own baseline.

The pilot structure was adapted to each service rather than imposed uniformly, because the teams vary in size and working patterns.

How it works: A small number of staff use the tool for the full six months, running their whole caseload through it. The others continue working the old way, giving a live comparison. In one service, a second group joins halfway and is onboarded by the first group rather than by the venture. This is to deliberately test whether the tool can survive being passed on internally where team composition naturally changes over time (something that would otherwise not be tested until full adoption is in place).

I designed the charity’s internal evaluation methodology to be deliberately independent of the ventures’ own evaluation methods. It runs at two fixed touchpoints, month three and month six, and it includes:

• A short survey of the staff involved
• An interview session built around open-ended questions
• Check-ins with the service lead and the founders, and
• An analysis template against the baseline metrics document. Each touchpoint ends in a plain decision: continue, adjust or stop at month three, and adopt, extend or stop at month six.

The evaluation framework treats stopping the pilot as a legitimate result, not a failure. Solid evidence that a tool doesn’t fit a service, with clearly documented reasons, is a successful pilot, because it produces decision-grade learning that informs the next initiative.

The measure for good innovation work is quality and traceability of the learnings, not every experiment succeeding or achieving its initial expectations.

A final report then judges two things separately: how well each tool worked in its service, and whether the method itself (discovery, callout, selection, pilot, evaluation) is worth running again as is or needs changes.

The full framework was designed to be run by the charity in-house, without me in the room: every handover document is written to be used independently, and the report template guides their thinking with questions that prompt an honest evaluation of both the pilots and the process that produced them.

Whether that handover holds is, itself, one of the things we are evaluating as well.

Initial findings are expected in early 2027.

Beyond the sector: what’s transferable to other organisations

Nothing in this methodology is exclusive to charities or public service organisations. Any company that wants to solve an internal challenge with outside help, whether by bringing in external providers or by building its own solution, faces the same four problems this project was built to solve.

How do you understand the problem well enough before committing to a solution?

• Run structured, unbiased discovery with the people closest to the work: conversations that surface how the problem actually plays out day to day, team by team.
• Listen to their ideas, not to write product spec, but to document assumptions and expectations, and translate them into needs
• Follow it with a thorough analysis that separates fact from interpretation
• Name your own assumptions out loud as you go
• Document the baseline metrics of how the system works today

How do you screen for problem-solvers without telegraphing their answers?

• Share the problem and the commercial stakes
• Hold back the success criteria
• Ask applicants (external providers or internal teams pitching a build) to propose their own definition of success and how they would measure it
• Then compare it to yours

How do you make selection decisions you can defend with data, not opinions?

• A weighted marking sheet agreed before applications open, scored consistently by multiple reviewers, with hard gates for non-negotiables that save time and prevent decision fatigue
• The same discipline works for choosing between vendors, between internal proposals, or between build and buy

How do you know if it worked, without depending on whoever built it to tell you?

• A vendor’s reporting is biased towards renewal
• An internal team’s reporting is biased towards its own survival
• Either way, evaluation needs to be made objectively and independently by the people who give the final word: fixed touchpoints, evidence gathered from staff before opinions form in the room, plain decisions at each point, and a guided thinking tool that judges the pilot itself and also reflects on the whole process that produced it

The same discipline fits a company evaluating startups for a venture-client programme, a hospital group assessing clinical software, a local authority testing civic tech, or a private company piloting a solution it built itself.

The sector changes, and so does the challenge and the solution, but not the evidence-based approach.

If your organisation is working out how to achieve new commercial outcomes without repeating expensive mistakes, this is exactly what I facilitate. Get in touch and I will happily talk through how I’d approach your context.