If you've been frustrated by your story not doing what you want it to, this is for you.

We want hits, we want a big audience. Or at least we do while being true to our subject. Yet, we see what big, successful Hollywood films do with their stories. And subconsciously, we think that's what we need to do with ours.

Hollywood has taught us that a happy ending is what we watch a story for. The main character has to succeed, eventually. Even if they've been through difficulties, they change if they have to, they grow, and they get what they want.

But people watch documentaries for a different reason. They have different expectations.

They know life. Yes, they want a compelling story, but we all understand how rare a truly happy ending is. Your audience want to see a story that reflects the happy sadness of real life, but with a compelling narrative climax.

And yet most documentary makers keep chasing a happy ending – or at the very least, success. You keep chasing it when you are researching, when you are shooting, in the edit. And this search for a “successful outcome” is where most of the documentaries I help with come apart.

Overruns, missed delivery, confused commissioners, shelved films. These are real-life consequences of not getting your storytelling right. But the issue usually isn't that there isn't enough success in your material – it's that you don't yet have the craft to work honestly with what you have.

Take the notorious – to me – follow doc, where the doc maker, with camera on shoulder, follows a person or an event, but doesn't know when to stop. All they know is they are seeing someone trying to achieve something, and if they fail, then – to this storyteller – their story has failed.

But great stories aren't like that, and neither is real life.

For example, I worked with a student who was following a doctor who was trying to cure addiction, via medication rather than therapy. He was a fascinating subject, with a powerful story to tell, but the problem for the student was that he wasn't succeeding.

The doctor did trials, he explored solutions, some of which seemed promising, but he never succeeded. To the documentary maker, this was a failure. And so she kept following him, hoping that he would succeed. She was burning through her budget, had collected a huge amount of footage, and was confused about where to go next – whether to abandon the project, or keep going until the perfect moment came.

And then we began to work on her idea, and I showed her how real life rarely feels like a triumph – and that doesn't mean there isn't a great documentary story there.

Here’s the definition of a story I use when a film is stuck:

A story is about the dramatic consequences of trying to solve a problem.

Not succeeding, necessarily. Just the consequences.

In the case of the addiction doctor, our remade story was still asking the question: Is this doctor able to find a cure for addiction that doesn't rely on therapy? But the climax showed how the answer was no, because the path he was on was a dead end, and was clouded by his own addiction.

That was a compelling story. We could see the mistakes he was making – moments where we could see the bad choices, and a slow revelation of how this seemed as though it had to end. It had a powerful momentum, and a climax that showed us the consequences of trying to solve a problem in his way.

It was a tragedy – a lesson in what not to do. And documentaries should embrace tragedy.

For one thing, it's very like real life. But it's also a great story, because it shows the audience what went wrong, and why.

Some of the best stories I've worked on were successful because they dealt with this head on. And they were not-quite-tragedies, but still not successes.

Two examples from my work – both sport, which seems to demand success as your story goal. They were hits, not despite failure, but because of it.

With Formula 1: Drive to Survive I was Series Editor of the first season, working with the team to define the storytelling style of the series. But the teams we worked with were nowhere near pole position.

They were losers.

They couldn't be about “succeeding” in a traditional Hollywood sense.

So we had to look at failure, and find the compelling story within it. Be honest about the behaviour that led to the failure. Tell a story that you couldn't turn away from, even if you were looking through your fingers.

At the same time we had to look for the success in the failure. When a driver, or a manager, or whoever, felt good about themselves, even if they had failed.

They were stories that showed that feeling of failing on one level but succeeding on another – showing, clearly and honestly, that the attempt itself was worthwhile, even when the outcome wasn't a win.

I call it the Rolling Stones ending – you can't always get what you want, but it might be what you need.

We had to be ruthlessly honest with what we had, and create a real story that felt true to the audience, without chasing the obvious happy ending.

The same went for Sunderland 'Til I Die – another Netflix sports series I worked on – the team were tumbling down the leagues, which wasn't the plan when the series started, and we had to find great stories in that apparent failure: how it drew the community together, how it showed resilience.

The audiences were really happy with those non-Hollywood endings.

Just now, I'm working with another DocFix student on a story about a woman who is failing to achieve her dream (nothing at all to do with sport, something far more personal and sensitive). At first, the director found it painful to see, and tried to turn the small successes into big ones.

But by recognising the failure, and leaning into it, the real ending emerged. In the end, the woman who is the subject of the film went back to her previous life – which brought her real joy. The Rolling Stones were right again.

In fact, for stories where you are trying to change the world, it's often better that they don't succeed. That leaves the audience with a huge desire to do something about the problem that remains unresolved. A perfect example of documentary storytelling that can genuinely change the world.

So: notice when you're chasing the mirage. Notice when you're trying to turn small successes into big ones because you've been told that's what a story needs.

The real story is almost always in what actually happened – and your job is to have the craft to see it, and tell it.

In DocFix we take the ending you actually have, apply this system to it, and turn it into a structure commissioners and audiences lean into, rather than feel cheated by. If that's where you are with your current film, that's the work we do together.