S2. Ep26 - 5 Reasons AI Agents Fail — And How to Build One That Actually Works

Katie (00:26)

Hello and welcome back to another episode. Hi, hello, I'm Katie. And as always, I have Noel here with me today. How are you doing, Noel?

Noel (00:38)

I'm doing great this week, how are you doing?

Katie (00:41)

Yeah, all good, thank you. We're trying to survive another heatwave, aren't we?

Noel (00:44)

Yeah, it's getting a bit warm, isn't it? Never mind.

Katie (00:52)

Yeah, never mind. So we always kick the podcast off with any updates, any news in the AI or automation world that will affect people using it on a business level. So Noel, are there any updates or news that we should be made aware of?

Noel (01:19)

So I know we've talked about this for a few weeks now, but old Fable 5 did come back, which is fabulous. It was supposed to be taken away today, which is Tuesday the 8th of July, but I believe we've been given another five days.

Katie (01:25)

It did, yes. I had that pop up, I think it was yesterday.

Noel (01:43)

Yeah. So by the time this podcast comes out, you've got about two days left, so make the most of it. I'm not sure why they're taking it away yet again, but hey, we'll soon find out. One thing that this whole thing has proved and shown is the sovereignty of AI models.

Noel (02:10)

Obviously in the US they've got all of the main labs. They've got Anthropic, OpenAI, Gemini — they're all in the US. One thing that's come out recently as well is OpenAI have got GPT 5.6 ready to release, but the US government have forced them to only give it to select users, and to really monitor what they're up to and what they can do with it before they give it to the general public. So I think we're going to see a lot more of that sort of thing going forward, unfortunately. Who knows when 5.6 will come out.

Katie (02:51)

Maybe as soon as we stop recording the podcast.

Noel (02:56)

Exactly. Well, that's when Fable 5 was announced as coming back — the evening after.

Katie (03:02)

Yeah. I think we had finished recording at, let's just say, 11.30am, and I think it was 6pm the same day it came out.

Noel (03:11)

Yeah. Well, today we're recording on... sorry, Wednesday — it's usually Tuesday, and that's usually when it comes out. So there are no shifts in the sand from last night. Well, I suppose there was, because they were telling us we've got another five days, so that's handy.

Katie (03:19)

Yeah, that was handy, because I can't remember when I got that yesterday, but I got the pop-up when I was using it and I was like, oh, cool.

Noel (03:40)

Yeah, I've been using it a lot this week with some of my projects. And obviously Clyde launched last week, which has gone really well — big, huge news.

Katie (03:47)

Very good. Yes! Big news! Big news! Huge news!

Noel (04:04)

As I record right now, there are still a handful of free lifetime spots left. They may be gone by Thursday, but if you want to grab one, check it out on the website — we'll leave a link. You can grab a free account and test out Clyde. And there are lots of updates I've added in this week.

Katie (04:24)

And Clyde is your AI agent, just in case anyone is listening for the first time and wondering, what the heck is Clyde? It is your AI agent that you have built yourself in response to Clawdbot... OpenClaw...

Noel (04:26)

He is. Moltbot was the other name.

Katie (04:47)

You wanted something that was more secure and more user friendly, especially for beginners.

Noel (04:53)

Yes, 100%. You do need some sort of technical knowledge to use those sorts of tools, unfortunately, whereas with this one it's getting easier. So although it was easy last week, it's got easier this week, because we now allow access to models from OpenRouter, which has hundreds of AI models to pick from. But obviously it's kind of difficult to work out: which one do I pick? Do I pick the cheapest one? How well is that going to perform?

Noel (05:22)

So this week — I know AI stuff loves a benchmark — I created the Clyde Bench. I'm going through and testing all of the models and how they perform with tasks within Clyde. So when users come in and say they want to use OpenRouter, we've got it in a rank system so they can see which ones perform best. So yeah, lots of awesome stuff going on in there.

Katie (05:49)

Okay, shall we dive in to this week's podcast episode?

Noel (05:54)

Yeah, let's do it.

Katie (05:55)

Okay, Noel, do you want to start us off? Because we were having this conversation, I believe, yesterday, and I was like, this is so interesting and I feel like it would make a really good discussion point. So do you want to start off with how we even started having this conversation in the first place?

Noel (06:04)

So with the World Cup, England played at 2am. So I was scrolling LinkedIn at 2am, as you do, and I came across a post that was basically showing an n8n AI agent, and they were saying that AI agents always confidently fail. So they all say, "Yeah, I've done it," and actually they haven't done their task.

What really annoyed me was that the image they put on there was actually created by OpenAI. It wasn't a real life image from n8n. So there's a bit of misinformation going on. And I went through the comments, and there were lots of people that were like, "I don't know why anyone uses them, they're just rubbish." And I was like, no, they're not — that's just one person's opinion, which has technically been falsified really with the AI image. Thankfully LinkedIn shows you in the top left hand corner that it's an AI image.

And then once I commented on that post, LinkedIn was like, oh, you're really interested in this, so I'm going to give you loads more. So clearly there's a problem with how people understand agents. And in this episode, we're going to go through a few things that can make them better — make your experience better.

Katie (07:45)

Yeah, I think so. Because why do they work for some people but not for other people? People can be using the same models, the same technology, the same plugins, all of these different things, yet someone gets amazing results and someone doesn't.

Katie (08:11)

And you think you're doing something wrong, but actually it could just be due to a lot of different factors, couldn't it?

Noel (08:21)

Yeah, there's so much that goes into it. Especially when you're on no code platforms like n8n, Zapier and Make, there is quite a bit of wiggle room in what they can do, and some platforms are better than others as well. But yeah, there are lots of factors that go into it.

Katie (08:24)

So the first thing really is to ensure that you've got the right platform for what you are wanting to build, what AI agent you want.

Noel (08:49)

Yes. I've tested all of them apart from Zapier, because their pricing is just ridiculous. I'm not paying 80-odd dollars a month just to try an agent — they're not worth that amount of money to me. Whereas platforms like n8n and Make, you can get started for free and use them on a far lower pay plan.

But with n8n, which was shown in that post — although they were the first platform to come out with agents, and they went viral in late 2024, I would say their agent systems are actually worse than the likes of Make.com. Their agents do confidently fail and misuse tools all the time. I don't know what goes on in the background, what's in the code of n8n, but something in there isn't quite right and has never really been right. I've had people even on a call say, "I'm just going to use my agent to book in our next call," and it just didn't. It came back with, "Yeah, I've done it," and we were like, no, no, you didn't. And we looked through it, and no, it hadn't. So although they were the first, I would say probably not the best. Sometimes it will work, obviously, but it's a bit tricky.

Noel (10:16)

Whereas with Make.com, what I find with them is they must have better guardrails within their agent system, so it's using tools more consistently. It just performs far, far better. And they took their time. They didn't dive into agents — it took them probably about five months to release their first iteration of agents. Thankfully they did take their time, because it actually worked really well out of the box. Even with a really terrible system prompt, it would still work out for you, which is pretty awesome.

Katie (10:44)

Yeah, that's really good to know. Okay, so first things first, just make sure that you are using the right platform.

Noel (10:59)

Yes. There are obviously coded options as well. So if you are a developer, or you're getting into that sort of thing, there are frameworks that you can use from Anthropic or LangChain that you can code with, and then it's kind of up to you to make those guardrails and things that make it work better. So coding wise, it is a bit more reliable. But for no code, which is probably what most people listening to this are going for, I would say Make is where to go.

Katie (11:28)

All right. Not sponsored by Make.com. Unfortunately! But Make.com, if you want to sponsor us, then we will take that sponsorship, thank you very much. Our email is hello@makeautomations.ai.

Noel (11:29)

Not sponsored by... no! Exactly. One day.

Katie (11:52)

And you can address it to myself or to Noel. Okay, so what would we do once we've ensured that we're using the right platform for what we want to build?

Noel (12:08)

The next, and probably most important, bit is picking the AI model, which is essentially the brain behind your agent. So we need to go off and pick the right one for your use case. Unfortunately, they're not all built the same. Each provider has different models that are available. ChatGPT has got GPT 5.5, soon to be 5.6. Anthropic have got Haiku, Sonnet, Opus and Fable. They all vary in intelligence and they all vary in capability, so you need to go off and find the one that's going to fit your particular use case. It's really quite crucial to get that right.

Katie (12:51)

How do you make sure that you are actually using the correct model for the agent you want to build?

Noel (12:57)

Fortunately, most of them, especially the big ones from OpenAI and Anthropic, provide benchmark scores for agent calling, agent logic and agent tool use. So basically, pick the one that's the smartest out of those to get yourself started. Yes, it will probably be more expensive to start out with, but at least then you can maybe dial it back and use an earlier model which has those sorts of agent capabilities, then test that out and see how it works. That would then help you reduce your cost over time.

Katie (13:34)

Okay, so we're ensuring that we pick the correct AI model to ensure that essentially the brain we're giving our AI agent is the best that we can possibly give it.

Noel (13:49)

Yes, definitely. And it doesn't just come down to intelligence either. There was something that popped up in the Clyde community this morning, which was: could we show within Clyde what the agent can do? Is it just text only? Does it do files? Does it do video? Does it do images? Depending on your use case, you need to take that into account as well, because some of them won't be able to read PDFs. So if you need that, then make sure you pick a model that will allow you to do that.

Katie (14:22)

Yeah, and I guess the same as well — if you want to create images, then you need to make sure that you're using a model that can actually generate those images.

Noel (14:32)

Yes, definitely. They all make it really clear as to what they do, so it's just making sure that you've picked the right one.

Katie (14:37)

Yeah, okay, that makes a lot of sense. Okay, what would be next on your list, Noel?

Noel (14:43)

So the next one is getting the right system prompt, and most importantly with that, having the right tool descriptions as well. I'll go through tools very quickly. When you have an AI agent, you can assign tools to it. That could be connecting it to Slack, it could be connecting it to Notion, things like that. Each of those connections, in an agent world, is called a tool.

So when you connect those together, you need to describe why that tool is there. What does it do? When should the AI agent use it? So when your query comes through, it goes: okay, they're talking about Slack, there's a Slack tool there, they want to create a message, I'll use the create a message tool — and it will go off and use that properly. Making sure they're defined will make sure the agent is using those tools, and it doesn't do that "Yeah, I've done it" when it hasn't done it. That kind of mitigates that issue. You can pull all of that within the system prompt, and platforms like Make.com allow you to edit those tool descriptions when you connect different applications in. So you can really make sure that the agent knows what to do, how to do it and when to do it. Really, really important.

Katie (16:12)

Yeah, and then what about the prompts?

Noel (16:16)

So the prompts are really important. They need to be in a structured format. You need to give it a goal. What's its purpose? What does it need to do? What is it expected to do? The kind of outputs you're expecting. What it can expect from you as the user is also good to put in — these are the kind of things you're going to get, and this is then what we want out of it.

Katie (16:46)

Yeah, you need to be really direct, don't you? Don't include any fluffy language or additional words or thoughts or "would likes". It's very direct, isn't it? So it's really clear and it can't be misinterpreted.

Noel (16:46)

Yes. I would say if you're using ChatGPT or Claude, you're technically prompt engineering by asking it and going through all that kind of stuff, but when it comes to agents, that's a bit of a separate skill to get nailed down. What I would say, though, is that ChatGPT and Claude are actually very good these days at creating those prompts for you. So if you have an agent and you kind of know what you want, you can ask it to create that prompt for you in the right format, and it will do that for you. They used to be absolutely abysmal at creating those, but these days, thankfully, they're actually pretty good, so they'll help you out a hell of a lot.

Katie (17:42)

Yeah, good to know. Okay, what else would you say is important when building out your AI agent to make sure it works exactly how you want it to work?

Noel (18:06)

So for this one, it's to have the expectation that you'll need to continually work with it, especially in the early stages. You need to put time aside to test it properly, then use it on a day to day basis, and basically learn how to iterate and make it better going forward. You might get lucky and absolutely nail it in one go the first time of trying, and it just works perfectly for the rest of its time. But I would always expect to have a little bit of a play and refine it over time. It's getting into that mindset of going: right, these are the results it gave me recently — am I happy with those? Is there anything I need to update within the agent? That's really quite important, and it also helps you in the long run, so you don't get any of these overconfident errors and things like that.

Katie (19:07)

Yeah, okay, that's really good to know, really helpful. And is there anything else that we need to be aware of?

Noel (19:11)

Yes, there's one more, and it's very, very important, this one. We've mentioned it a few times in multiple episodes, I think, but you need to understand when to use an agent. You may have come up with a use case and thought, "I'm going to use an agent," but actually it's important to learn when not to use an agent, or maybe when you just don't need one.

Katie (19:23)

Like when is it an AI agent and when is it an automation? Which we have talked about several times — we've done a whole episode on it, which we will link below if you want to go and check that out. Just in case you want to make sure that what you're building in terms of an AI agent isn't actually just something that you could use an automation for.

Noel (19:53)

Exactly, yes. Because if you're looking for consistent outputs, maybe an agent's probably not the way to go. Or if you're wanting to start off a workflow via a conversation, then that's probably an agent — you can't start an automation with a conversation, unfortunately. Not yet. Maybe one day.

Katie (20:20)

Not yet. Yeah, and it's probably going to be sooner than we think as well, let's be honest.

Noel (20:41)

Yeah, when we've all got implants from Elon Musk and stuff in our heads, and we can just think about it and it starts.

Katie (20:47)

No, it won't be that far away. Honestly, we could be sat here this time next year and we'd be like, oh yeah, you can start a conversation and create an automation. It's going to happen.

Noel (20:52)

Yeah, crazy times. I know it will. It wasn't on our prediction list, though.

Katie (21:13)

No, but it's going on my prediction list. Am I allowed to start a prediction list halfway through the year?

Noel (21:19)

That's cheating, isn't it? Although half of mine were done in the first month.

Katie (21:24)

Yes, okay, well that's going on mine. Okay, great.

Noel (21:30)

Fair enough, I'll add it to the list. But yeah, they're kind of the main points to get you that better experience, so you don't run into these sorts of issues with your AI agents going forward. So hopefully that's been helpful for everyone.

Katie (21:39)

And of course, you can use Clyde as your AI agent if you want to. There's a free Skool community, so if you do need some extra help or guidance, Noel is in there answering your questions and tech related woes, as I like to call them. Making sure that you are doing all of these things: using the right platform, picking the correct model, your prompts and your tools, being a bit patient and not expecting things to go right the very first time, and actually ensuring that what you are building is for an agent and couldn't just be an automation — as well as a host of other things.

Like Noel — if you've listened to the podcast before, you know Noel is a wealth of knowledge. We have lots and lots of episodes, so if this is your first time listening, please do go back and listen to some others. Noel gives all of his information for free. And like we said, there are just now a handful of those free Clyde spaces available, so go and grab them. You can either find it via the website or via the Skool community.

And of course, if you don't want to join Clyde, you can also come and join us on our LinkedIn group, which is AI Automations for Business. It's completely free, and in there you can let us know what you're building and how you're using AI in your business. We absolutely love hearing how you're using AI or automations within your business — honestly, the things that we've been told have just been absolutely fascinating and so, so interesting. So we love hearing it. Again, we'll put the link in the show notes, but otherwise you can just search LinkedIn for AI Automations for Business and you'll find us, and it's free to join.

But thank you so, so much for listening to this week's podcast episode, and we'll catch you for another one very soon.