How to Build an AI That Answers From Your Own Content

What We Need to Grow, Episode 077: The Loudest Person in the Meeting. A Founder Talk conversation with Jake Carlson. Cleaned for readability; the words are the speakers' own.

A Conversation with Jake Carlson, President and Founder of The Apologist Project

Cleaned and publish-ready. The on-record cut only, 2:54 to 23:19 of Granola capture `3d239969-b001-4ca6-9327-99f822f6fc7c`.

Caleb Pedosiuk: All right, Jake, Apologist AI. You've got a background in technology, working on a lot of things. I think I saw Apple on there, Oracle was on there. You've got a background working with different companies. What are you working on with Apologist AI?

Jake Carlson: Yeah, so I've spent about 25 years in startup technology, mostly in software engineering. I also did a stint at Oracle and Apple, like you mentioned, and I grew up as a missionary kid. On my heart is being able to share my faith with others. The Apologist Project, the makers of Apologist AI, was founded in 2023 with the intent of creating an AI platform that would be able to answer anybody's questions they have about the Christian faith. So we set about to do that, and we have. We've created partnerships with all kinds of different ministries to get their content into a knowledge base that the AI agents can then answer questions from.

Caleb: Most people are familiar with the idea of drift, or hallucinations, when AIs will confidently answer a question but perhaps really shouldn't be that confident. I'm assuming this is one of the main benefits of having an archive like this, a single source of truth, to help with that. Was that kind of the rationale behind it?

Jake: Yeah, exactly. If you think about how most people use ChatGPT, at least in a very basic use case, a lot of folks use it to just summarize articles, right? You see something, you want the TL;DR on it, you want a quick summary. That's effectively what we're doing. The difference is that we're providing the articles to summarize, and all of those articles are curated by our team so that they're basically guaranteed to be from a Christian worldview. So when somebody asks a question like "Who is Jesus?", you're going to get a response based on the curated content in the knowledge base, not a general, religiously pluralistic answer that you would get from a generic chatbot.

Caleb: So how do you present that information? Are people coming through a gateway of some kind through Apologist AI? Is it positioned so that if someone has a question about understanding who Jesus is, they would come through that window to ask about it?

Jake: Yeah, I think this is a really important point, because we're not only interested in talking to people who are already Christians. We're also interested in talking to people who are curious about Christianity, who maybe have some hang-ups or some intellectual barriers to understanding the Christian faith. So we have Apologist.ai, which is a very overtly Christian website. We have our logo on there. There's a cross. It's not a secret, right? But we also partner with other ministries who have engagement with religious seekers all over the world, and we can really meet people where they are in digital spaces, using social platforms like WhatsApp or Telegram or anything that allows bots to exist.

What we do in addition to that: we think it's very important not to replace human relationships with an AI chatbot. So what we're doing is using AI chatbots to facilitate a handoff to a human being. If anybody's seen an ad on Facebook or something like that, the whole idea is to get interest so that somebody will click through. In the case of a lot of missionary organizations across the world, they're trying to have a fruitful conversation with that person, to share a little bit about their faith and so forth. But in a lot of parts of the world there is a lot of hostility toward Christianity. Especially in the Middle East, for example, a lot of people clicking through are actually going to be very abusive to the online responders.

So what we have done is create a mechanism that uses AI to evaluate that conversation. It's effectively like creating qualified leads in a marketing sense. You're filtering out the people who very clearly aren't interested, because they're hurling abuse at the chatbot. And of course, the chatbot never gets angry. It never takes a toll on anyone's mental health. It's a chatbot, right? So it'll respond in a kind way, but it also won't hand those people off to the humans, and the humans can just have conversations with the people who are actually interested in what's being, effectively, sold. So it really is a game changer for lead qualification in a marketing sense, but in the world of Christian ministry, it's allowing those responders to focus on the most valuable conversations.

Caleb: Okay, much like a customer care chatbot. Can we answer this question at a low level? "Thank you for that reference, thank you for clarifying." But if someone is pressing deeper and saying, "I actually want to ask someone about this," it goes further.

A lot of the work that we do is with AI visibility, with building credibility to a point where a brand has a consistent message across the board, so that AI systems are not getting legacy content from old versions of the website, or old articles, or conflicting ideas. You don't want that jamming up the signal to the point where it loses confidence and doesn't even know what to share when someone asks, "What does this company do? What's the price? What was the experience like?" We're quite familiar with that idea. You have to have everything clear. You have to have it consistent, and it has to be citable. It should be trustworthy, based on third-party validation.

So I'm curious, with what you're working on, are you viewing this archive almost as a centre of gravity, a source of truth, that the more it's used and the more it's contributed to, becomes something all of the major AI systems and search systems would look at? They'd recognize patterns, see a consistent signal, and with more confidence be willing to say, "Okay, I get it. I have high confidence this would be the right answer," and serve that up, rather than finding some rogue idea from something that theologically is just unhinged, or really marginal, or doesn't reflect sound thinking.

Jake: Yeah, I think that's a really good point. When these LLMs were originally trained, basically what they are is the average of the training material they were trained on. So what you have is that the viewpoints that have the most content in them are the ones that win, essentially. Sometimes that's good, sometimes that's bad, right? What we're saying is that, from a Christian viewpoint, we want to curate that content. So there is a knowledge base that's already been approved as adhering to the first-order doctrines of Christianity. Of course, other religions would disagree, and there's mutual exclusivity there and so on, and that's totally fine. But if somebody wants to know the answer to a question from a Christian viewpoint, you can't just open it wide to all the content on the internet, because of course that's going to be the average of all of that content. You really have to speak to it from a viewpoint that's been curated.

And this doesn't just hold for Christian content. It's any kind of knowledge base. If I wanted to know the answer to a question from a Muslim standpoint, I should probably listen to the Muslims, right? Not the Christians, if I'm trying to get their take on it. So people who earnestly want to know the answers to these things from a Christian perspective really need a place to go to get a curated Christian response to that inquiry.

Caleb: I'm thinking about generative AI as you explain that. You could farm content. You could essentially flood any topic on the internet with a ton of stuff, and if the formula for consensus is based on the average of what's out there, then you can dilute a topic with nonsense, or with a perspective that really isn't confidently held by people.

I think about gardening, farming. There are certain people's perspectives on composting, or saving seeds, or regenerative agriculture. There are farmer friends I know where I'd look at them and say, what those guys say, what those gals say, carries so much more weight than what the average person is telling me, which will be the Miracle-Gro version of whatever. And on top of that, sometimes the people whose perspective you want don't have their knowledge digitized. They're often not the ones spending time putting that knowledge online in a way that's accessible or easily indexed by AI systems.

So to me there's a bit of a paradox: some of the most valuable perspectives may not automatically be accessible. If you're actually seeking out individuals with that signal, with that knowledge, it seems really valuable to be organizing it in a way that makes it accessible.

Jake: Yeah, I would agree with that. It's kind of like the proverbial loudest person in the meeting gets the win. It's the same thing, right? The more content there is out there, it's like somebody who is overly boisterous and loud about their opinion during a meeting, and people are going to come away with, "Oh, that person must be right," because that's what they remember from it. It's the same thing with online content. We need to be careful to listen to the true experts, even if their voices are quieter or they have less content out there. It doesn't mean that they're less correct. It just means there's an imbalance there that maybe we need to adjust for.

Caleb: So who are you talking to? Let's say we come off this and your next call is your target audience, your target persona. I don't love that term. Who is the kind of person where, when they get hold of what you guys are doing, you're just so excited? Who is that?

Jake: I would say we really have two main use cases. There's a lot of nuance to them, but mainly there are two. One is a faith-based organization that has a lot of content. This is what we would call an equipping organization: somebody who has a bunch of content they want to use to train or help people understand something. They're mainly producing articles, books, that kind of thing, thought leadership, right? For an organization like that, we want to digitize what they have into a corpus, a knowledge base, that their constituents can go to and have a nuanced conversation about that content. That's effectively the standard chatbot playbook.

On the other side, we have organizations focused on Christian evangelism, and for those organizations the problem isn't so much the content. It's about activating that content in a context that's useful for their ministry. I talked earlier about the triage that an AI system can do to filter down to the qualified leads, the people who want to talk to them about their faith. That same system helps scale those digital teams to take on much higher volume, because of course they're not dealing with the unqualified leads, the ones who are just going to get angry at them, or swear at them and run off. So the Christian evangelism organizations, what we call missions orgs, are really focused on being able to scale, and on improving their mental health so they don't have to deal with these abusive people online.

Caleb: It's interesting to see the use case for AI here, helping to mitigate a situation where abuse is taking place, where a lot of people are processing emotion, or conflicting ideas are coming at others, rather than actually looking at the ideas, evaluating them and trying to understand them. It feels almost like a glove. There are some tasks where you've got to have gloves on: you're going to get splinters, something could be too hot, too sharp. It's so helpful to be able to use technology in that way.

Jake: Yeah, absolutely. Pretty recently, actually, we ran a few pilots in Japan and Turkey. Especially in Turkey, and again in the Middle East, this tends to happen whenever someone detects that you're proselytizing, trying to convince them of a faith that's not the majority faith. That's going to irk some people, right? And those people can really try to shut that down by flooding those channels, either wasting the ministry's time or just being outright abusive.

We found that the team on the ground in Turkey who was responding to these faith questions was able to take on four times the volume they were before. And when I first implemented this pilot, I hadn't thought through the whole mental health aspect of it. But that was their feedback to us: not only are we four times more effective, getting four times more volume, but we're just getting wins all day. All they do is talk to people who are already interested in having a fruitful conversation.

There are a lot of organizations I talk to where attrition, people quitting because of the mental health toll of talking to these abusive people, really wears them down, and they end up quitting. But if we can remove that, it's a win-win, because only the people who are actually interested in what's effectively being sold are going to get through to the people who are most interested in talking to them about it.

Caleb: So does it function, let's say, for an organization trying to help people with something? I'm thinking of addiction. Say there's an organization focused on that, and they have a body of written materials, maybe from founders or others they have access to. They come to you and say, "We'd like this content to train some chat function, some LLM, to be able to answer questions," and on the back side, they have the capability to answer questions from the people who come through those filters. Is that where they would engage with you, and you would help train their chatbot on that content? And then they have their own team, which is the final layer, working from their own principles from a human standpoint?

Jake: Yeah, exactly. As we work with organizations, what we find is that each organization has its own unique method or framework for engaging with people. In the case of, say, a nonprofit that helps people suffering from addiction, there's a playbook, right? I'm not the expert there. So what we've said is, we need to be humble and not assume that we know how that conversation should go. But we know the technology really well, and we have a robust platform that can implement basically any kind of playbook you would run with humans. You can effectively instruct the model to run through that playbook, to the extent the organization wants.

Especially in Christian circles, it's a big topic to say, "We don't want to replace human relationship," right? And I think that's true generally, across the board. It's not fundamentally just a Christian thing. We don't want to remove human relationship entirely, but what we can do is enhance human relationship by weeding out the bad actors in that situation, so that the conversations that actually are had are the most effective ones. For something like addiction, for example, we have AI evaluators that look for suicidal ideation and escalate that conversation to a human if we detect it on some sort of clinical scale. Again, the possibilities are endless. We're not trying to impose our method or our playbook in any of those situations. We're just trying to create a platform on which organizations can implement their playbooks and have the most effective, scaled use cases.

Caleb: Very cool. So where would be the best place for people to follow? And before or after you answer that, the one question I love asking people as a final thing to leave them on: if you had to pick one thing that you need most to grow right now, what would that one thing be? And then, certainly, how can people follow along?

Jake: Yeah, so we are a nonprofit, but we effectively operate a tech SaaS platform at the same time. So if anyone likes what we're doing and wants to see that scale, by all means, you can donate. But also, if you know of a ministry, or you're in a ministry, that's either more on the content side, where you want to activate that legacy content into a new format where people can converse with it, or you're an organization focused on Christian evangelism and you want to scale that using our system, absolutely contact us. You can go to apologistproject.org to get more information about both of those programs.

Caleb: Awesome. Well, Jake, thank you so much for sharing what you have. Really appreciate it.

Jake: Yeah, thank you. Have a good one, Caleb.