The Story of Creating Team Mirai's AI Fact Checker (Evidence Checker) in 2 Days - How Should Japan Deal with Cognitive Warfare?

Introduction

Team Mirai created an AI fact checker (hereinafter referred to as "checker").

This is to somehow address the major problem of misinformation, which is shaking democracies around the world, and to avoid the division of the world even a little.

As Komeito has also announced its intention to use it, it is a tool that can be used across political parties, and we believe that it is an epoch-making tool that has the potential to improve the quality of democracy.

Below, I will write about my self-introduction, the background to creating the checker, what the checker is trying to address, technical explanations, what it was like to actually run it, and what we will do in the future.

Self-introduction

I am a Theravada Buddhist and have no interest in political parties or ideologies.

It would be good if I could be of service to all living things.

This time, I participated because I thought that developing with Team Mirai would be the most useful to the world, and I have no intention of siding with a specific political party.

Background to creating the checker

Originally, I wanted to do something about the spread of hatred and division in the world.

The Evame site where this article is posted is also a multilingual blog site that I developed, where articles written by users are automatically translated into multiple languages. I am developing this site because I thought that sharing knowledge and stories globally could avoid division.

Amidst all of this, Mr. Yasuno proposed the checker, and I thought that it would certainly be useful to the world.

Also, since I had previously created a Buddhist AI that answers based on the scriptures of the Pali Tripitaka, I started development because I knew that I could do it using OpenAI's VectorStore based on that experience.

As expected, the function itself was completed in 2-3 hours, but it took 2-3 days to release due to authentication around the X API and GCP permissions, and adjustments to the prompt to improve accuracy. As an aside, Team Mirai was a real startup, so Mr. Yasuno was initially handling the permissions around this area.

What the checker is trying to address

The threat of misinformation spreading through social media is already a major political risk and has become a new battlefield called "cognitive warfare."

In Romania, there is a strong suspicion that influencers were bought by foreign countries in the 2024 presidential election and the results were manipulated, and the presidential election was actually rerun.

In the United States, it is said that social media is fuel that boosts hatred, making policy compromises difficult and sustaining the risk of mobilizing violence.

With conventional human monitoring, it was practically impossible to check social media 24 hours a day. Also, the problem of inaccurate information spreading from the dissemination of misinformation to its correction is serious. Team Mirai's fact checker was developed as a concrete solution to respond immediately to this issue.

Technical explanation

This system operates in 5 steps.

1. Register reliable information such as manifestos in the system

2. Automatically collect posts with specific keywords (ex. Team Mirai, Takahiro Yasuno) on Twitter/X

3. Truth determination by comparing with reliable information using RAG using OpenAI's GPT and VerctorStore

4. Notify Slack if suspicious information is detected

5. Humans make judgments and take action as necessary.

It is important that humans make the final judgment. AI only plays the role of selecting a large amount of information, and humans are responsible for taking action.

Since it is open source, I would like people who want to know more deeply to take a peek at the code from the following.

GitHub - team-mirai-volunteer/fact-checker: Twitter/X投稿をリアルタイム監視し、AI powered ファクトチェックで政党関連の誤情報を自動検出‧ Slack通知するシステム

Twitter/X投稿をリアルタイム監視し、AI powered ファクトチェックで政党関連の誤情報を自動検出‧ Slack通知するシステム - team-mirai-volunteer/fact-checker

github.com

What it was like to actually run it

1. I received a comment from an expert that "what the parties involved do is not called fact-checking."

   This is as you pointed out, and it was subsequently decided to change the name to Evidence Checker (I'm too busy with the election to change the settings, though...).

2. Team Mirai is not yet well known and not as much misinformation flows as with famous political parties, so the effect was limited.

   The checker that Team Mirai is running only investigates based on Team Mirai's manifesto, etc., so it cannot investigate information on other political parties, nor can it investigate general information. I think this is an issue that needs to be addressed.

   For example, in Taiwan, there is a public-private partnership fact-checking organization called Cofacts, and there is a mechanism in which AI and humans cooperate to check 3,000 posts a day via the LINE bot with AI + humans.

   https://news.yahoo.co.jp/expert/articles/ea08db1d092604950430e920febc63682cd1b6b9

   I would like to do something like this in Japan as well.

3. I thought that what is actually creating division is not so much clear misinformation as malicious remarks.

   I can't say for sure because this is not a quantitative analysis, but when I actually look at the rhetoric during the election, there were many malicious words flying around that I couldn't clearly judge as misinformation, such as "〇〇 is incompetent" or "Japan has become bad because of 〇〇." In the future field called "cognitive warfare," I believe that it is also necessary to counter such rhetoric.

What we will do in the future

"Cognitive warfare" is an issue that will greatly affect people's lives not only in this election but also in the future.

After the election, I would like to counter this within the framework of Digital Democracy 2030.

デジタル民主主義2030

デジタル民主主義2030プロジェクトポータルサイト

dd2030.org

I don't know exactly what to do, but referring to the example of Taiwan, I think it would be a good idea to counter malice with memes.

In Taiwan, they are countering misinformation with an approach called "humour over rumour." For example, when a rumor spread that "Toilet paper will run out!", Premier Su of the government posted a cartoon of "shaking his butt," countering the misinformation with humor, saying that there is only one butt and there is no point in hoarding it.

How Taiwan used memes to fight pandemic rumours

Audrey Tang, Taiwan’s Digital Minister, shares three ingredients behind Taiwan’s successful pandemic response.

govinsider.asia

In Japanese style, would it be "laughter over malice"?

If there is anyone who is interested in making the world a better place with laughter, I would like them to participate in Digital Democracy 2030. It would be very helpful to have not only engineers but also comedians and video creators.