When I Ran the Checker, the Checker Was Wrong First

Pillar: P1 Builder Exploration Log (B Track · First Person) · Funnel: TOFU slug candidate: blog-p1-checker-failed-first Written: 2026-07-27 · Length: Approximately 4,300 characters

🔵 Speaker·route confirmed (2026-08-11 · §0-0-1-C dual route)

• operating_track: INDEPENDENT_EDITORIAL — First person narrative 11 times measured. Changing to company voice invalidates the narrative (confession of personal error is the point).
• cta_route: FOUNDER_AI_READINESS — Explicit choice according to §0-0-1-A as CTA is included.
• entry: content_buildlog_checker ← Former content_evergreen was a shared bucket of 22 pieces and could not be attributed.
• Measurement separation: Save·profile visit·conversation = Builder log learning indicators (MKT OS combined ❌) / CTA pass funnel_lead_submit_success = MKT OS lead (combined ⭕).

---

1. Ran the Checker on Twelve Manuscripts

We decided to attach three types of footnotes to every piece we publish externally: the sources and dates checked, numbers not used in the text, and the method of creation.

The second item is a bit unique. It’s about disclosing what was not used rather than what was. It specifies what numbers were omitted due to lack of verification and why. Since anyone can claim “unverified data is not included,” we thought it better to show the actual list of omissions.

The problem was that we didn’t count whether this rule was followed for each manuscript. So, we created a checker. It was a simple tool that scanned the manuscript folder to indicate whether all three footnotes were present.

The results were as follows. Only two out of twelve manuscripts had all three footnotes, and six had none.

I recorded these numbers as they were. I also noted that even manuscripts marked as having passed the gate lacked footnotes. I concluded that the gate wasn’t catching footnotes.

---

2. Upon Opening the Manuscripts, Footnotes Were Present

After writing that conclusion, I opened each manuscript to fill in the footnotes.

Three of them already had all the footnotes.

The reason my checker missed them was as follows. One used “things not used in this text” instead of “numbers not used in this text.” Another used “referenced materials” instead of “reference materials.” It was a matter of spacing. The last one used a completely different title, “Notice Regarding Numbers.”

All three adhered to the rules in content. Only the format varied.

The gate was doing its job. The mistake was in my checker.

This sequence was a bit painful for me. I ran the checker, got the numbers, drew conclusions from those numbers, and only after embedding those conclusions into the record did I open the manuscripts. Had I opened the manuscripts first, the checker’s error would have been revealed in five minutes.

---

3. But Upon Opening the Manuscripts, Something Else Emerged

As I continued to read through the manuscripts to fill in the footnotes, I found something the checker couldn’t have seen initially.

In one manuscript, there was this sentence:

“○○ research warns: The more frequently AI is used, the lower the critical thinking. A statistically significant inverse correlation was found in a study of ○○○ people.”

It mentions an institution, sample size, and statistical significance. To the reader, it appears to be verified research.

We have a list of external research that can be cited. It includes only those we’ve opened and cross-checked for numbers and context. This research was classified as **“known by institution name only”** on that list. It was not to be cited. The sample size was also a number not on the list.

Honestly, at that moment, I wanted to just write “Source: ○○ research” and move on. Filling in the footnote box would complete the format.

Faced with that temptation, I reopened the list, and it was marked as “unverified.” I learned once again that noting a source in a footnote and having the right to cite it are different matters. Covering it with a footnote makes it appear verified.

I removed the sentence entirely. In its place, I included something we directly observed. It was about how interview summaries don’t capture the points where people trail off. I didn’t include numbers, and noted the omission and the reason in the footnote of that text.

On the same day, in two other manuscripts, internal storage names and tool names were found directly in the text. These were also things the checker couldn’t see.

---

4. Automation Checks “Presence,” Humans Check “Qualification”

To summarize:

	What Automated Checks See	What Humans Need to Open to See
Footnotes	Presence of title strings	Content adherence to rules
Citations	—	Is this source citable?
Identifiers	—	Is this a name that shouldn’t be exposed?

Automated checks look at presence or absence. They are fast, comprehensive, and tireless.

What humans need to check is qualification. Is this source of a citable grade? Is this name okay to be exposed? Is this number an example or a measurement? These are not judgments that can be made with string checks.

And today, I got the order wrong. I confirmed the automated check results as facts before human verification. It should have been the other way around.

---

5. So What Did I Change?

I did three things.

First, I didn’t erase the wrong conclusion but left it as a correction history. In the place where I wrote “the gate isn’t catching footnotes,” I attached the fact that it was wrong and why it was wrong. If erased, the same mistake will be made again.

Second, I revised the footnotes. Nine manuscripts. Six had no footnotes at all, so I wrote new ones, and three had content according to the rules but varied titles, so I standardized the format. Out of the twelve scanned, eleven are now up to standard, and the remaining one was excluded as it won’t be published.

While filling them, I noted the actual omissions in each manuscript’s “numbers not used.” Examples include:

• The ratio of eight out of ten interviewees declining is an example for explanation, not an actual measurement
• The number of people coached is an expression of scale, not an analysis conducted with a defined sample
• Performance of design improvements was not measured

The third item is particularly so. A piece about improvements feels lacking without improvement numbers. It makes one want to include them. Since they weren’t measured, they can’t be included, and it’s right to note that they weren’t measured.

Third, I fixed the checker. However, this time I also noted what the checker cannot see. Citation qualifications and identifier exposure need human review.

---

6. Heard a Similar Story

Recently, I attended a public session of a team different in scale and field from ours.

The lasting impression was that we ended up in a similar place. It was about how no matter how far automation is pushed, there remains a need for human verification.

I thought we learned this by stumbling through it ourselves. Knowing that others reached a similar conclusion made me realize it wasn’t a rule born from our unique circumstances.

(This paragraph is based on my memory of what I heard. I did not transcribe the statements verbatim, nor did I include company names, speakers, timing, or numbers. The reasons are noted in footnote ② below.)

---

7. If You Do One Thing Today

If you’re running a checker, it might be good to check if that checker has ever been wrong.

The method is simple. Pick just one item that the checker marked as “pass” and open it yourself. Conversely, do the same for one marked as “fail.” You don’t need to look at ten items. One will reveal what the checker sees and what it misses.

Today, I went in the opposite order and left a wrong conclusion in the record once.

---

Footnotes

① Sources Referenced (Checked on 2026-07-27)

• No external research or statistics cited. The narrative of this text is based on the 2026-07-27 internal manuscript audit log (R0·first-hand experience). The results of the check, correction history, and actions are recorded in our publication queue documents and execution ledger.
• The “list of external research that can be cited” mentioned in section 3 is our internal operating rule. It is not a publicly available standard.

② Numbers Not Used in This Text

• 🔴 Institution name, sample size, and research year of the study omitted in section 3 — This study was classified as “unverified” in our citation list, so it was omitted from the manuscript, and mentioning it here would repeat the same mistake, so it was masked with ○○. We will disclose it once we secure and cross-check the original.
• 🔴 Company name, speaker, original statement, and specific timing of the public session in section 6 — We only have an automatic text conversion record and have not yet verified the original. Since the original has not been verified, names and quotes were not included. Instead, it was noted as my memory of what I heard, without numbers and timing.
• Number of manuscripts — The number of manuscripts in the text is the actual number of files in our manuscript folder. Twelve scanned manuscripts (eleven candidates for publication + one not to be published), two judged complete by the checker, three additional found upon manual review, nine revised (six new + three format adjustments). However, this is our folder situation and not an industry metric, so it wasn’t converted to a percentage (%).
• 🔴 These numbers were also wrong once. In the draft, the title said “11 manuscripts,” the text said “twelve,” and section 5 said “eleven.” It was pointed out during pre-publication verification that a text advising to recount before finalizing numbers didn’t have its own numbers aligned, and it was corrected.

③ Method of Creation

The draft was written using AI and verified and edited by humans. The checks, corrections, and removals described in the text were actually performed, and the records are kept in our ledger. No generative images, audio, or video were used.

---

CTA

If you have set any inspection rules for the texts you publish, you can pinpoint what those rules actually catch and what they miss with a 2-minute diagnosis. Feel free to answer as much as you like. https://www.star-t.io/ko/ai-readiness?entry=content_buildlog_checker&utm_source=blog&utm_medium=organic&utm_campaign=buildlog_2026&utm_content=blog-p1-checker-failed-first