Blog Content Quality Regression Systematic Fix

TITLE: How a Low Threshold Degraded Six Weeks of Blog Quality

TL;DR

• structural_qa() threshold dropped from 75 to 30, bypassing quality gates for 6 weeks
• LLM silently generates polished but shallow content when input structure is incomplete
• Technical vocabulary density dropped ~40% without any errors or crashes
• Fix in two steps: restore threshold + add CI validation in .github/workflows/blog-publish.yml
• Quality thresholds should live in CLAUDE.md as P1 rules, not as unversioned config

Summary

Six weeks of degraded blog quality came down to a single line in scripts/auto-publish-blog.py.

During a visual redesign rollout, a developer temporarily lowered the structural_qa() quality gate score from 75 to 30 to meet a deadline. The plan was to restore it afterward. It never got restored. The result: six articles in obs/04-content-pipeline/_inbox/ passed the degraded quality check and were published with technically shallow content—complete-looking, well-formatted, and readable, but missing specific commands, real-world examples, and logical depth.

The most insidious part is that no errors occurred. LLM-based generation doesn’t throw an exception when input structure is incomplete; it simply generates coherent-sounding output from whatever context it has. The missing sections like ## 核心技术细节 never appeared, but the articles still compiled and published. We caught the issue only through manual review comparing vocabulary density across before/after article sets, discovering a ~40% drop in technical term density.

The surface-level fix was obvious: restore the threshold. But the real root cause runs deeper—quality thresholds were treated as run parameters rather than quality contracts. They lived in a script file with no change control, no monitoring, and no automated guardrails.

The full remediation involved two steps. First, restore THRESHOLD = 75 and backfill the six affected articles with their missing sections. Second, add a CI-stage validation step in .github/workflows/blog-publish.yml that checks inbox files for required structure before any generation step runs. This ensures the threshold cannot be bypassed silently again.

Key Takeaways

• If you lower a quality gate for any reason, block the merge immediately with a P1 tracking issue.
• If you generate content with LLMs, treat missing input sections as silent failures, not harmless omissions.
• If you manage a content pipeline, enforce structural requirements in CI, not just in scripts.
• If you rely on threshold values to control quality, document them in your project standards as immutable rules.
• If you notice gradual quality decline without errors, instrument vocabulary density and structural completeness as automated metrics.

Read the Full Article (Chinese)

This is an abstract. The full technical walkthrough, including the complete structural_qa() implementation and CI validation logic, is available in the original Chinese article.