robots_meta Plugin — SEO Indexing Control

Date: 2026-05-18

Problem

The site has 500+ pages: ~560 blog posts across 12 layouts, plus 22 zen PWA apps, tools, subscription pages, feeds, retired pages, and other scaffolding. Without explicit indexing control, Google sees everything — wasting crawl budget on pages that hurt SEO (thin content, duplicate, non-canonical).

Approach

A Jekyll post_render hook plugin that injects <meta name="robots"> into every rendered HTML page using a priority chain so you can express rules as simply as possible:

  1. Front matter robots: — per-page override, injected verbatim into content=""
  2. noindex_patterns — glob patterns that always force noindex, regardless of layout
  3. index_layouts / index_paths / index_patterns — any match → index
  4. default: noindex — fallback for everything not matched

Config location

_config.yml under robots_meta: — the full block lives there so it’s version-controlled and visible in one place.

robots_meta:
  default: noindex
  index_layouts:    [post, bpost, bpostnoads, bookpost, ...]
  index_paths:      [/]
  index_patterns:   [/categories/**, /tags/**]
  noindex_patterns: [/zen*/, /zen*/**, /apps/**, /token/**, ...]

Why default: noindex

The site has more non-blog pages than blog pages by count. Opt-in indexing (list what should appear in Google) is safer than opt-out (list what shouldn’t). A new section added to the site is silently noindexed until explicitly added to index_layouts or index_patterns.

Per-page override

Any page or post can override with front matter:

---
robots: noindex          # keep this post out of Google
robots: index, follow    # force index even if pattern would noindex it
robots: noindex, nofollow, noarchive   # verbatim — any robots directive works
---

Adding new sections

Implementation notes