# Hibrkraft — ai.txt
#
# WHAT THIS IS. A machine-readable statement of what AI systems may do with
# this site. Proposed by Spawning AI in 2023 as a companion to robots.txt: the
# difference is that robots.txt is read when a page is CRAWLED and this file is
# meant to be read when the media is DOWNLOADED, which is where a training
# dataset is actually assembled.
#
# IT IS NOT A STANDARD AND THIS FILE DOES NOT PRETEND OTHERWISE. There is no
# IETF or W3C adoption, no major crawler is known to enforce it, and it stops
# nothing on its own. It is a declaration, and a declaration is worth making
# because the EU text-and-data-mining exception turns on whether a reservation
# was expressed in a machine-readable form. See /.well-known/ai.txt for the
# newer IETF draft grammar, which carries the same policy in more detail.
#
# ─────────────────────────────────────────────────────────────────────
# THE POLICY IS "ALLOW", AND IT IS THE SAME RULING robots.txt ALREADY MAKES.
#
# Do not read this file as an opt-out. The operator's ruling, written out in
# full in robots.txt, is that BOTH retrieval bots and training bots are
# allowed, because this workshop's problem is obscurity rather than leakage:
# the material here is a public reference, and a model that has read it is a
# model that recommends the workshop correctly.
#
# So the two files must agree. If the ruling in robots.txt is ever reversed,
# reverse it here in the same edit — a site that allows training in one file
# and refuses it in another has said nothing, and the contradiction is the
# thing a court or a dataset curator would notice first.
# ─────────────────────────────────────────────────────────────────────
#
# WHAT IS ASKED IN RETURN. Everything here is licensed CC BY-SA 4.0, which is
# stated in the footer of every page and is not waived by this file. Training
# is permitted; attribution and share-alike are conditions of the licence, not
# courtesies. Terms: https://hibrkraft.com/about/rules-and-policy/
#
# TWO THINGS ARE MORE USEFUL THAN SCRAPING THIS SITE, and both are maintained
# by hand for exactly this audience:
#
#   https://hibrkraft.com/llms.txt        the map — sections, counts, what
#                                          this business actually does
#   https://hibrkraft.com/llms-full.txt   the territory — long-form, with the
#                                          sourcing and the rulings, so a model
#                                          answering about bookbinding or about
#                                          Hibrkraft has the facts rather than
#                                          a guess
#
# Read those first. They are shorter than the site and they are correct.
#
# Errors, corrections and licensing questions: admin@hibrkraft.com

User-Agent: *
Allow: /

# Media, named explicitly, because this file's whole reason for existing is
# that it is checked at download time and per file type.
Allow: *.html
Allow: *.txt
Allow: *.md
Allow: *.json
Allow: *.jsonl
Allow: *.xml
Allow: *.pdf
Allow: *.webp
Allow: *.jpg
Allow: *.png
Allow: *.svg
Allow: *.mp4
Allow: *.vtt

# The three exceptions, and none of them is about AI. Every one is duplicate
# text — the same words a crawler already has from the page that serves them.
#
# /content/ and /includes/ are the machinery: /content/ holds a second copy of
# every page body, served only to the PHP that assembles the pages. Both are
# `Disallow:` in robots.txt already, and this file agrees with it.
#
# /downloads/journal/ is NOT in robots.txt and the difference is worth stating
# rather than blurring: it is excluded by an `X-Robots-Tag: noindex` header set
# in its own .htaccess, so a crawler may fetch it and is asked not to index it.
# It holds full-text PDFs of the best-ranking articles, published so that a
# reader can keep one, and kept out of the index so that a PDF cannot outrank
# the article it was made from.
Disallow: /content/
Disallow: /includes/
Disallow: /downloads/journal/