# Backyard Zoology thematic analysis

This folder is independent of `facebook-archive/`. Open `index.html` for the third overview or `reading.html` for the written thematic analysis. The archive’s Post list and Image wall both link to Themes. All three views work offline; theme results open the existing post reading copies and reuse their thumbnails.

The current snapshot covers 513 retained posts. It includes locations, animals, recurring topics, explicit season mentions, a separate publication-season index, and eight curated narrative sequences. Exact text evidence distinguishes the Page’s words from shared-post words. Counts are posts with mentions, not sightings or independent animals. Multiple labels can apply to a post; broad animal groups include their more specific labels. Rule matches can include jokes, comparisons and ambiguous phrasing. No inference is made from images.

## Refreshing after an archive update

Run from the project folder:

```sh
.venv-facebook/bin/python scripts/analyse_facebook_themes.py
```

This command reads local `facebook-archive/posts.json` and makes no Facebook or other network requests. It preserves the existing 5 GB free-space reserve. It writes only this analysis folder; the archive collector does not run it automatically or overwrite this folder. The existing analysis remains browsable until you choose to refresh it. Keep both folders together so post and thumbnail links continue to work.

The `posts/<post-id>.json` cache records a hash of the Page/shared text and a signature for each text rule. An unchanged rerun reuses every text classification and leaves unchanged per-post files untouched. A new or changed post evaluates its rules again. Editing one theme rule reevaluates only that rule for every post, preserving the other cached results. Changes to dates, titles, URLs and thumbnail paths update the view without rerunning unchanged text rules. An analysis engine change should increment `ENGINE` in the script to invalidate old rule results.

Posts absent from a later catalogue are excluded from the current overview, while their old caches are retained for possible reuse. The build report records their number. The text hash includes both Page and shared-post text, so shared text changes invalidate that post’s classifications too.

## Reusable analysis inputs

- `taxonomy.json`: editable theme labels, descriptions, literal aliases, optional regex `patterns` and optional regex `exclude_patterns`. Multiword aliases also match hyphenated words and compact hashtags. Animal `parent` IDs propagate evidence into broader groups. Theme IDs should remain stable so existing links keep working.
- `story-arcs.json`: curated story descriptions and selected chapters, with post IDs, source (`page` or `shared`), an exact anchor phrase, interpretive note, and reviewed text hash. A connected episode is distinguished from a sequence of recurring encounters. New posts are automatically indexed by text themes; extending a curated story requires selecting and reviewing additional chapters.
- `overrides.json`: optional, reviewed per-post additions and removals, separate from automated results. These are bound to the reviewed text hash. A changed text hash makes an override inactive until reviewed.

Story anchors are checked on every build. A changed reviewed text hash, missing anchor or missing source post excludes the affected chapter and adds a review item to `build-report.json`. The other chapters and automatic text themes continue to work. When accepting an updated chapter, review the full new text, update its note/anchor if needed, and copy the new `text_sha256` from that post’s analysis cache. Do not just replace a hash without reviewing the text.

To override a classification, use this shape in `overrides.json` (replace the illustrative fields with an actual ID, hash, theme and literal quote):

```json
{
  "schema_version": 1,
  "posts": {
    "POST_ID": {
      "text_sha256": "COPY_FROM_posts/POST_ID.json",
      "remove": ["animals-flies"],
      "add": [
        {"theme_id": "topics-citizen-science", "source": "page", "quote": "EXACT SOURCE WORDS"}
      ]
    }
  }
}
```

A removal affects that specific label. To remove an animal-group label supported only by a rejected child match, remove that broader label too. Additions require evidence actually present in the declared source; missing quotes are flagged. Overrides are not edits to archived post text.

## Generated outputs

- `index.html`: searchable theme overview, theme intersections/unions, counts, selected story chapters, linked post cards, pagination, and per-card evidence. URL fragments retain filters for bookmarks and direct theme links.
- `reading.html` and `report.md`: written thematic interpretation, count tables and story summaries. The interpretation is an editorial reading, separate from automatically counted mentions.
- `analysis.json`: current source text snapshots, memberships, evidence offsets, dates, links and selected chapters. This is analysis data, not an alternative authoritative archive export.
- `posts/`: reusable per-post classifications and text fingerprints.
- `build-report.json`: source/configuration hashes, reuse statistics and review queue.
- `verification/`: offline browser screenshots and verification results.

Posting seasons use Australia/Sydney publication months: December–February summer, March–May autumn, June–August winter, September–November spring. This is a browsing index, not the season at a travel destination or the date a photograph was taken. Explicit season words are indexed separately and can refer to past events. The common “spring in my step” idiom is excluded.

The analysis covers the retained catalogue, whose historical completeness remains unverified. Short captions and pronouns can leave an animal unidentified; these posts are still accessible in All posts and by text search. Shared text, punctuation, spelling variants and contextual ambiguity can also affect coverage. Review evidence before treating a thematic match as a factual assertion.

## Temporary internet sharing with Tailscale

In Terminal, from the project folder, start the local published-file server:

```sh
.venv-facebook/bin/python scripts/serve_facebook_themes.py
```

Open `http://127.0.0.1:8765/` to check it locally. In a second Terminal:

```sh
tailscale funnel http://127.0.0.1:8765
```

If using the Homebrew CLI-only installation and the daemon is not running, first run these commands one at a time in the sharing Terminal:

```sh
sudo /opt/homebrew/bin/brew services start tailscale
sudo /opt/homebrew/bin/tailscale up
```

Follow the sign-in link, then run the Funnel command above (prefix it with `sudo` if the daemon requires administrator access). Use `services start`, not `services run`: this formula requires the root service, and Homebrew rejects `run` as root. `start` also registers the daemon to start at boot. After sharing, stop and unregister it with `sudo /opt/homebrew/bin/brew services stop tailscale` if you only want a temporary setup.

Use the public HTTPS URL printed by Tailscale. If Funnel is not enabled for the tailnet, follow the setup/approval link printed by its CLI. Anyone with the resulting URL can browse the theme analysis, linked archive pages and saved media. Keep the Mac awake and both Terminal commands running while sharing. Press Ctrl+C in the Funnel Terminal to stop internet sharing, then Ctrl+C in the local-server Terminal to stop the server. The commands run in the foreground; no persistent background Funnel is requested. See [Tailscale's Funnel reference](https://tailscale.com/docs/reference/tailscale-cli/funnel).

The dedicated server only serves published analysis documents, archive overview/post pages, post text/metadata and media named in the retained catalogue. It does not expose the project directory, browser profile, raw source captures, analysis caches or directory listings, and makes no copies of the large media collection. It binds only to localhost. Restart the server after an archive refresh to include any newly published files.

## Validation

```sh
.venv-facebook/bin/python -m unittest discover -s scripts -p 'test_facebook_themes.py'
.venv-facebook/bin/python scripts/verify_facebook_themes.py
```

The first command checks incremental reuse, rule edits, changed posts, stale manual annotations, evidence attribution and a season idiom. The second verifies evidence offsets against all retained posts, membership counts, every post’s accessibility, filters, shared-text search, story order, three-view navigation and responsive layout in offline Chrome.