---
description: Test an Ananke website against multiple local Hugo versions
---

# Test ananke websites with local hugo versions

Use these instructions when testing an Ananke-based website against multiple local Hugo binaries.

The goal is to verify that the website builds correctly across a requested Hugo version range, collect full logs per Hugo version, and compare output differences, warnings, and errors.

## Hugo executable location

Assume Hugo binaries are stored in:

`~/github.com/davidsneighbour/hugov/executables/`

The binary names follow this pattern:

```text
hugo-v0.162.1
hugo-v0.162.1-extended
```

Only use the **non-extended** Hugo binaries for these tests.

Do not use binaries ending in:

```text
-extended
```

## Version range interpretation

When the user says something like:

```text
test between 160 and 162
```

interpret it as:

```text
v0.160.0 <= version < v0.163.0
```

In other words:

* `160` means all available `v0.160.x` versions.
* `161` means all available `v0.161.x` versions.
* `162` means all available `v0.162.x` versions.
* Include every patch version found locally in that range.
* Do not invent missing versions.
* Do not download Hugo versions.
* Only test versions that exist as local non-extended binaries.

Examples:

```text
test between 160 and 162
```

should include binaries such as:

```text
hugo-v0.160.0
hugo-v0.160.1
hugo-v0.161.0
hugo-v0.162.0
hugo-v0.162.1
```

but only if they exist locally.

It must exclude:

```text
hugo-v0.160.0-extended
hugo-v0.162.1-extended
```

## Required test workflow

Run the test from the root of the Ananke website repository unless the user explicitly gives another path.

For each matching Hugo binary:

1. Resolve the absolute path of the Hugo binary.
2. Extract the semantic version from the filename.
3. Run the Hugo build command using that binary.
4. Capture stdout and stderr.
5. Store the full output in a log file.
6. Record the exit status.
7. Record warnings.
8. Record errors.
9. Compare the generated output and command output across all tested versions.
10. Summarise differences.

## Log directory

Store logs in:

```text
~/.logs/ananke-hugo-tests/
```

Create the directory if it does not exist.

Each test run should create a timestamped run directory:

```text
~/.logs/ananke-hugo-tests/YYYYMMDD-HHMMSS/
```

Inside that directory, write one log file per Hugo version.

Use filenames sorted naturally by version:

```text
hugo-v0.160.0.log
hugo-v0.160.1.log
hugo-v0.161.0.log
hugo-v0.162.1.log
```

Also create a summary file:

```text
summary.md
```

The summary file must include:

* tested repository path
* test timestamp
* requested version range
* resolved Hugo binaries
* command used per version
* exit status per version
* warning count per version
* error count per version
* output differences
* build output differences
* final judgement

## Hugo command

Use this default command unless the user requests another one:

```bash
hugo --gc --minify
```

Before each Hugo run, remove the generated output directory if it exists.

Default output directory:

```text
public/
```

This prevents stale output from one version affecting another version.

## Build output isolation

Each Hugo version must build into its own output directory so that generated files can be compared.

Use this pattern:

```text
.tmp/hugo-version-tests/v0.162.1/public/
```

Run Hugo with:

```bash
hugo --gc --minify --destination ".tmp/hugo-version-tests/v0.162.1/public"
```

Before each run, remove that version's output directory.

Do not remove unrelated folders.

## Comparison requirements

After all builds finish, compare:

1. Exit status
2. stdout and stderr
3. warnings
4. errors
5. generated file lists
6. generated file contents

The comparison must identify:

* versions that failed
* versions that passed
* warnings that appear only in specific versions
* errors that appear only in specific versions
* files generated by one version but missing in another
* files with changed contents
* relevant differences in generated HTML, CSS, JS, XML, JSON, and text files

Ignore irrelevant noise where appropriate, such as:

* timestamps generated by the build
* absolute temporary paths
* known cache paths
* ordering differences that do not change generated output

Do not hide real build differences.

## Warning and error detection

Scan logs case-insensitively for lines containing:

```text
warning
warn
error
failed
deprecated
panic
timeout
```

List matching lines grouped by Hugo version.

When warnings or errors repeat across all versions, list them once and mark them as shared.

When warnings or errors only happen in some versions, highlight the affected versions.

## Required final response format

When reporting results, use this structure:

```markdown
## Hugo version test summary

Repository: `...`

Requested range: `...`

Resolved Hugo versions:

* `v0.160.0` - passed
* `v0.160.1` - passed
* `v0.161.0` - failed
* `v0.162.1` - passed

## Issues

### Errors

* `v0.161.0`: ...

### Warnings

* Shared across all versions: ...
* `v0.162.1` only: ...

## Output differences

### File list differences

* `v0.160.0` generated `...`, but `v0.162.1` did not.

### File content differences

* `public/index.html` differs between `v0.160.0` and `v0.162.1`.
  * Difference: ...

## Logs

Logs were written to:

`~/.logs/ananke-hugo-tests/YYYYMMDD-HHMMSS/`

Summary file:

`~/.logs/ananke-hugo-tests/YYYYMMDD-HHMMSS/summary.md`

## Final judgement

...
```

## Behaviour rules

* Do not use Hugo extended binaries.
* Do not download missing Hugo versions.
* Do not assume a patch version exists.
* Do not stop after the first failing version.
* Continue testing all resolved versions.
* Always keep the full log output.
* Always compare results after all runs.
* Always list the tested Hugo binary paths.
* Always show when no matching Hugo versions were found.
* Always distinguish between command output differences and generated site output differences.
* Always use explicit version numbers in the final summary.