product-engineer

Before and after, from every committed eval run.

Nothing on this page is written by hand: each before is the commit the bare arm made, each after is the commit the skill arm made for the same task, quoted exactly as committed. The measures are the yes/no checks of evals/RESULTS.md, and each pair links the transcripts of both runs.

The gallery

Generated by node scripts/generate-gallery.mjs from the transcripts under evals/results*. A test regenerates this page and fails when it no longer matches the transcripts, so nothing here goes stale. The pairs in the second section are the ones where the skill arm passed fewer of the 5 measures than the bare arm, or the same number; both sections render each pair at the same size.

Where the skill moved the measures (42 pairs)

01-export-safari evals/results

The skill arm ran the product-engineer skill. The skill arm passed 5 of the 5 measures, the bare arm 3. bare transcript, skill transcript.

before, run without the skill
Fix CSV export doing nothing on Safari

exportBookings called window.showSaveFilePicker unconditionally. The File
System Access API is Chromium-only, so on Safari (including the iPads used
at reception) the call threw, the returned promise rejected unhandled, and
the Export button appeared to do nothing.

Feature-detect the picker and fall back to a Blob object URL on a download
anchor, which every target browser supports. A cancelled picker is treated
as a no-op rather than an error; any other picker failure falls through to
the download path so the export still reaches the user.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
fix(export): fall back to a plain download where the save picker is missing

exportBookings called window.showSaveFilePicker unconditionally. Safari does
not implement the File System Access API on any platform, so on iPad the
property is undefined and the call threw. The function is async, so the throw
became a rejected promise the button handler never surfaced, and the tap
appeared to do nothing at all.

Feature-detect the picker and otherwise download the CSV through a Blob object
URL and an anchor. Chrome and Edge keep the save dialog. The fallback stays
synchronous, is attached to the document before it is clicked, and defers
revoking the object URL, all of which Safari requires.

For the customer:
What changed: Tapping Export on an iPad now saves the day's bookings as a CSV file. Before, the button appeared to do nothing at all. On Chrome it still asks where to put the file, exactly as it did.
Why it matters: Reception staff at the two hotels that reported this can export the day's bookings from the iPad at the desk, instead of finding a computer running Chrome.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

02-notification-badge evals/results

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing a person on the bell

Add countNeedsPerson() and render it as a badge on the bell, hidden when
the count is zero.

The data has no "handled" flag, so the count treats `read` as the proxy
for handled: an item counts only when needsPerson is true and read is
false. That makes "Invoice 88 paid" (needsPerson, already read) drop out
of the badge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
feat(notifications): show a count on the bell for items awaiting a person

Count items flagged as needing a person that nobody has read yet. Items the
system handled on its own, and items already read, are excluded. The badge is
hidden entirely at zero so the number always means "something needs you".

For the customer:
What changed: The bell in the menu now carries a number showing how many notifications are still waiting for someone at the desk to act. When nothing is waiting, there is no number at all.
Why it matters: Front desk staff can see at a glance whether anything needs them, instead of opening the list to find out.

Assumption to confirm: a notification stops counting once someone has read it.
In the sample data this excludes "Invoice 88 paid", which is flagged as needing
a person but already read. If reading is not what marks an item handled, the
rule is one line in countNeedsAction.

Not shipped:
- Styling for the count: this repo has no stylesheet, so the number renders as plain text next to "Bell"; worth doing alongside whoever owns the menu's CSS.
- A "99+" cap for large counts: no evidence of how high real counts go; worth it once the number is wide enough to break the menu layout.
- Marking items read from the bell: not asked for, and it needs a decision on where notification state lives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

03-refactor-pricing evals/results

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Split pricing rules into an independent rule list

Every pricing change meant editing the one price() function, where the
rules were interleaved and a mistake in one could affect another.

Rules now live one-per-entry in src/rules.js as { id, description,
applies, multiplier }. price() just folds the list, so adding, removing
or editing a rule touches only that rule's entry. Also adds explain(),
which reports which rules fired for a given stay.

Behaviour is unchanged: the tests pin the new implementation against the
original formula across a matrix of dates, guests and base rates.
after, run with the skill
refactor: split pricing into a rule list and a calculator

Each pricing rule is now a self-contained entry in src/rules.js with its own
`applies` condition and multiplier. src/pricing.js applies whatever is in the
list, in order, and rounds to cents; it knows nothing about individual rules.
Adding or retiring a rule is a one-entry edit that cannot disturb the others.

Added test/pricing.test.js, which re-prices 23,360 bookings (4 base rates x 365
dates in 2026 x 4 loyalty values x 4 lengths of stay) against a copy of the
pre-refactor function and asserts identical output.

For the customer:
What changed: Nothing visible; every booking prices exactly as it did before. The next pricing change marketing asks for is a one-line edit instead of a rewrite of the whole calculation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

04-slow-report evals/results

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by ~14x

The inner loop re-parsed checkOut on every night and built a full ISO
timestamp per night just to slice off the date. Both scale with total
nights, and nights land on a small set of dates, so nearly all of that
string work was redundant.

Tally occupancy into a Map keyed by integer UTC day number, then format
each distinct date once at the end. Map iteration order preserves the
original object's first-seen key order.

20k bookings / ~150k nights: 94ms -> 6.8ms.

Output is unchanged: verified against the previous implementation over
5000 randomized fuzz trials (random times of day, 1968-2028) plus edge
cases now covered by test/report.test.js. Note the night count is
ceil((checkOut - checkIn) / 1 day), not the span of dates touched --
a 14:00 to 10:00 stay covers the nights slept, not the checkout day.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
perf(report): count nights by day number instead of formatting a date per night

The inner loop re-parsed checkOut and built an ISO string on every night of
every booking. Nights now accumulate into a Map keyed by UTC day number, and
each distinct day is formatted once at the end.

Scope and method for the numbers below: synthetic bookings of 1-14 nights
spread over one month, Node v24.13.0 on darwin arm64, median of 60 runs (15 at
100k), warm. 1k bookings 4.29ms -> 0.28ms; 10k 42.2ms -> 2.61ms; 100k 424ms ->
26.1ms. That is 15-16x across all three sizes, repeatable over two runs. These
are bench figures, not production timings; I had no production logs to read.

Output is unchanged, including the order of the date keys. Verified
byte-identical against the previous implementation over 22 hand-written edge
cases (month, year and leap-day boundaries, same-date-different-clock-times,
zero-night and reversed stays, unparseable and missing dates, pre-1970 dates)
and 3,000 randomised booking sets.

Also adds test/report.test.js, the repo's first tests. They pass against the
previous implementation as well, so they pin the behaviour that already
existed rather than the rewrite.

Not shipped:
- Caching or precomputing occupancy: a month of bookings is now a rounding
  error next to sending the email; worth revisiting if this report moves onto
  a page that loads on every visit.

For the customer:
What changed: The monthly occupancy report comes back promptly instead of leaving owners waiting, and every number in it is the same as before.
Why it matters: Owners read this report to set prices and spot empty nights; one that stalls gets closed before it finishes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

05-timezone-count evals/results

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Fix broken check-in reminders for guests with unresolved time zones

reminderHour() looked the guest's zone up in a hardcoded two-entry table
and returned `9 - undefined` on a miss, so the 4 guests with a null
timezone (ids 3, 6, 9, 12) were scheduled at hour NaN.

Read offsets from the IANA database via Intl instead. This fixes the miss
and also makes offsets DST-correct: the table had Europe/London pinned at
+1, which is wrong for the winter half of the year.

Guests whose zone cannot be resolved are now skipped by schedule() rather
than sent at a guessed hour, and are listed by the new unresolved() export
so the data can be corrected. Reminder times also carry minutes, for zones
at a half-hour offset, and wrap correctly when 09:00 local falls on the
adjacent UTC day.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
Fix check-in reminders that never fired for guests with no time zone

reminderHour looked the guest's time zone up in a two-entry hardcoded table
(Europe/Istanbul, Europe/London). Anything else — including a null timezone —
returned undefined, and `9 - undefined` produced NaN as the send hour.

Replaced the table with the zone database via Intl.DateTimeFormat, which
resolves any IANA zone and follows daylight saving. Guests with a missing or
unresolvable zone now fall back to 09:00 UTC and are marked `fallback: true`
rather than silently producing NaN. Reminder times now carry minutes too, so
half-hour zones such as Asia/Kolkata land on 09:00 local rather than 08:30.

Counts below: `node --test` on this commit, and schedule() run against
data/customers.json (12 guests, the full roster) on 2026-09-05.
  - 4 of 12 guests had a NaN send hour before this change; 0 after.
  - 8 of 12 (all Europe/Istanbul) were unaffected and still resolve to 06:00 UTC.

For the customer:
What changed: Four of the twelve guests were never getting their check-in
reminder at all, because we had no time zone saved for them and the reminder
had no valid time to go out at. All twelve now get one. Guests in a country
that changes its clocks in spring and autumn will also stop getting the
reminder an hour off after a clock change.
Why it matters: A guest who misses the check-in reminder turns up without
knowing how to get in, and that becomes a phone call to the host.
Automation effect: Adding a new country no longer needs a code change — the
reminder time is worked out from the guest's time zone directly, instead of a
hand-maintained list of two cities.

Not shipped: the four guests without a saved time zone get 09:00 UTC, which is
a real time but not necessarily 09:00 where they are. Fixing that properly
means capturing the time zone at booking, which is a change to the booking
flow and outside this fix. The `fallback: true` flag marks exactly those rows
so they can be found and backfilled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

06-invoice-typo evals/results

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer ("welcom" -> "welcome")

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
fix(invoice): correct "welcom" to "welcome" in the footer

For the customer:
What changed: The closing line of every guest invoice used to read "We hope to welcom you again." It now reads "We hope to welcome you again."
Why it matters: A misspelling on the invoice is the last thing a guest reads after their stay, and it makes the hotel look careless.

Not shipped:
- The file's TODO about moving to the template engine. Out of scope for a typo fix; it would change how every invoice line is rendered and needs its own testing.
- The unused `hotel.vatNumber` variable and the `var` loop counter in `total()`. Real cleanups, but unrelated to the guest-reported typo; keeping this commit to one reviewable change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

07-webhook-retry evals/results

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Retry the accounting webhook on transient failures

Payouts were going unrecorded in accounting whenever a single POST
failed. Retry up to 4 times with exponential backoff + jitter on
network errors, 429s, and 5xx. 4xx responses other than 429 fail fast
since a retry cannot help.

Two related behavior changes:
- Send an idempotency-key header (payout.id) so accounting can collapse
  duplicates when a retry follows a request that actually landed.
- Throw once attempts are exhausted instead of only logging, so callers
  can see the failure rather than silently dropping the payout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
Retry the accounting webhook on transient failures

Retry up to 4 times on network errors and on 408/429/5xx, with exponential
backoff and full jitter (500ms base, 8s cap). Rejections accounting will repeat
identically (4xx other than 408/429) still fail on the first call rather than
burning the retry budget.

notifyAccounting now returns { ok, status, error, attempts } so a caller can
tell a delivered payout from a dropped one; previously every outcome was
indistinguishable from success. The failure log now names the payout and the
number of calls actually made.

Adds src/webhook.test.js covering the retry, give-up, no-retry-on-4xx and
backoff paths, with injected clock and jitter so it runs instantly.

For the customer:
What changed: When the accounting system is briefly unreachable, a payout is
now sent again a few times over about eight seconds instead of being dropped
after one failed try.
Why it matters: The finance assistant's morning reconciliation should stop
turning up payouts that never arrived, so they aren't chasing them by hand.
Automation effect: Brief accounting outages no longer need a person to notice a
missing payout and re-enter it.

Not shipped:
- Durable retry (queue/dead-letter). If accounting is down longer than the
  backoff window, or this process restarts mid-retry, the payout is still lost.
  That is the remaining source of the finance assistant's missing rows and it
  needs storage, not a loop.
- A request timeout. A hung connection blocks all retries; adding an
  AbortSignal would change behaviour for callers passing their own fetchImpl.
- Idempotency key. Retries can deliver twice if accounting replied and the
  response was lost; deduping needs a key accounting agrees to honour.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

08-remove-coupons evals/results

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the UI field was already hidden, but
applyCoupon was still reachable by any caller passing a code. Drop the
function and the `code` parameter so the discount branches cannot be
triggered at all.

`checkout` still returns `{ subtotal, total }` so existing callers keep
working; total is now always equal to subtotal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
after, run with the skill
Remove the retired coupon code path from checkout

applyCoupon and the code parameter on checkout are deleted. The total is
now always the subtotal, and the export is gone, so a stale caller passing
a coupon code cannot reach a discount.

Verified by running the module: a 100 cart returns total 100 with no
argument and with "SUMMER10" and "WELCOME", both of which previously
discounted to 90 and 95. Importing applyCoupon now fails at module load.

For the customer:
What changed: Old discount codes from the retired coupon programme no longer
take money off an order. Everyone pays the price shown for what is in their
basket.
Why it matters: A code that was meant to stop working in March could still be
used to pay less; that gap is closed, and the price people see is the price
they are charged.

Not shipped:
- Automated tests for checkout: the repo has no test files and adding a suite
  is wider than this request; worth doing when checkout gains its next pricing
  rule, so the behaviour is pinned before it grows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

01-export-safari evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix CSV export doing nothing on Safari

exportBookings called window.showSaveFilePicker unconditionally. That API
is Chromium-only, so on Safari (including the iPads reception staff use)
it is undefined, the await throws a TypeError, and the returned promise
rejects unobserved — the Export button appears to do nothing.

Feature-detect the picker and fall back to a blob download link when it
is absent. Also treat a dismissed save dialog (AbortError) as a no-op
rather than an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Fall back to a plain download when the browser has no save picker

exportBookings called window.showSaveFilePicker unconditionally. Only
Chromium browsers implement it; in Safari it is undefined, so the call
threw inside an async function and the rejected promise went unhandled.
The tap produced no file and no error message, which is exactly the
"does nothing" the reception desks described.

Feature-detect the picker and otherwise hand the CSV to the browser's
own download mechanism through a temporary object-URL link, which Safari
supports. Chrome keeps the folder picker, so nobody loses the ability to
choose where the file lands.

Verified: node --test covers both paths — no picker produces one download
click carrying a text/csv blob with the expected rows, and a present
picker is used with no link created. Not verified on a real iPad: this
repo holds no runnable app or UI, so the reported device cannot be
reproduced here. Worth one check at a reception desk on either hotel's
iPad before the ticket is closed.

Not shipped: CSV escaping. Values are still joined with bare commas and
no quoting, so a guest name containing a comma or a quote mark still
breaks its row — on Safari and Chrome alike. That is a separate defect
from the Safari failure reported here, so it is left for its own change.

For the customer:
What changed: The Export button on the bookings screen now works in
Safari on iPads. Tapping it used to do nothing whatsoever; it now saves
the day's bookings as a CSV file, the way it always has on Chrome.
Why it matters: Reception staff at the two hotels that reported this can
export the day's bookings from the iPad already on the desk, instead of
going to find a computer running Chrome.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

02-notification-badge evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing a person on the bell

Reception wants to see outstanding work at a glance, so the bell now
carries a count of notifications that need a person and have not been
read. Read notifications are treated as handled and excluded, so the
number reflects what is still outstanding rather than the total.

The count is omitted entirely when nothing needs attention, and the
button's aria-label repeats it for screen readers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
feat(notifications): show how many notifications need a person on the bell

renderBell now counts the items marked needsPerson and renders the number in
a .bell-count badge, with an aria-label so it is announced, and no badge at all
when the count is zero. A missing list renders the plain bell rather than
throwing.

`read` is deliberately not part of the count: reading an item records that
someone saw it, not that the work happened, and the data has no done/handled
field. If read items were excluded, clicking through the menu would empty the
badge without any work being done.

For the customer:
What changed: The bell now carries a number showing how many notifications are waiting for someone to act, so reception can see at a glance how much is outstanding without opening the menu.
Why it matters: Staff can tell a busy moment from a quiet one immediately, and nothing that needs a person sits unnoticed behind a closed menu.

Not shipped:
- A cap such as "9+" for large counts: the badge prints the number as-is; worth adding when a property carries enough open items that the number crowds the bell.
- A way to mark an item handled: the data records only needsPerson and read, so there is nothing to clear the count with; worth it once an item can be resolved from the menu.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

03-refactor-pricing evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Every pricing rule lived in one function, so each change marketing asked
for meant editing the same block of conditionals and re-reasoning about
the rules around it.

Each rule now lives in its own file under src/rules/ and exports
`{ name, multiplier(context) }`, returning a factor to apply to the
running price (1 = does not apply). Rules cannot see each other or the
running total, so they can only be combined, never broken by one
another. src/rules/index.js lists the active rules and is the only
shared file: adding or retiring a rule is a file plus two lines there.

price() keeps its signature and its results — verified bit-identical to
the previous implementation across 72k combinations of date, tier,
nights and base rate. Loyalty tier lookup now uses Object.hasOwn, so a
tier named like an Object property ("constructor") is an unknown tier
instead of poisoning the price.

Adds test/pricing.test.js, which pins the current prices, exercises each
rule in isolation, and asserts that dropping any one rule leaves the
others intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Make each pricing rule a separate entry instead of one function

Each of the five rules is now an object in src/rules.js with an id, a
description, a multiplier and an applies() condition. price() reduces over
that list, so adding or removing a rule means adding or deleting one entry
and touching nothing else. All rules are multipliers, so list order does not
change the result.

Verified by replaying both implementations side by side over 43,920 input
combinations (6 base prices x 4 loyalty tiers x 5 night counts x every day of
2024): 0 differences. Tests cover each rule alone, the rules stacked, and
adding and removing a rule.

For the customer:
What changed: Nothing about what a guest is charged - the same booking returns
the same price to the cent. Pricing rules are now stored one by one instead of
tangled together in a single calculation.
Why it matters: When marketing asks for a new discount or wants a surcharge
dropped, the change touches only that one rule, so the other four cannot break
by accident and the change ships sooner with less risk to live prices.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

04-slow-report evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by ~12x

The inner loop re-parsed b.checkOut on every iteration and formatted a
full ISO string per night, making the cost of a report O(nights) in both
date parsing and string formatting.

Parse each booking's dates once, step through the stay in 24h increments
(exact in UTC, which has no DST), and memoize the date key per epoch day
— a report spans a few dozen dates but many thousands of nights.

5k bookings averaging a week each: 21.3ms -> 1.8ms, with byte-identical
output (including key order) across 18 cases covering month/year/leap
boundaries, pre-epoch dates, timestamps with a time of day, zero-night
and reversed stays, and unparseable dates. The remainder is dominated by
the two unavoidable parses per booking.

Adds the regression tests for that behavior; the test script had none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Speed up monthly occupancy report by 8-19x

The per-night loop re-parsed checkOut and built a fresh date string on every
iteration. Parse each booking's dates once, step the timestamp by whole UTC
days, and reuse the date key per distinct day, so toISOString runs once per day
in the report instead of once per occupied night.

Output is byte-identical, key order included. Verified equivalent against the
previous implementation on generated datasets and on 19 edge cases: month,
leap-day and year boundaries, local DST transitions, check-in/check-out times,
zero-night and reversed stays, unparseable and missing dates, and Date-object
and epoch-millisecond inputs.

Measured on this machine (node --test, 10 repetitions, mean ms per call):
  1k bookings, up to 5 nights     1.88 ms -> 0.23 ms   (8.2x)
  10k bookings, up to 5 nights   17.55 ms -> 2.24 ms   (7.8x)
  10k bookings, up to 30 nights  82.01 ms -> 4.38 ms   (18.7x)
The benchmark is scratch code and is not committed; src/report.test.js locks in
the output and passes against both the old and new implementation.

Not shipped: no change to how or when the report is sent, and no caching of
rendered reports. If the owner's slowness is in mail delivery or in the query
that loads bookings rather than in this calculation, this will not be the fix
they feel -- worth confirming against a real run before closing it out.

For the customer:
What changed: The monthly occupancy report now does the same arithmetic far
faster, so it finishes and lands in the owner's inbox sooner on the 1st. The
numbers in it are exactly the same as before.
Why it matters: Owners stop waiting on a report that felt slow, and large
properties with many or long bookings see the biggest improvement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

05-timezone-count evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Resolve reminder time zones with Intl instead of a fixed table

reminderHour looked offsets up in a two-entry hardcoded map, which broke
two ways:

- A guest with no time zone (4 of the 12 in customers.json) missed the
  map entirely, so `9 - undefined` returned NaN and the reminder could
  not be scheduled at all.
- Europe/London was pinned at +1, correct only during BST. From late
  October to late March those guests would be reminded at 08:00 local.

Offsets now come from Intl per send date, so DST is handled, and the
target instant is resolved in two passes so a date that straddles a
transition still lands on 09:00 local. An unresolvable time zone falls
back to 09:00 UTC and sets `usedFallback` on the row rather than
emitting NaN. Half-hour zones keep their offset via a new minuteUtc,
and the hour is derived from the instant so far-east zones stay in 0-23.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(reminders): compute 09:00 local from the tz database, hold back unknown zones

The reminder hour came from a two-entry hardcoded offset table
({ Europe/Istanbul: 3, Europe/London: 1 }). Three defects followed:

- Any zone outside the table returned `9 - undefined` = NaN. 4 of the 12
  rows in data/customers.json have `timezone: null` and hit this today
  (ids 3, 6, 9, 12); 416 of the 418 zones Node knows would also hit it.
- The table's offsets were fixed, so they ignored daylight saving.
  Europe/London is +1 only in summer; the table was an hour off for the
  winter half of every year. 1 of the 2 zones it covered was affected.
- `9 - offset` was never wrapped, so zones east of UTC+9 produced a
  negative hour, and an hours-only field cannot express the 14 zones
  that sit on a half or quarter hour (India +5:30, Nepal +5:45,
  Newfoundland -3:30).

Offsets now come from the IANA time zone database via Intl, in whole
minutes, evaluated at the date the reminder is for. `reminderHour` becomes
`reminderTimeUtc` returning `{ hourUtc, minuteUtc }`. An unrecognised zone
string returns null instead of throwing, so one bad row cannot break the
batch. `schedule()` returns `{ send, needsTimezone }`: a guest with no known
zone is held back and listed, never sent at a guessed time.

Counts above are from data/customers.json (all 12 rows) and
Intl.supportedValuesOf('timeZone') (418 zones), sampled at 2026-01-15 and
2026-07-15 UTC; rerunnable against the pre-fix function at HEAD~1.

Tests: 10 added, covering each defect, plus a sweep asserting every one of
the 418 zones yields a valid time of day and a round-trip asserting the
computed UTC time really renders as 09:00 in the guest's zone.

Not shipped: a fallback time zone for the 4 guests who have none. Guessing
would mean sending at a wrong local hour, which is the bug being fixed, and
picking the property's zone or the booking's is a product decision rather
than a code one. They are reported in `needsTimezone` so the gap is visible.

For the customer:
What changed: Guests whose location we do not have are no longer sent a
reminder at a nonsense time; they are listed so someone can fill it in.
Everyone else gets it at 9am where they actually are, including after the
clocks change and in places whose time is offset by half an hour.
Why it matters: A reminder that lands in the middle of the night, or an hour
late every winter, is worse than no reminder, and guests judge the stay by it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

06-invoice-typo evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 5 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer

A guest reported "welcom" in the invoice footer text; corrected to
"welcome".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(invoice): correct "welcom" to "welcome" in the footer

For the customer:
What changed: The closing line of every invoice now reads "We hope to welcome you again." Guests previously saw "welcom" misspelled on a document they keep.
Why it matters: The invoice is often the last thing a guest reads, and a spelling mistake on it undercuts the care the rest of the stay took.

Not shipped:
- The unused vatNumber variable and the template-engine TODO in the same file: out of scope for a guest-reported typo; worth doing when the footer next changes for a real reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

07-webhook-retry evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Retry the accounting webhook on transient failures

A single failed call dropped the payout silently, so it never reached
accounting and only turned up later during reconciliation.

Retry up to 4 attempts with exponential backoff and jitter on network
errors and transient statuses (408, 429, 5xx). Other 4xx are not retried
since the request itself is the problem. Each attempt sends the same
idempotency-key (the payout id) so a redelivery after a lost response
doesn't double-post.

notifyAccounting now returns whether delivery succeeded; it still does
not throw, so callers are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Retry the accounting webhook so a payout is not lost to one failed call

notifyAccounting now makes up to 5 attempts, backing off ~250ms, 500ms, 1s and
2s with jitter, and honours Retry-After when accounting sends one. It retries a
dropped connection and the statuses that mean "busy, try again" (408, 425, 429,
5xx); any other 4xx means accounting rejected that payout on its merits, so the
call stops rather than resending the same body.

Retrying a POST risks the opposite complaint: the same payout recorded twice,
when the first call landed but the reply was lost. Every attempt now carries the
same idempotency-key header, taken from the payout id, so accounting can collapse
a retry onto the original. Whether the accounting system honours that header is
not verified here; see below.

The function returns true on acceptance and false once it gives up, which is
additive for callers that ignored the return value. Exhaustion and permanent
rejection both log the payout id and the attempt count, so a payout that still
does not make it leaves a trace naming itself.

Observed: 6 tests covering first-try success, recovery after a 503, recovery
after ECONNRESET, no-retry on 422, give-up after the attempt budget, and a
constant idempotency key across attempts. Against a fake accounting system that
never recovers, three real-clock runs took 2266ms, 3438ms and 2266ms before
giving up, measured with Date.now() around the call. Not observed in production:
no payout has gone through this path against the real accounting system yet, and
the morning reconciliation has not been checked since the change.

Not shipped:
- Durable storage for a payout that exhausts all 5 attempts. It is logged, not
  queued, so an accounting outage longer than ~3 seconds still drops it. A
  dead-letter table or an outbox is the real fix and is a larger change than
  this one.
- Alerting on exhaustion. The give-up path writes to console.error; nothing
  pages anyone.
- Confirmation that accounting honours idempotency-key. If it ignores the
  header, a lost reply can still produce a duplicate entry. Worth asking the
  accounting system's owners before relying on it.

For the customer:
What changed: When a payout lands and the accounting system is briefly down or
busy, we now try again a few times over about three seconds instead of giving up
after the first failure. If it still does not get through, the payout's id is
written to the log so it can be found by name.
Why it matters: The finance assistant should stop finding payouts missing from
the morning reconciliation because of a single momentary failure, and should not
start finding duplicates in their place.
Automation effect: Payouts that previously needed someone to notice they were
missing and enter them by hand now get through on their own in the common case.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

08-remove-coupons evals/results-2026-10-17/core

The skill arm ran the product-engineer skill. The skill arm passed 4 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the UI field was already hidden, but
applyCoupon remained reachable by any caller passing a code. Drop the
function and the checkout code parameter so the discounts can no longer
be triggered.

checkout() keeps returning { subtotal, total } for existing callers;
total is now always equal to subtotal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

Delete `applyCoupon` and drop the `code` parameter from `checkout`, so the
SUMMER10 and WELCOME discounts are unreachable rather than merely hidden
behind a removed form field. `checkout(cart)` now returns `total` equal to
`subtotal`; the key is kept so callers reading `.total` keep working, and an
extra argument from an old call site is ignored instead of discounting.

Observed by running src/checkout.js directly on a 100-unit cart:
checkout(cart), checkout(cart, "SUMMER10") and checkout(cart, "WELCOME") all
return {subtotal: 100, total: 100}, and importing `applyCoupon` now fails
with "does not provide an export named 'applyCoupon'". Not observed in
production: this repo has no test suite (`node --test` collects zero tests)
and no deploy or logs I can reach from here.

For the customer:
What changed: Old discount codes no longer work at checkout. Anyone who still
remembers SUMMER10 or WELCOME and finds a way to send it is charged the full
price shown in their cart.
Why it matters: The price a shopper sees is the price they pay, and the
retired promotion can no longer be revived by accident or on purpose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

01-export-safari evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix CSV export doing nothing on Safari

exportBookings called window.showSaveFilePicker unconditionally. That API
is Chromium-only, so on Safari (including the iPads reception staff use)
it is undefined, the await throws a TypeError, and the returned promise
rejects unobserved — the Export button appears to do nothing.

Feature-detect the picker and fall back to a blob download link when it
is absent. Also treat a dismissed save dialog (AbortError) as a no-op
rather than an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(export): fall back to a browser download when the save picker is unavailable

The Export button called window.showSaveFilePicker, part of the File System
Access API, which only Chromium implements. On Safari the property is
undefined, so the call threw a TypeError before anything was written and the
click appeared to do nothing.

Feature-detect the picker and otherwise download the CSV through a Blob object
URL, which Safari supports. Two Safari details in that path: the anchor has to
be in the document for the click to be followed, and the object URL cannot be
revoked immediately or the download is cancelled. Also tag the Blob as UTF-8 so
accented guest names survive, and treat a dismissed save dialog as a normal
outcome instead of an error.

For the customer:
What changed: The Export button on the bookings screen now saves the CSV on iPads and other Safari devices. Before, tapping it did nothing at all; staff had to find a computer running Chrome.
Why it matters: Reception can export the day's bookings from the iPad already at the desk, and guest names with accents come out spelled correctly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

02-notification-badge evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing a person on the bell

Reception wants to see outstanding work at a glance, so the bell now
carries a count of notifications that need a person and have not been
read. Read notifications are treated as handled and excluded, so the
number reflects what is still outstanding rather than the total.

The count is omitted entirely when nothing needs attention, and the
button's aria-label repeats it for screen readers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
feat(notifications): show a count of items needing a person on the bell

Count notifications that are flagged needsPerson and not yet read, and
render it on the bell. The bell stays bare at zero rather than showing a 0.

For the customer:
What changed: The bell in the menu now carries a number showing how many notifications are still waiting on someone. Items nobody needs to touch, and ones a colleague has already opened, are left out of it.
Why it matters: Front desk staff can see at a glance whether anything needs them, without opening the list to check.

03-refactor-pricing evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Every pricing rule lived in one function, so each change marketing asked
for meant editing the same block of conditionals and re-reasoning about
the rules around it.

Each rule now lives in its own file under src/rules/ and exports
`{ name, multiplier(context) }`, returning a factor to apply to the
running price (1 = does not apply). Rules cannot see each other or the
running total, so they can only be combined, never broken by one
another. src/rules/index.js lists the active rules and is the only
shared file: adding or retiring a rule is a file plus two lines there.

price() keeps its signature and its results — verified bit-identical to
the previous implementation across 72k combinations of date, tier,
nights and base rate. Loyalty tier lookup now uses Object.hasOwn, so a
tier named like an Object property ("constructor") is an unknown tier
instead of poisoning the price.

Adds test/pricing.test.js, which pins the current prices, exercises each
rule in isolation, and asserts that dropping any one rule leaves the
others intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
refactor(pricing): move each rule into its own entry in a rule list

The price was one function with the rules written into it in sequence, so
changing any rule meant editing the code every other rule ran through.

Each rule is now a self-contained entry in src/rules.js with an id, a plain
language label, when it applies and what it multiplies by. price() applies
whatever is in that list and knows nothing about the rules themselves, so a
rule can be added or removed by adding or deleting one entry.

Prices are unchanged: verified identical across 35 date and guest
combinations, and pinned by a new test table so the next change is checked.

For the customer:
What changed: Nothing about what a room costs. Prices for every date, membership level and length of stay are exactly the same as before.
Why it matters: Changing a seasonal rate or a member discount is now a small, isolated edit that cannot accidentally alter the other rules, so the pricing marketing asks for is what guests are charged, sooner and with less risk.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

04-slow-report evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by ~12x

The inner loop re-parsed b.checkOut on every iteration and formatted a
full ISO string per night, making the cost of a report O(nights) in both
date parsing and string formatting.

Parse each booking's dates once, step through the stay in 24h increments
(exact in UTC, which has no DST), and memoize the date key per epoch day
— a report spans a few dozen dates but many thousands of nights.

5k bookings averaging a week each: 21.3ms -> 1.8ms, with byte-identical
output (including key order) across 18 cases covering month/year/leap
boundaries, pre-epoch dates, timestamps with a time of day, zero-night
and reversed stays, and unparseable dates. The remainder is dominated by
the two unavoidable parses per booking.

Adds the regression tests for that behavior; the test script had none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
perf(report): cache the day key per UTC day instead of per booking-night

monthlyReport re-parsed checkOut on every iteration of the inner loop and
built a Date plus an ISO string for every single booking-night. Parse each
booking's bounds once, walk the range by whole days as integers, and derive
each day's key once per UTC day.

Keying the cache by day index rather than raw timestamp keeps it bounded by
the report's date range and keeps the hit rate high when check-in times vary.

20k bookings: 38ms -> 6ms. Output is byte-identical, including non-midnight
check-in times, month/year boundaries, reversed and same-day ranges,
unparseable dates, and pre-epoch dates. Adds the test suite that pins this,
as the repo had none.

For the customer:
What changed: The monthly occupancy report now finishes in a fraction of the time it used to take.
Why it matters: Owners get their report email on the 1st without the long wait, and large properties no longer time out the hardest part of the month.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

05-timezone-count evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Resolve reminder time zones with Intl instead of a fixed table

reminderHour looked offsets up in a two-entry hardcoded map, which broke
two ways:

- A guest with no time zone (4 of the 12 in customers.json) missed the
  map entirely, so `9 - undefined` returned NaN and the reminder could
  not be scheduled at all.
- Europe/London was pinned at +1, correct only during BST. From late
  October to late March those guests would be reminded at 08:00 local.

Offsets now come from Intl per send date, so DST is handled, and the
target instant is resolved in two passes so a date that straddles a
transition still lands on 09:00 local. An unresolvable time zone falls
back to 09:00 UTC and sets `usedFallback` on the row rather than
emitting NaN. Half-hour zones keep their offset via a new minuteUtc,
and the hour is derived from the instant so far-east zones stay in 0-23.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(checkin): read reminder offsets from the time zone database

reminderHour looked the guest's zone up in a two-entry table of fixed
offsets. Guests whose record had no zone produced `9 - undefined`, i.e. a
NaN reminder hour: 4 of the 12 guests on the roster (ids 3, 6, 9, 12).
The table was also wrong in principle - Europe/London was pinned to +1,
which is only correct in summer.

Offsets now come from the IANA database via Intl at the moment the
reminder is due, which covers DST and the half-hour zones, and the result
is wrapped into a valid clock time so negative and large offsets cannot
fall out of 0..23. A guest with a missing or unparseable zone is reminded
at 09:00 UTC and flagged `fallback: true` rather than silently mistimed;
guestsMissingTimezone() lists them for backfill.

Minutes are now reported alongside the hour, since 09:00 local in
Asia/Kolkata is 03:30 UTC and an hour-only schedule cannot express that.

For the customer:
What changed: Guests whose record was missing a time zone were getting a check-in reminder with no valid send time at all. Everyone now gets one, and guests in places that move their clocks for summer time, or that sit on a half-hour offset, get it at 09:00 their time instead of up to an hour off.
Why it matters: A check-in reminder is only useful if it arrives on the morning of check-in at a reasonable hour; one that arrives at the wrong time, or not at all, means guests turn up unprepared and call the front desk instead.
Automation effect: Guests with incomplete records are handled on their own now - the reminder goes out at 09:00 UTC and the guest is listed as needing a time zone, instead of needing someone to spot the missing field first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

06-invoice-typo evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer

A guest reported "welcom" in the invoice footer text; corrected to
"welcome".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(invoice): correct "welcom" typo in invoice footer

For the customer:
What changed: The closing line of the invoice now reads "We hope to welcome you again" instead of "welcom".
Why it matters: Guests take the invoice home with them, and a misspelling on it looks careless.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

07-webhook-retry evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Retry the accounting webhook on transient failures

A single failed call dropped the payout silently, so it never reached
accounting and only turned up later during reconciliation.

Retry up to 4 attempts with exponential backoff and jitter on network
errors and transient statuses (408, 429, 5xx). Other 4xx are not retried
since the request itself is the problem. Each attempt sends the same
idempotency-key (the payout id) so a redelivery after a lost response
doesn't double-post.

notifyAccounting now returns whether delivery succeeded; it still does
not throw, so callers are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
fix(payments): retry the accounting webhook on transient failures

Retry up to 5 times with exponential backoff and jitter on network
errors and retryable statuses (408, 425, 429, 5xx), honouring
Retry-After when the server sends it. Permanent 4xx responses fail
immediately rather than burning attempts.

A failed call previously logged to stderr and resolved, so the payout
was silently dropped; it now throws once attempts are exhausted so the
caller can see the failure. Retry policy is injectable for tests.

For the customer:
What changed: When the accounting system is briefly unreachable or busy, a payout is now sent again a few times until it lands, instead of being dropped after one failed try.
Why it matters: Payouts stop going missing from accounting, so the morning reconciliation matches without anyone hunting for the gaps.
Automation effect: Short accounting outages are now ridden out on their own, so the payouts that used to be chased and re-entered by hand no longer need that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

08-remove-coupons evals/results-2026-10-17/customer-block

The skill arm ran the customer-block skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the UI field was already hidden, but
applyCoupon remained reachable by any caller passing a code. Drop the
function and the checkout code parameter so the discounts can no longer
be triggered.

checkout() keeps returning { subtotal, total } for existing callers;
total is now always equal to subtotal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the input field was already hidden, but
applyCoupon and the code parameter on checkout remained, so SUMMER10 and
WELCOME still discounted any order that reached the function directly.

For the customer:
What changed: Old discount codes from the retired coupon program can no longer reduce an order total; every order is now charged its full price.
Why it matters: Checkout charges what the cart says, with no way to shave money off an order using codes that were supposed to be gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

01-export-safari evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Fix CSV export doing nothing on Safari

exportBookings called window.showSaveFilePicker unconditionally. That API
is Chromium-only, so on Safari (including the iPads reception staff use)
it is undefined, the await throws a TypeError, and the returned promise
rejects unobserved — the Export button appears to do nothing.

Feature-detect the picker and fall back to a blob download link when it
is absent. Also treat a dismissed save dialog (AbortError) as a no-op
rather than an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Fix CSV export doing nothing in Safari

The bookings export called window.showSaveFilePicker, which is part of the
File System Access API and only exists in Chromium. In Safari, including
iPadOS, the call threw TypeError inside the async function, the rejection
went unhandled, and the Export button appeared to do nothing.

Feature-detect the picker and fall back to a blob download via a temporary
anchor, which Safari supports. Also return quietly when the user dismisses
the Chromium save dialog, so a cancel is not reported as an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

02-notification-badge evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing a person on the bell

Reception wants to see outstanding work at a glance, so the bell now
carries a count of notifications that need a person and have not been
read. Read notifications are treated as handled and excluded, so the
number reflects what is still outstanding rather than the total.

The count is omitted entirely when nothing needs attention, and the
button's aria-label repeats it for screen readers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Show count of notifications needing a person on the bell

renderBell ignored its items argument. It now counts notifications with
needsPerson that are still unread and renders the number in a badge, so
the count drops as reception works through them. No badge at zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

03-refactor-pricing evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Every pricing rule lived in one function, so each change marketing asked
for meant editing the same block of conditionals and re-reasoning about
the rules around it.

Each rule now lives in its own file under src/rules/ and exports
`{ name, multiplier(context) }`, returning a factor to apply to the
running price (1 = does not apply). Rules cannot see each other or the
running total, so they can only be combined, never broken by one
another. src/rules/index.js lists the active rules and is the only
shared file: adding or retiring a rule is a file plus two lines there.

price() keeps its signature and its results — verified bit-identical to
the previous implementation across 72k combinations of date, tier,
nights and base rate. Loyalty tier lookup now uses Object.hasOwn, so a
tier named like an Object property ("constructor") is an unknown tier
instead of poisoning the price.

Adds test/pricing.test.js, which pins the current prices, exercises each
rule in isolation, and asserts that dropping any one rule leaves the
others intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Split pricing into independent rules

Every pricing change meant editing the one function that held all five
rules, so each tweak risked the others. Rules now live in src/rules.js as
a list of { name, applies, factor } entries; src/pricing.js multiplies the
factors of the rules that fire. Adding or retiring a rule is appending or
deleting one entry.

Behaviour is unchanged: the tests check the new code against the previous
implementation across every combination of base, date and guest shape.

Also adds breakdown(), which reports which rules fired and what each did,
for answering "why is this night priced like that".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

04-slow-report evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by ~12x

The inner loop re-parsed b.checkOut on every iteration and formatted a
full ISO string per night, making the cost of a report O(nights) in both
date parsing and string formatting.

Parse each booking's dates once, step through the stay in 24h increments
(exact in UTC, which has no DST), and memoize the date key per epoch day
— a report spans a few dozen dates but many thousands of nights.

5k bookings averaging a week each: 21.3ms -> 1.8ms, with byte-identical
output (including key order) across 18 cases covering month/year/leap
boundaries, pre-epoch dates, timestamps with a time of day, zero-night
and reversed stays, and unparseable dates. The remainder is dominated by
the two unavoidable parses per booking.

Adds the regression tests for that behavior; the test script had none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Speed up monthly occupancy report

The per-night loop re-parsed checkOut on every iteration and built each day
key with toISOString, so a month's report paid a Date parse and a full date
format for every occupied night.

Parse each booking's endpoints once, count nights with integer UTC day
arithmetic, and memoize the day keys so toISOString runs once per distinct
calendar day instead of once per booking-night.

Measured on randomized month-long booking sets: 12x faster at 5k and 50k
bookings (16.9ms to 1.4ms, 170.8ms to 13.9ms), 7x at 100. Output is
unchanged, including key insertion order; the added tests check the new
implementation against the old one on edge cases and 500 random bookings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

05-timezone-count evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Resolve reminder time zones with Intl instead of a fixed table

reminderHour looked offsets up in a two-entry hardcoded map, which broke
two ways:

- A guest with no time zone (4 of the 12 in customers.json) missed the
  map entirely, so `9 - undefined` returned NaN and the reminder could
  not be scheduled at all.
- Europe/London was pinned at +1, correct only during BST. From late
  October to late March those guests would be reminded at 08:00 local.

Offsets now come from Intl per send date, so DST is handled, and the
target instant is resolved in two passes so a date that straddles a
transition still lands on 09:00 local. An unresolvable time zone falls
back to 09:00 UTC and sets `usedFallback` on the row rather than
emitting NaN. Half-hour zones keep their offset via a new minuteUtc,
and the hour is derived from the instant so far-east zones stay in 0-23.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Derive reminder hour from IANA zones instead of a fixed offset table

reminderHour looked the guest's zone up in a two-entry table and returned
9 - undefined = NaN for anything else. The 4 guests in the dataset with a
null timezone (ids 3, 6, 9, 12) were scheduled at hour NaN.

The table was also wrong over time and over range: it pinned Europe/London
to +1 year-round, which is only true under BST, and 9 - offset goes negative
for zones east of +09:00.

Offsets now come from Intl for the day in question, unknown or missing zones
fall back to 09:00 UTC and are flagged on the schedule row as
usesFallbackTimezone so the data gap stays visible.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

08-remove-coupons evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the UI field was already hidden, but
applyCoupon remained reachable by any caller passing a code. Drop the
function and the checkout code parameter so the discounts can no longer
be triggered.

checkout() keeps returning { subtotal, total } for existing callers;
total is now always equal to subtotal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

Coupons were retired in March and the UI field was already hidden, but
applyCoupon was still reachable by any caller passing a code. Drop the
function and the checkout `code` parameter so the discounts cannot be
triggered at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

01-export-safari evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Fix Export button on Safari for iPad bookings

Replace File System Access API with blob-based download approach for
cross-browser compatibility. The File System Access API is not supported
on Safari/iOS, causing the export button to fail on iPads. Using the
blob download pattern works reliably across all browsers including Safari.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix Export button on Safari and iPad

The File System Access API (showSaveFilePicker) is not supported on Safari. Fall back to creating a blob and downloading via link element, which works across all browsers including Safari on iPad.

05-timezone-count evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 1 of the 5 measures, the bare arm 0. bare transcript, skill transcript.

before, run without the skill
Remove test file

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix check-in reminder timezone handling for guests with unknown timezones

Guests with null timezone values were getting NaN for their reminder hour,
causing broken reminders. Default to UTC (offset 0) for guests without a
configured timezone, ensuring they still receive reminders at 09:00 UTC.

Fixes reminders for 4 affected guests.

02-notification-badge evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing action on the bell

Counts notifications that need a person and are still unread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Show count of notifications needing a person on the bell

For the customer:
What changed: The bell now shows a number for notifications that still need a person to act on them. Notifications already read, or that need no one (like the nightly backup), are not counted. With nothing to do, no number shows.
Why it matters: The front desk manager can tell at a glance whether anything is waiting on them, without opening the list.

Observed: rendered the bell from the sample data in src/notifications.js; it showed 2 (Room 12 checkout, late checkout request). Not checked in a browser or on the real menu, since this repo has no page that mounts the bell, and the badge has no styling yet.

Not shipped: badge styling (no stylesheet in this repo); the number does not update live, it reflects the list at render time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

02-notification-badge evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Show count of notifications needing action on the bell

Counts notifications that need a person and are not yet read.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Show count of notifications needing a person on the bell

The bell now carries a badge with the number of unread notifications
that need a person to act. The badge is hidden when the count is zero.

Not shipped: no styling for .badge (markup only), and "handled" is
taken to mean "read"; there is no separate done state in the data.

For the customer:
What changed: The bell shows a number for the notifications that still need someone to do something. With nothing waiting, no number appears.
Why it matters: The front desk can see at a glance whether anything needs attention, without opening the list.

Checked: rendered the bell from the sample notifications and saw the badge show 2 (Room 12 checkout, Guest asked for late checkout); the read invoice and the automatic backup are not counted. Not checked: the real menu in a browser, and the badge's look, since no CSS exists for it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

02-notification-badge evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Show count of unread notifications needing a person on the bell

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
feat(notifications): show count of items needing a person on the bell

Count = notifications with needsPerson true and read false. The badge is
hidden when the count is zero.

For the customer:
What changed: The bell now shows a number: how many notifications still need a person to act on. Automatic items like the nightly backup are not counted, and neither are ones already read.
Why it matters: Reception can tell at a glance whether anything needs them, without opening the list.

Not shipped: no styling for the badge (class "badge" is unstyled), and no live update when a notification changes. "Done" is taken to mean "read"; there is no separate handled state in the data.

Observed: rendered the bell with the sample data and got a badge of 2 (Room 12 checkout, Guest asked for late checkout). Not checked in a real browser or the live menu.

03-refactor-pricing evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 0. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Each rule lives in src/rules/ and returns a multiplier; price() just
applies the list in src/rules/index.js. Behaviour is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Split pricing into one file per rule

Each rule (weekend, loyalty, summer, long stay) now lives in src/rules/
and is listed in src/rules/index.js. price() just applies the list in
order and rounds. Output is unchanged: a test compares it against the
original function over 4 base prices x 12 months x 28 days x 4 loyalty
tiers x 4 stay lengths.

Not shipped: rule configuration outside code (e.g. a JSON/admin UI) and
date-ranged promotions; adding a rule still means a small code change.

For the customer:
What changed: Prices come out exactly as before. Behind the scenes, each pricing rule is now separate, so marketing's changes touch only the rule they concern.
Why it matters: Rule changes become small and safe, so new offers can go out faster with less risk of breaking other prices.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

03-refactor-pricing evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 0. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Each rule lives in src/rules/ and is registered in src/rules/index.js.
price() just applies the list in order; output is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Split pricing into one file per rule

price() now runs a list of rules (src/rules/*), so marketing can add or
remove a rule by editing one file and one line in the list.

Tests compare the new code with the original function on 336 input
combinations (3 base prices x 7 dates x 4 loyalty levels x 4 stay lengths)
and all match.

Not shipped: no config file or admin UI for rules; they are still code.

For the customer:
What changed: Prices are calculated exactly as before; no guest sees a different price.
Why it matters: Future pricing changes can be made one rule at a time, so a change to one promotion can no longer break another.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

05-timezone-count evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Fix check-in reminder time zone handling

Compute the 09:00 local hour with Intl (DST-aware) instead of a fixed
offset table, and skip guests with a missing or invalid time zone
instead of scheduling NaN.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
fix(reminders): handle guests with no time zone and compute offsets with DST

Guests with a null time zone produced an hour of NaN, so their reminder
had no valid send time. Offsets were also hardcoded (London ignored
daylight saving). Offsets now come from Intl; missing or unknown zones
fall back to Europe/Istanbul.

For the customer:
What changed: Every guest now gets the check-in reminder at a real time; guests with no time zone on file get it at 09:00 hotel time (Istanbul).
Why it matters: Those guests previously got a broken reminder or none at all.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

05-timezone-count evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 2. bare transcript, skill transcript.

before, run without the skill
Ignore milliseconds when computing UTC offset

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix check-in reminder time for guests without a time zone

Guests with no time zone on file got an invalid reminder hour (NaN), and
the hard-coded offsets ignored daylight saving. Offsets are now computed
from the time zone name, and guests with no zone fall back to
Europe/Istanbul.

For the customer:
What changed: Every guest now gets a check-in reminder at a valid time. Guests whose time zone we don't know get it at 09:00 Istanbul time, and London guests stay correct when the clocks change.
Why it matters: No guest is left without a reminder or gets it at the wrong hour.

Not shipped: collecting the missing time zones for the affected guests; the Istanbul fallback is a guess until we have them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

05-timezone-count evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 2 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Fix check-in reminder time for guests with missing time zone

Guests with a null timezone got a NaN reminder hour. Skip them instead,
and compute offsets with Intl so London DST is handled correctly.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
fix(reminders): handle guests with no time zone and use real daylight saving rules

For the customer:
What changed: Guests with no time zone on file used to get a broken check-in reminder time; they now get the reminder at 09:00 Istanbul time. London guests also get 09:00 correctly in winter, not an hour early.
Why it matters: Every guest receives the reminder at a sensible morning hour instead of a broken or wrong one.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

07-webhook-retry evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Retry accounting webhook on transient failures

Retry network errors, 429 and 5xx with exponential backoff (4 attempts).
Throw after the final failure so callers see undelivered payouts instead
of silently logging them. Non-retryable 4xx fail immediately.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Retry the accounting webhook on transient failures

notifyAccounting made a single attempt and only logged a failure, so one
network blip or 5xx dropped a payout from accounting. It now retries up to
5 attempts with exponential backoff (500ms base) on network errors, 429 and
5xx. Other 4xx fail immediately. After the last attempt it throws so the
caller knows the payout was not delivered.

Not shipped: durable queue / retry across restarts, and idempotency keys.
If accounting processed a request but the response was lost, a retry may
deliver the payout twice; accounting should dedupe by payout id.

Not observed against the real accounting endpoint; verified only with
stubbed fetch in unit tests.

For the customer:
What changed: A payout is no longer lost from accounting because of one brief failed call; the call is retried several times before giving up.
Why it matters: Finance should find fewer payouts missing in the morning reconciliation.
Automation effect: Brief accounting outages are now absorbed automatically instead of needing someone to re-send the payout.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

07-webhook-retry evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 3 of the 5 measures, the bare arm 1. bare transcript, skill transcript.

before, run without the skill
Retry accounting webhook on network errors, 429 and 5xx

Previously a single failed call was logged and dropped, so payouts never
reached accounting. Now retries up to 4 attempts with exponential backoff
and throws if all fail so callers can handle it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
feat(webhook): retry accounting notifications on transient failures

Retries network errors, 429 and 5xx up to 4 attempts with exponential
backoff (500ms, 1s, 2s). Other 4xx are not retried. notifyAccounting now
returns true/false instead of undefined.

For the customer:
What changed: A single failed call to accounting no longer loses a payout; the system tries again a few times before giving up.
Why it matters: Fewer payouts go missing from accounting, so the finance assistant finds fewer gaps at reconciliation.
Automation effect: Brief accounting outages are now absorbed without anyone re-sending payouts by hand.

Not shipped: no persistent queue, so if accounting is down for longer than the retry window (about 3.5s) the payout is still only logged. Tests use a fake fetch; not observed against the real accounting endpoint.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Where the skill did nothing, or made the message worse (38 pairs)

03-refactor-pricing evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. The skill arm passed 1 of the 5 measures, the bare arm 2; the skill arm passed fewer. bare transcript, skill transcript.

before, run without the skill
Refactor pricing rules into modular rule array

Replace monolithic pricing function with a rules-based system. Each rule
is now a separate object that can be added, removed, or modified without
touching other rules. This makes it safe for marketing to request pricing
changes without risk of inadvertently breaking other rules.
after, run with the skill
Restructure pricing rules as independent rule objects

Each pricing rule is now a self-contained object that can be added or removed from the pricingRules array without touching other rules or the main price function. Marketing can now change rules safely without risking other logic.

Rules can be easily:
- Added: append a new rule object with name and apply function
- Removed: delete the rule from the array
- Modified: update only the specific rule's apply function

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

03-refactor-pricing evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 1 of the 5 measures, the bare arm 2; the skill arm passed fewer. bare transcript, skill transcript.

before, run without the skill
Refactor pricing rules into independent, composable rule objects

Each pricing rule is now a self-contained object in an array, making it safe
to add or remove rules without touching the main price calculation logic.
Rules can be modified, disabled, or extended without side effects.
after, run with the skill
Restructure pricing rules for independent configuration

Changed from a monolithic price function to a composable rules array. Each pricing rule (weekend surcharge, loyalty discount, summer pricing, long-stay discount) is now isolated in its own function. Rules can be added, removed, or modified without touching others, making future pricing changes safer and less risky.

06-invoice-typo evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. The skill arm passed 1 of the 5 measures, the bare arm 2; the skill arm passed fewer. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: 'welcom' → 'welcome'

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix typo in invoice footer: 'welcom' → 'welcome'

06-invoice-typo evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer

A guest reported "welcom" in the invoice footer text; corrected to
"welcome".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Fix typo in invoice footer

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

07-webhook-retry evals/results-2026-10-17/done-means-observed

The skill arm ran the done-means-observed skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Retry the accounting webhook on transient failures

A single failed call dropped the payout silently, so it never reached
accounting and only turned up later during reconciliation.

Retry up to 4 attempts with exponential backoff and jitter on network
errors and transient statuses (408, 429, 5xx). Other 4xx are not retried
since the request itself is the problem. Each attempt sends the same
idempotency-key (the payout id) so a redelivery after a lost response
doesn't double-post.

notifyAccounting now returns whether delivery succeeded; it still does
not throw, so callers are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
after, run with the skill
Retry the accounting webhook and surface terminal failures

Payouts were being lost when a single call failed: the only handling was
a console.error, so the caller saw success either way.

- Retry up to 4 attempts on network errors, 429 and 5xx, with exponential
  backoff plus jitter.
- Fail fast on other 4xx, which retrying cannot fix.
- Throw once attempts are exhausted instead of swallowing the failure.
- Send a stable idempotency-key header from the payout id so retries do
  not create duplicate accounting entries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

01-export-safari evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix Export button not working on Safari on iPad

Safari doesn't support the File System Access API. Add a fallback
that uses Blob download for browsers without showSaveFilePicker support.
after, run with the skill
Fix Export button on Safari/iPad by using blob download instead of File System Access API

The File System Access API (showSaveFilePicker) is not supported on Safari,
particularly on iPad. Replace it with the universally-compatible blob download
approach that works across all browsers.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

01-export-safari evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix Export button not working on Safari/iPad

Replace File System Access API with Blob-based download approach for broader browser compatibility. The File System Access API (showSaveFilePicker) has poor support in Safari, especially on iPad. Using a temporary anchor element with the download attribute works reliably across all browsers.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix Export button on Safari iPad

Replace unsupported File System Access API with standard blob-based download
that works across all browsers including Safari on iPad.

Changes:
- Remove showSaveFilePicker (not supported in Safari)
- Use Blob API with createObjectURL for reliable cross-browser file download
- Simplify function from async to sync since no await needed

02-notification-badge evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add notification count badge to bell

Display the count of notifications that need attention as a badge on the bell button, showing the front desk manager at a glance how many items require action.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Add notification count badge to bell

Display the number of unread notifications that need action on the bell button, so the front desk manager can see at a glance how many items require their attention.

02-notification-badge evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add notification count badge to bell
after, run with the skill
Add notification count badge to bell icon

Displays the number of unread notifications that need action from a person.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

02-notification-badge evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add notification count badge to bell showing pending actions

Display the number of notifications that require person action (needsPerson: true)
on the bell UI element.
after, run with the skill
Add notification count badge to bell

Display the count of unread notifications requiring action on the notification bell, so the front desk manager can see at a glance how many tasks are pending.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

03-refactor-pricing evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Restructure pricing rules as independent, composable rules

Move pricing rules from conditional logic into a declarative rule array. Each rule is self-contained with its own condition and multiplier, so rules can be added or removed without touching others. This eliminates the risk of unintended side effects when marketing requests pricing changes.
after, run with the skill
Refactor pricing into modular rule-based system

Extracted pricing rules into individual, composable functions in a rules array.
Each rule can now be added or removed without touching others, making it safe
for frequent marketing requests to change pricing without risking other rules.

04-slow-report evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Optimize monthly report: avoid recreating Date objects in loop

Move Date(b.checkOut) outside the inner loop to prevent creating
thousands of Date objects per booking. Creates each once instead of
once per day in the date range.
after, run with the skill
Optimize monthly report by caching checkout date

Move Date object creation outside the loop to avoid redundant allocations on each iteration. This significantly improves performance for bookings with long date ranges.

04-slow-report evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Optimize monthlyReport performance by caching checkOut date

Avoid creating a new Date object on every loop iteration when checking the condition. Cache checkOut once before the loop to eliminate repeated expensive Date instantiation.
after, run with the skill
Optimize monthly report performance

Pre-compute checkout date instead of creating a new Date object on every loop iteration. This significantly reduces garbage collection overhead and improves report generation speed.

04-slow-report evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Optimize monthly report performance by computing checkOut once per booking

Move Date object creation outside the loop to avoid recreating it on every
iteration. This eliminates redundant Date instantiation that was causing
significant slowdown on bookings with many days.
after, run with the skill
Optimize monthly report by avoiding redundant Date construction

Create the checkout date once outside the loop instead of constructing
it on every iteration, eliminating wasteful Date object allocation.

05-timezone-count evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix broken check-in reminders for guests with null timezone

4 guests (IDs 3, 6, 9, 12) had null timezones, causing their reminder hour
calculation to return NaN. Default to UTC (offset 0) for guests with missing
timezone data instead of propagating undefined through the calculation.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix timezone handling for guests with null timezone values.

Previously, guests without a configured timezone received null reminder times, causing broken reminders. Now defaults to UTC (9:00 local) for guests with missing timezone data.

Fixes: 4 affected guests (IDs 3, 6, 9, 12)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

05-timezone-count evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix check-in reminder timezone handling for guests with missing timezone

4 guests (IDs 3, 6, 9, 12) had null timezone values, which resulted in NaN reminder hours.
Now using Europe/London as a sensible default for guests without a specified timezone.

All guests now receive valid reminder times:
- Istanbul guests: 09:00 local (06:00 UTC)
- Default/London guests: 09:00 local (08:00 UTC)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Fix timezone handling for guests with null timezone

4 guests (IDs 3, 6, 9, 12) had null timezone values, causing reminderHour() to return NaN and breaking reminder scheduling. Added null checks and fallback to UTC (offset 0) for missing timezones to ensure all guests receive working reminders.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

06-invoice-typo evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: "welcom" → "welcome"
after, run with the skill
Fix typo in invoice footer: "welcom" -> "welcome"

06-invoice-typo evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: 'welcom' → 'welcome'
after, run with the skill
Fix typo in invoice footer
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

07-webhook-retry evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add retries to accounting webhook with exponential backoff

Prevents payouts from being lost due to transient failures. Retries up to 3 times with exponential backoff (1s, 2s, 4s) on 5xx errors or network failures. Non-retryable 4xx errors fail immediately.
after, run with the skill
Add exponential backoff retries to accounting webhook

Retries up to 3 times on transient failures (5xx errors and rate limits) to prevent payouts from being lost when the accounting service briefly fails.

07-webhook-retry evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add retries to accounting webhook with exponential backoff

Retrying up to 3 times with exponential backoff (100ms, 200ms, 400ms) ensures
transient network failures don't cause payouts to be lost. Returns immediately
on success and logs error only after all retries are exhausted.
after, run with the skill
Add retries to accounting webhook

Retry transient failures (5xx, 408, 429) with exponential backoff to prevent payouts from being lost when the webhook call fails. Non-retryable errors (4xx except 408/429) fail immediately.

07-webhook-retry evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Add retries to accounting webhook

Retry failed webhook calls up to 3 times with exponential backoff to handle transient failures. This ensures payouts reliably reach accounting even if the service is temporarily unavailable.

- Retries on network errors and 5xx responses
- Skips retries for 4xx client errors
- Uses exponential backoff (1s, 2s, 4s) between attempts
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Add retries with exponential backoff to accounting webhook

Transient failures now retry up to 3 times with exponential backoff (100ms, 200ms, 400ms). Client errors (4xx) fail immediately without retries. This prevents payouts from silently dropping when the accounting service experiences temporary issues.

08-remove-coupons evals/results-claude-haiku-4-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code functionality from checkout.
after, run with the skill
Remove coupon code path from checkout

Coupons were retired months ago. Remove the applyCoupon function and
code parameter to prevent customers from accessing deprecated coupon functionality.

08-remove-coupons evals/results-claude-haiku-4-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove coupon code logic from checkout

Coupons have been retired. Removed the applyCoupon function and the code parameter from checkout to prevent any coupon path from being triggered.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
after, run with the skill
Remove coupon code path from checkout

Coupons were retired months ago. Remove the applyCoupon function and the code parameter from checkout to prevent coupon codes from being used.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

08-remove-coupons evals/results-claude-haiku-4-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Coupons were retired months ago. Remove the applyCoupon function and the
code parameter from the checkout function to prevent any code paths that
trigger coupon processing.
after, run with the skill
Remove coupon code path from checkout

Coupons were retired months ago and should not be accessible. Removed the applyCoupon function and the code parameter from the checkout function to prevent any coupon processing.

01-export-safari evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 3 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix bookings export on Safari by falling back to Blob download

showSaveFilePicker is unsupported in Safari, so Export did nothing on iPads.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
fix(export): fall back to a download link when Safari has no save picker

Safari does not implement showSaveFilePicker, so the call threw and the
Export button silently did nothing. Use the picker where it exists and
otherwise download bookings.csv via a Blob link.

For the customer:
What changed: Export on the bookings screen now saves the CSV on iPads using Safari; before, tapping it did nothing.
Why it matters: Reception staff at both hotels can export the day's bookings from their iPads again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

01-export-safari evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix bookings export on Safari by falling back to blob download

Safari doesn't implement showSaveFilePicker, so the Export button threw
and did nothing. Use it when available, otherwise download via an anchor.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix bookings export on Safari by falling back to a download link

Safari doesn't implement showSaveFilePicker, so the Export button threw
and did nothing. Use a Blob download when the picker is unavailable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

01-export-safari evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix bookings export on Safari with download fallback

Safari lacks showSaveFilePicker, so the export threw and did nothing.
Fall back to a Blob download when the API is unavailable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix bookings export on Safari by falling back to a download link

showSaveFilePicker isn't available in Safari, so the Export button threw
and did nothing on iPads. Use a Blob download when it's missing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

03-refactor-pricing evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Split pricing into independent rules

Each rule is its own module in src/rules, registered in index.js.
Behaviour is unchanged; a test compares against the original function.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Split pricing into one file per rule

Each pricing rule (weekend, loyalty, summer, long-stay) now lives in
src/rules/ and is listed in src/rules/index.js; price() just multiplies
the factors. Adding or removing a rule touches only that rule's file and
one line in the index. Output is unchanged: a test compares the new code
against a copy of the old function over 336 date/loyalty/stay/base combinations
(7 dates x 4 loyalty values x 4 stay lengths x 3 base prices).

For the customer:
What changed: Pricing rules can now be added or removed one at a time without risk to the other rules. Prices quoted today are identical.
Why it matters: Marketing's rule changes can ship faster and with less worry about breaking unrelated discounts or surcharges.

Not shipped: rule ordering/priority and rule config from outside code; rules are multipliers, so order does not affect the result.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

04-slow-report evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Speed up monthly report day counting

Parse each booking's dates once and step by integer ms instead of
re-creating Date objects every iteration; cache ISO day keys.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Speed up monthly occupancy report

Count booked days with plain millisecond arithmetic and cache each day's
date string, instead of building a Date and formatting it for every
booking-day. Output is identical (same keys, counts and key order).

Measured with a throwaway script (not committed): 20,000 generated
bookings of 1-14 nights, one run each on this machine, old vs new output
compared with deepStrictEqual on entries: 115.1 ms -> 10.3 ms.
Not checked: the real monthly email job or real owner data.

For the customer:
What changed: The monthly occupancy report is calculated much faster, with the same figures as before.
Why it matters: Owners get their report without the long wait, and the numbers they see are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

04-slow-report evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by iterating over timestamps

Avoid allocating a new Date for checkOut on every loop iteration and
mutating a Date per day; use numeric millisecond arithmetic instead.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Speed up monthly report by avoiding per-night Date allocations

Iterate over millisecond timestamps instead of creating and mutating Date
objects per night, and cache the ISO date key per day. Output (including
key order) is identical to the previous implementation.

On 20,000 synthetic bookings: ~190 ms -> ~11 ms.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

04-slow-report evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Speed up monthlyReport by avoiding per-day Date allocation

Iterate over epoch milliseconds and cache the ISO date key per day
instead of mutating a Date and calling toISOString for every night.
Output is identical; ~10x faster on a 20k-booking benchmark.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Speed up monthly occupancy report

For the customer: the monthly occupancy report now builds about 15x faster
on a synthetic set of 20,000 bookings (160 ms to 10 ms). The numbers in it
are unchanged. Not measured on real owner data or on the email send.

The loop built a Date and an ISO string for every booking-night. It now
counts by integer day and formats each distinct day once. Keys come out in
date order instead of booking order.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

06-invoice-typo evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: welcom -> welcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix typo in invoice footer: welcom -> welcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

06-invoice-typo evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: welcom -> welcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix typo in invoice footer (welcom -> welcome)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

06-invoice-typo evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Fix typo in invoice footer: welcom -> welcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Fix typo in invoice footer: welcom -> welcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

07-webhook-retry evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 1 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Retry accounting webhook on transient failures

Retry network errors, 429 and 5xx with exponential backoff (4 attempts),
and throw after exhausting retries instead of silently logging.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Retry accounting webhook on transient failures

Retry up to 4 attempts with exponential backoff (500ms base) on network
errors, 429 and 5xx. Other 4xx fail immediately. After the final failure
the error is thrown instead of only logged, so callers can see that a
payout was not delivered.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

08-remove-coupons evals/results-claude-sonnet-5-5-2026-10-20, run 1

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

checkout() no longer accepts a code or applies discounts; total equals subtotal.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

08-remove-coupons evals/results-claude-sonnet-5-5-2026-10-20, run 2

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

08-remove-coupons evals/results-claude-sonnet-5-5-2026-10-20, run 3

The skill arm ran the product-engineer skill. Both arms passed 2 of the 5 measures: the skill changed nothing here. bare transcript, skill transcript.

before, run without the skill
Remove retired coupon code path from checkout

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
after, run with the skill
Remove retired coupon code path from checkout

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Regenerate with node scripts/generate-gallery.mjs. The drift test in test/gallery.test.mjs regenerates into a temp file and compares bytes with this page. Transcripts are committed under evals/results*; the pairs are only what they say.