The patcher uploads the patched ROM and then fires a platform scan to
register it. When a second scan runs against the same platform around the
same time (a filesystem-watcher rescan, a scheduled rescan, or another
manual scan on a multi-worker setup), both scans could see the new file as
absent from the DB and each insert it, producing two identical library
entries for one patched file.
A platform folder can't physically hold two entries with the same name, so
a ROM is uniquely identified by (platform_id, fs_name). Enforce that with a
unique index instead of the previous plain index, which makes the duplicate
impossible. The scan's early ROM insert now adopts the row created by a
concurrent scan (catching the integrity error and skipping) instead of
failing, and ROM rename pre-checks for a name collision so it returns a
clean 409 rather than hitting the constraint.
Includes a migration that removes any pre-existing duplicates (keeping the
lowest id; dependents cascade) before upgrading the index to unique.
Fixes#3590
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0135UV8Xn2XHkRhjzhm9UptP
The /api/roms list endpoint did several O(library) computations on every
request. On a 100k-rom library each request took 4-5s. This addresses the
dominant costs, all measured on a real 100k-rom MariaDB.
- Cache rom_id_index: the full ordered id list backing virtual scroll was
recomputed (the sibling-dedup window over the whole library) on every
request, even limit=1, and shipped uncached. Memoise the unscoped scan
under the same versioned cache as the other sidecars. 2815ms -> 7ms on hit.
- Slim the sibling-dedup query: the inner derived table materialized all of
Rom (including JSON metadata blobs) for 100k rows, and carried a wide unused
fs_name_no_ext through the window's temp table (spilling the sort to disk),
plus a pointless inner ORDER BY. Select only the columns the window needs.
2.79s -> 0.86s, identical results, no schema change.
- Rewrite with_char_index: replace row_number() over the whole library (full
materialization + double filesort) with a per-letter COUNT and an
accumulate. Identical output, drops a filesort layer.
- Add idx_roms_sibling_cover covering index for the sibling_roms view
self-join, so the 7-way metadata-id OR resolves from the index instead of
reading wide rows per parent. ~8x on dense pages warm, far more cold.
AI assistance: written with Claude Code (diagnosis, query rewrites, migration,
tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an RBox3D primitive that builds a rotatable, fake-3D game box from
three flat ScreenScraper scans (front, back, spine) using CSS 3D
transforms. Box proportions derive from the images themselves; it rotates
via pointer drag, arrow keys / gamepad D-pad, and the right analog stick,
drifts gently when idle, and honours prefers-reduced-motion.
The game detail hero (CoverColumn) upgrades to the spinning box when the
"3D box" boxart style is selected and the rom has the full set of faces,
falling back to the flat cover otherwise.
Backend: persist the box-2D-side (spine) scan locally, mirroring the
existing box-2D-back handling — new BOX2D_SIDE media type + box2d_side_path
on ss_metadata, opt-in via scan.media.
- RBox3D primitive + Storybook story (controls + keyboard-rotation play())
- useBoxFaces composable resolving the three faces + a `complete` gate
- box3d-alt i18n key across all locales
- backend BOX2D_SIDE persistence + tests
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019itLXRfJXGGbhPY3JyqnuN
Show a "where you left off" screenshot on the Home continue-playing rail and
the live-activity board, with a small cover-art thumbnail (PIP) in the corner
so the game stays identifiable. Both render at the image's natural aspect.
Backend:
- New shared util `continue_playing_screenshot(rom, latest_save)` resolving the
image in priority order: latest save's screenshot, then title screen, then
first gameplay screenshot (None → frontend falls back to cover art).
- `SimpleRomSchema.screenshot_path` populated only on the `last_played` query;
`get_latest_saves_for_roms` batch handler (+ tests).
- ActivityEntry / ActivityEntrySchema gain `screenshot_path`, computed from the
session player's latest save in both the socket and REST heartbeat paths.
Frontend:
- New shared `CoverArtPip.vue` (bottom-right 2D cover thumbnail), reused by
GameCard and ActivityCard.
- Home continue-playing rail uses `screenshot_path` + PIP, natural aspect (no
forced hero/style).
- Activity board: screenshot-forward cover + PIP, and a wrapping flex layout so
cards share a uniform height with natural-ratio widths (gallery-card
behavior).
- GameCover only keys the measured ratio by rom id for the rom's own cover, so
a `coverSrc` override (screenshot) never pollutes the gallery's ratio cache.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gate automatic account creation on OIDC login behind a new OIDC_ALLOW_REGISTRATION environment variable. Defaults to true, preserving the current auto-provisioning behavior; set it to false to run OIDC in an "existing users only" mode, where a login from an email without an existing RomM account is rejected with a 403 instead of silently creating one. Existing users are unaffected either way.
Adds the config constant, env.template and docs entries, and tests covering the enabled/disabled and existing-user paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the name_sort_key_custom flag/migration in favour of a flagless rule: a
key is "custom" when it no longer equals compute(name). Apply that consistently
across all three write paths so a manual sort key survives renames while a
derived key keeps following the name:
- @validates re-derives on name assignment only when the stored key still
matches the derived value; direct name_sort_key assignment stores a
normalized custom key (or reverts to derived when cleared). Handles both
kwarg orders at construction.
- update_rom mirrors the same check for the bulk update() path it bypasses.
- The edit endpoint only writes the key when the user actually changed the
field, delegating the untouched case to update_rom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the separate `sort_name` column into `name_sort_key`, which is now
the single user-settable sort field: always normalized and indexed for fast
ordering, derived from `name` by default, and overridable. A new
`name_sort_key_custom` boolean marks user/metadata overrides so they survive
renames and rescans.
- Drop the `roms.sort_name` column; repurpose migration 0085 to add
`name_sort_key_custom`.
- Derive the key via `@validates("name")` unless pinned custom; the edit
dialog, unmatch flow, and ES-DE gamelist <sortname> set custom keys.
- update_rom / scan_rom keep the columns in sync explicitly (bulk update and
construction bypass / reorder the validator).
- Frontend: edit field drives name_sort_key (empty when auto), api sends the
override only when custom, regenerated types updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Include order_by / order_dir / group_by_meta_id in the char-index cache
key. The computed AlphaStrip positions depend on ordering and grouping,
so keying only by user reused a stale index when those changed.
- Gate the relevance-ranking test assertions to MySQL/MariaDB. Relevance
ordering uses MATCH ... AGAINST; PostgreSQL (also in the CI matrix) falls
back to name ordering, where those assertions don't hold.
- Clarify the order_by API docstring: relevance ordering applies on
MySQL/MariaDB; other databases fall back to name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralize the *_no_tags / *_no_ext / *_extension columns (derived from a
file name) behind @validates hooks instead of computing them by hand at
every write site:
- Add pure helpers (compute_file_name_parts and friends) to models.base;
the filesystem base handler now delegates to them.
- Add @validates on Rom (fs_name), BaseAsset (file_name, inherited by all
asset subclasses), and Firmware (file_name).
- update_rom keeps the fs_name-derived columns in sync on bulk update(),
which also fixes the rename path never updating fs_extension.
- Drop the now-redundant computations at the scan/rename call sites.
Also fix the migration backfill loop and a pre-existing list[str | None]
type mismatch surfaced in scan_handler. Add tests for the helpers, the
validators, and the update_rom bulk-sync path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements RFC 8628-style device authorization so clients
(argosy-launcher, grout) can pair by display instead of manually
copying tokens. Device posts to an open /api/auth/device/init with
its identifier and requested scopes; the server returns device_code
+ user_code + QR URL. User scans QR, lands at /pair/device, approves
(optionally editing name/scopes/expiry); the device's next poll on
/api/auth/device/token returns a ClientToken bound 1:1 to a newly-
created (or deduped) Device record. Downstream endpoints
(/play-sessions, /sync/negotiate) infer device_id from the bound
token so the client doesn't have to ship it on every call.
- Migrations 0080/0081: devices.client_device_identifier (unique
per user) and client_tokens.device_id FK (ON DELETE SET NULL)
- Five new endpoints under /api/auth/device (init/pending/approve/
deny/token) with Redis-backed state, per-IP rate limits, and
RFC-compliant error codes (authorization_pending, slow_down,
expired_token, access_denied)
- HybridAuthBackend surfaces bound device_id on request.state and
bumps devices.last_seen with a 5-minute debounce
- /api/users/me returns current_device_id for bound tokens so a
device can identify itself from its token alone
- Frontend approval screen at /pair/device with editable scopes/
name/expiry (defaults to Never), 3s auto-close countdown
- ClientApiTokens settings list shows bound-device chip
- 20 i18n keys added to all 17 locales; generated models updated
- 52 new tests across 13 classes; full suite 1334 passed
Planning and review assisted by Claude Code.
Adds a few new indexes to handle full-text searches instead of doing
`ILIKE` matching, improving performance substantially.
Alongside that, a few other things were done in order to improve search
performance, such as caching filter values so they're not computed on
each request to /api/roms. Overall, this should have a very noticeable
impact on large collections when using the search feature.
- Added "esbuild" version "^0.28.1" to frontend package overrides.
- Updated "exclude-newer-package" in pyproject.toml to include "vcrpy" with a date of "2026-06-17".
- Modified uv.lock to reflect the new "vcrpy" version "8.2.1" and removed platform-specific markers for dependencies.
Extract has_structure_path_a as its own cached property and have
has_structure_path_b delegate to it, removing duplicated isdir checks.
detect_library_structure and get_platforms_directory now read the named
properties instead of re-implementing the roms-path check inline.
Keep the inconclusive/bad-structure fallback defaulting to Structure A so
a malformed library raises FolderStructureNotMatchException rather than
listing the bare library root as a flat list of platforms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a top-level `roms/` folder exists (Structure A), never detect the
library as Structure B, even if individual `<platform>/roms/` directories
also exist. This prevents existing Structure A libraries from being broken
after upgrading to 4.9.0.
- `has_structure_path_b` in `config_manager.py` now returns `False` early
when `{LIBRARY_BASE_PATH}/{ROMS_FOLDER_NAME}` is an existing directory
- `detect_library_structure()` in `platforms_handler.py` now explicitly
checks Structure A (`os.path.exists(roms_path)`) before consulting
`cnfg.has_structure_path_b`
- Updated test to assert Structure A wins when both layouts coexist
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
Match the json-patch camelCase key names the Hasheous endpoint expects,
and update the lookup_rom test assertions accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Hasheous ByHash endpoint now accepts an array of hash objects rather
than a single one. Send the hashes of every top-level file (using
chd_sha1_hash exclusively for files that have one) to improve lookup
accuracy, instead of picking only the largest file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
saves responses now include one device_syncs entry per device that has
synced a save, not just the caller's, so clients can tell which devices
hold a save. is_current is computed per entry and the caller's own entry
is ordered first for backward compatibility.
add a saves.origin_device_id column (migration 0081) recording the
device that created a save, set on initial upload only, surfaced as
origin_device_id on the save schema.
ScreenScraper matching skipped the stronger jeuInfos (romnom + systemeid)
lookup for any file without a hash, falling straight through to the weaker
jeuRecherche name search. Files are un-hashed for NON_HASHABLE_PLATFORMS
(PS3/4/5, Switch, Wii U, Xbox, etc.) and whenever SKIP_HASH_CALCULATION is
set, so those platforms matched worse than they could.
The transport already supports a hash-less jeuInfos?romnom=...&systemeid=...
request, so relax lookup_rom's early-return: only bail when there is neither
a hash nor a filename to match on. jeuRecherche stays the last-resort
fallback, keeping this quota-neutral.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scan loaders no longer eager-load `Rom.files` (#3425 + follow-ups), so
the hash-based metadata lookups can't rely on `rom.files` being populated —
`hasheous`/`ss` `lookup_rom` read `RomFile.is_top_level`, which dereferences
`RomFile.rom.full_path` and would raise `DetachedInstanceError` once the
session closed.
Add `DBRomsHandler.rom_files_for_rom_id`, which loads a ROM's files on demand
with the `RomFile.rom` backref eager-loaded (`load_only(fs_path, fs_name)`).
The scan path uses it as a fallback only when the filesystem walk yielded no
files (e.g. an unchanged rescan), behind a per-ROM `functools.cache` helper so
the playmatch/hasheous/ss lookups share a single DB fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SS metadata handler pre-encoded the name-search term with quote()
before handing it to the service layer, which percent-encodes the query
again via yarl's with_query(). This double-encoded any character that
needs URL-encoding (e.g. "+" -> "%2B" -> "%252B"), so the request URL
carried a doubly-escaped term.
Pass the raw (unidecode-transliterated but un-percent-encoded) term to
search_games() in both _search_rom() and get_matched_roms_by_name() and
let the URL builder encode it exactly once. The scan now sends e.g.
recherche=...%2B... instead of ...%252B...
This is a request-correctness fix. It does not, on its own, make every
previously-unmatched title match: ScreenScraper's jeuRecherche normalizes
punctuation and applies its own relevance ranking, so some titles still
return no results for the full filename-derived term (verified directly
against the API). Improving name-search robustness is a separate concern.
Add TestSearchTermEncoding regression tests covering the un-pre-encoded
term, preserved unidecode transliteration, and a single-encoded request
URL (%2B, never %252B).
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restore the "platform only" contract of `get_roms_by_fs_name` (per its
docstring) by dropping the `selectinload(Rom.files)` + `joinedload`. That
load only existed for `scan_rom`'s rare `fs_rom["files"] or rom.files`
fallback, but it forced files (and a per-file join back to roms) for every
ROM in a scan batch — expensive on large platforms, and only used when the
filesystem scan yielded no files.
Instead, fetch the persisted files on demand: `scan_rom` now resolves match
files via a small helper that returns the filesystem-scanned files, falling
back to `db_rom_handler.get_rom_files_by_rom_id(rom.id)` only when there are
none. The new getter eager-loads the `RomFile.rom` backref so `is_top_level`
keeps working on the detached results (the rare path was already latently
broken on master, which loaded files without the backref).
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
Add a shared `multi_file_rom` fixture (a game folder with multiple
RomFile rows) and an endpoint-level test that downloads it via
`GET /api/roms/{id}/content/{file_name}`. This exercises the multi-file
download path end-to-end, which builds each mod_zip manifest entry from
`file.rom.full_path` after the handler session has closed — the exact
path that 500'd with `DetachedInstanceError` before the backref fix.
The download endpoint had no test coverage for multi-file ROMs (the
`rom` fixture has no RomFile rows), which is why the regression slipped
through. Reuse the new fixture in the handler-level regression test too.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
PR #3425 dropped `lazy="joined"` from `RomFile.rom` and removed the
`joinedload(RomFile.rom)` from the ROM loaders to speed up the gallery
query. That left the `RomFile.rom` backref unpopulated. Single-file
downloads only read `RomFile.full_path` (built from `file_path`/
`file_name`), so they kept working, but multi-file (game folder)
downloads call `file_name_for_download()` / `is_top_level`, which read
`self.rom.full_path`. With no eager-loaded backref, that triggered a
lazy load on a detached instance once the handler session closed,
raising `DetachedInstanceError` and returning a 500.
Rather than reverting the loader changes (and the gallery gains), wire
the `RomFile.rom` backref up in Python from the parent ROM we already
hold in memory, via `set_committed_value`. This is zero extra DB cost
and only runs on the detail/download paths (`with_details` and
`get_roms_by_fs_name`); the optimized `filter_roms` gallery query is
untouched.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
New Nintendo 3DS games never matched ScreenScraper because the platform
was missing from SCREENSAVER_PLATFORM_LIST. With no entry, get_platform()
returns ss_id=None and scan_handler skips the entire ScreenScraper lookup
(hash and filename) for the platform, reporting everything as unmatched.
ScreenScraper has no separate New 3DS system; New 3DS games live under the
regular Nintendo 3DS system (ID 17). Alias New Nintendo 3DS to that system,
matching the existing Famicom->NES, Super Famicom->SNES, and DSi->DS aliases.
Fixes#3464
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
IGDB catalogues a console and its regional twin as two separate
platforms (SNES/Super Famicom, NES/Famicom). RomM locked each IGDB
search to a single platform id, so a region-exclusive title catalogued
under only the twin — e.g. the Japan-only Super Famicom game
"Rudra no Hihou" (platform 58) scanned from an `snes` folder
(platform 19) — was filtered out before name matching ran and never
matched.
Include a platform's regional twin in the IGDB platform filter so both
are searched. A non-twin platform keeps the exact existing query
(`platforms=[19]`); a twin produces an OR group
(`(platforms=[19] | platforms=[58])`), leaving all other platforms and
recorded cassettes unchanged.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
IGDB scans dropped games whose filename uses a localized (non-English)
title even when that title exists in IGDB's alternative_names. The
alternative_name wildcard search surfaced the correct game, but
_search_rom() rebuilt its name->game candidate dict using only the
primary English name, so the Jaro-Winkler re-check scored the localized
term below threshold and discarded the match (issue #3435).
Add _index_games_by_searchable_name(), which indexes each game by its
primary name plus alternative_names and game_localizations titles, and
use it for both candidate-building passes in _search_rom(). Primary
names keep precedence (lowest-igdb-id tiebreak); alternative/
localization titles fill in only names not already claimed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The archive branch of get_rom_files (introduced in #3412) was missing
the RAHasherService.calculate_hash call that exists in the non-archive
branch, causing all archive-format ROMs to produce an empty ra_hash
during scanning regardless of platform.
The RA hash call is now made for archive ROMs, mirroring the existing
non-archive behaviour. The RA_BUFFER_HASH_UNSUPPORTED skip logic in
RAHasherService already handles disc-based platforms (PSX, PS2, PSP,
Saturn, Dreamcast, etc.) so those continue to be excluded automatically.
Also improves handling of folder-based multi-file ROMs whose directories
contain compressed files. RAHasher cannot process archives via the /*
glob and fails with "Could not open file". The fix mirrors the existing
CHD folder logic: for cartridge platforms the largest archive in the
folder is passed directly to RAHasher for buffer hashing; for disc
platforms the call is skipped as buffer hashing is unsupported.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When sending a hash lookup to ScreenScraper, romnom was always set to the
archive filename on disk (e.g. Mario.zip). For single-file archives, the hash
is computed from the internal file (e.g. mario.n64), so sending the archive
name sends slightly incorrect info to ss.fr during a KO scrape.
When archive_members has exactly one entry, romnom now uses that member's
name. Multi-file archives and non-archive files continue to use the filesystem
filename unchanged.
Closes#3444
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
