Extract has_structure_path_a as its own cached property and have
has_structure_path_b delegate to it, removing duplicated isdir checks.
detect_library_structure and get_platforms_directory now read the named
properties instead of re-implementing the roms-path check inline.
Keep the inconclusive/bad-structure fallback defaulting to Structure A so
a malformed library raises FolderStructureNotMatchException rather than
listing the bare library root as a flat list of platforms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a top-level `roms/` folder exists (Structure A), never detect the
library as Structure B, even if individual `<platform>/roms/` directories
also exist. This prevents existing Structure A libraries from being broken
after upgrading to 4.9.0.
- `has_structure_path_b` in `config_manager.py` now returns `False` early
when `{LIBRARY_BASE_PATH}/{ROMS_FOLDER_NAME}` is an existing directory
- `detect_library_structure()` in `platforms_handler.py` now explicitly
checks Structure A (`os.path.exists(roms_path)`) before consulting
`cnfg.has_structure_path_b`
- Updated test to assert Structure A wins when both layouts coexist
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
Match the json-patch camelCase key names the Hasheous endpoint expects,
and update the lookup_rom test assertions accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Folder-stored disc games (the standard Redump .cue + .bin layout, .gdi
sets, multi-bin, etc.) never matched RetroAchievements. roms_handler
builds an `ra_path` ending in `/*` when the folder has no .chd, and
RAHasherService only rescued that glob for folders containing archives.
A folder of plain uncompressed tracks fell through, so the literal `*`
reached RAHasher via create_subprocess_exec (no shell to expand it),
which failed with "Could not open track" and stored an empty ra_hash.
Resolve the `/*` glob to a single real file before spawning RAHasher:
prefer a disc descriptor (.cue/.gdi/.m3u), which RAHasher follows to the
referenced tracks, otherwise fall back to the largest file in the folder
(raw .iso/.bin, or the main file of a multi-file cartridge set). This
mirrors the existing "pick the largest .chd" handling for CHD folders.
Fixes#3497.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Hasheous ByHash endpoint now accepts an array of hash objects rather
than a single one. Send the hashes of every top-level file (using
chd_sha1_hash exclusively for files that have one) to improve lookup
accuracy, instead of picking only the largest file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The IGDB service acquires a rate-limiter slot before each request like the
other metadata services, but unlike them had no direct unit test asserting it
(IGDB is otherwise exercised via cassette-backed handler tests). Add a focused
test so the limiter call can't be silently removed or moved after the request.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
ScreenScraper enforces a per-account *thread* (concurrency) cap rather than
a request rate. Requests can take several seconds, so spacing out request
starts at 1/s could still leave multiple requests overlapping in flight,
exceeding the cap and getting rejected with ScreenScraper's custom HTTP codes.
- Add ConcurrencyLimiter: a runtime-resizable, loop-agnostic limiter that
bounds simultaneous in-flight operations (held for the whole request via
async context manager), instead of spacing request starts.
- Switch the ScreenScraper service from the req/s RateLimiter to a module-level
ConcurrencyLimiter defaulting to a single thread.
- Recognize contributor/donor accounts: parse `ssuser.maxthreads` from each
response and raise the concurrency allowance to match, so supporters scrape
with their full thread count instead of the conservative default.
Adds unit tests for the limiter (blocking, wake-on-release, runtime resize)
and for the ScreenScraper slot-holding and thread-allowance updates.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
Wire the existing RateLimiter into the IGDB, ScreenScraper, MobyGames and
RetroAchievements services so requests are spaced under each provider's
documented req/s cap, instead of only reacting to HTTP 429 after the fact.
- IGDB: 4 req/s (documented hard limit)
- MobyGames: 1 req/s (free-tier burst cap)
- ScreenScraper: 1 req/s (free-tier throttle)
- RetroAchievements: 4 req/s (conservative; no published hard limit)
A slot is acquired before both the initial request and the timeout/429
retry. The reactive 2s 429 backoff is kept as a fallback. Tests neutralize
the shared limiters via an autouse fixture and assert acquire() is awaited.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
saves responses now include one device_syncs entry per device that has
synced a save, not just the caller's, so clients can tell which devices
hold a save. is_current is computed per entry and the caller's own entry
is ordered first for backward compatibility.
add a saves.origin_device_id column (migration 0081) recording the
device that created a save, set on initial upload only, surfaced as
origin_device_id on the save schema.
ScreenScraper matching skipped the stronger jeuInfos (romnom + systemeid)
lookup for any file without a hash, falling straight through to the weaker
jeuRecherche name search. Files are un-hashed for NON_HASHABLE_PLATFORMS
(PS3/4/5, Switch, Wii U, Xbox, etc.) and whenever SKIP_HASH_CALCULATION is
set, so those platforms matched worse than they could.
The transport already supports a hash-less jeuInfos?romnom=...&systemeid=...
request, so relax lookup_rom's early-return: only bail when there is neither
a hash nor a filename to match on. jeuRecherche stays the last-resort
fallback, keeping this quota-neutral.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scan loaders no longer eager-load `Rom.files` (#3425 + follow-ups), so
the hash-based metadata lookups can't rely on `rom.files` being populated —
`hasheous`/`ss` `lookup_rom` read `RomFile.is_top_level`, which dereferences
`RomFile.rom.full_path` and would raise `DetachedInstanceError` once the
session closed.
Add `DBRomsHandler.rom_files_for_rom_id`, which loads a ROM's files on demand
with the `RomFile.rom` backref eager-loaded (`load_only(fs_path, fs_name)`).
The scan path uses it as a fallback only when the filesystem walk yielded no
files (e.g. an unchanged rescan), behind a per-ROM `functools.cache` helper so
the playmatch/hasheous/ss lookups share a single DB fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Renaming a ROM gives its file a new internal id, but the EmulatorJS
player keeps a remembered file id ("disc") in localStorage and reuses
it on the next launch. After a rename that id is stale, so the content
download endpoint matched zero files and fell through to its multi-file
ZIP path, producing a download whose only entry was an empty .m3u
playlist. nginx's mod_zip decode step rejects the blank value (HTTP
400) and aborts the response, sending 0 bytes — which EmulatorJS
surfaces as a generic "network error" (issue #3470).
The frontend half (validating the remembered disc against the ROM's
current files) already landed on master in d1696cd04. This is the
backend half: when no files match the request, raise a clean 404
instead of building a broken empty-.m3u ZIP. This also covers a ROM
with zero files.
Add endpoint tests (auth, single-file, valid file id, stale file id
-> 404, missing rom -> 404) plus a `rom_file` fixture.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The update_rom path already re-parses filename tags when fs_name
changes (master commit d7a896b5da), but the headline scenario from
issue #3471 — an untagged ROM renamed to add (Europe) so the region
flag appears — was never asserted; existing coverage only exercised
the tag-removal direction.
Add a test that renames the untagged rom fixture to "test_rom
(Europe).zip" and asserts regions == ["Europe"], locking down the
add-region direction described in the issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The five pkgj feed tests created a ROM but no RomFile, so the per-file
feeds emitted only a header and the "in response.text" assertions never
actually verified output (pre-existing failures, also red on master).
Add a top-level `.pkg` GAME file (games feeds) or a DLC-category file
(dlc feeds), mirroring the pkgi_ps3 test, so the feeds produce rows.
This also gives real coverage of the new `include_files=True` path that
these feeds rely on.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
The SS metadata handler pre-encoded the name-search term with quote()
before handing it to the service layer, which percent-encodes the query
again via yarl's with_query(). This double-encoded any character that
needs URL-encoding (e.g. "+" -> "%2B" -> "%252B"), so the request URL
carried a doubly-escaped term.
Pass the raw (unidecode-transliterated but un-percent-encoded) term to
search_games() in both _search_rom() and get_matched_roms_by_name() and
let the URL builder encode it exactly once. The scan now sends e.g.
recherche=...%2B... instead of ...%252B...
This is a request-correctness fix. It does not, on its own, make every
previously-unmatched title match: ScreenScraper's jeuRecherche normalizes
punctuation and applies its own relevance ranking, so some titles still
return no results for the full filename-derived term (verified directly
against the API). Improving name-search robustness is a separate concern.
Add TestSearchTermEncoding regression tests covering the un-pre-encoded
term, preserved unidecode transliteration, and a single-encoded request
URL (%2B, never %252B).
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restore the "platform only" contract of `get_roms_by_fs_name` (per its
docstring) by dropping the `selectinload(Rom.files)` + `joinedload`. That
load only existed for `scan_rom`'s rare `fs_rom["files"] or rom.files`
fallback, but it forced files (and a per-file join back to roms) for every
ROM in a scan batch — expensive on large platforms, and only used when the
filesystem scan yielded no files.
Instead, fetch the persisted files on demand: `scan_rom` now resolves match
files via a small helper that returns the filesystem-scanned files, falling
back to `db_rom_handler.get_rom_files_by_rom_id(rom.id)` only when there are
none. The new getter eager-loads the `RomFile.rom` backref so `is_top_level`
keeps working on the detached results (the rare path was already latently
broken on master, which loaded files without the backref).
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
Add a shared `multi_file_rom` fixture (a game folder with multiple
RomFile rows) and an endpoint-level test that downloads it via
`GET /api/roms/{id}/content/{file_name}`. This exercises the multi-file
download path end-to-end, which builds each mod_zip manifest entry from
`file.rom.full_path` after the handler session has closed — the exact
path that 500'd with `DetachedInstanceError` before the backref fix.
The download endpoint had no test coverage for multi-file ROMs (the
`rom` fixture has no RomFile rows), which is why the regression slipped
through. Reuse the new fixture in the handler-level regression test too.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
PR #3425 dropped `lazy="joined"` from `RomFile.rom` and removed the
`joinedload(RomFile.rom)` from the ROM loaders to speed up the gallery
query. That left the `RomFile.rom` backref unpopulated. Single-file
downloads only read `RomFile.full_path` (built from `file_path`/
`file_name`), so they kept working, but multi-file (game folder)
downloads call `file_name_for_download()` / `is_top_level`, which read
`self.rom.full_path`. With no eager-loaded backref, that triggered a
lazy load on a detached instance once the handler session closed,
raising `DetachedInstanceError` and returning a 500.
Rather than reverting the loader changes (and the gallery gains), wire
the `RomFile.rom` backref up in Python from the parent ROM we already
hold in memory, via `set_committed_value`. This is zero extra DB cost
and only runs on the detail/download paths (`with_details` and
`get_roms_by_fs_name`); the optimized `filter_roms` gallery query is
untouched.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
New Nintendo 3DS games never matched ScreenScraper because the platform
was missing from SCREENSAVER_PLATFORM_LIST. With no entry, get_platform()
returns ss_id=None and scan_handler skips the entire ScreenScraper lookup
(hash and filename) for the platform, reporting everything as unmatched.
ScreenScraper has no separate New 3DS system; New 3DS games live under the
regular Nintendo 3DS system (ID 17). Alias New Nintendo 3DS to that system,
matching the existing Famicom->NES, Super Famicom->SNES, and DSi->DS aliases.
Fixes#3464
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
IGDB catalogues a console and its regional twin as two separate
platforms (SNES/Super Famicom, NES/Famicom). RomM locked each IGDB
search to a single platform id, so a region-exclusive title catalogued
under only the twin — e.g. the Japan-only Super Famicom game
"Rudra no Hihou" (platform 58) scanned from an `snes` folder
(platform 19) — was filtered out before name matching ran and never
matched.
Include a platform's regional twin in the IGDB platform filter so both
are searched. A non-twin platform keeps the exact existing query
(`platforms=[19]`); a twin produces an OR group
(`(platforms=[19] | platforms=[58])`), leaving all other platforms and
recorded cassettes unchanged.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
IGDB scans dropped games whose filename uses a localized (non-English)
title even when that title exists in IGDB's alternative_names. The
alternative_name wildcard search surfaced the correct game, but
_search_rom() rebuilt its name->game candidate dict using only the
primary English name, so the Jaro-Winkler re-check scored the localized
term below threshold and discarded the match (issue #3435).
Add _index_games_by_searchable_name(), which indexes each game by its
primary name plus alternative_names and game_localizations titles, and
use it for both candidate-building passes in _search_rom(). Primary
names keep precedence (lowest-igdb-id tiebreak); alternative/
localization titles fill in only names not already claimed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
