The IGDB service acquires a rate-limiter slot before each request like the
other metadata services, but unlike them had no direct unit test asserting it
(IGDB is otherwise exercised via cassette-backed handler tests). Add a focused
test so the limiter call can't be silently removed or moved after the request.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
ScreenScraper enforces a per-account *thread* (concurrency) cap rather than
a request rate. Requests can take several seconds, so spacing out request
starts at 1/s could still leave multiple requests overlapping in flight,
exceeding the cap and getting rejected with ScreenScraper's custom HTTP codes.
- Add ConcurrencyLimiter: a runtime-resizable, loop-agnostic limiter that
bounds simultaneous in-flight operations (held for the whole request via
async context manager), instead of spacing request starts.
- Switch the ScreenScraper service from the req/s RateLimiter to a module-level
ConcurrencyLimiter defaulting to a single thread.
- Recognize contributor/donor accounts: parse `ssuser.maxthreads` from each
response and raise the concurrency allowance to match, so supporters scrape
with their full thread count instead of the conservative default.
Adds unit tests for the limiter (blocking, wake-on-release, runtime resize)
and for the ScreenScraper slot-holding and thread-allowance updates.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
Wire the existing RateLimiter into the IGDB, ScreenScraper, MobyGames and
RetroAchievements services so requests are spaced under each provider's
documented req/s cap, instead of only reacting to HTTP 429 after the fact.
- IGDB: 4 req/s (documented hard limit)
- MobyGames: 1 req/s (free-tier burst cap)
- ScreenScraper: 1 req/s (free-tier throttle)
- RetroAchievements: 4 req/s (conservative; no published hard limit)
A slot is acquired before both the initial request and the timeout/429
retry. The reactive 2s 429 backoff is kept as a fallback. Tests neutralize
the shared limiters via an autouse fixture and assert acquire() is awaited.
https://claude.ai/code/session_01133QQuWvq8Zm25DZMP9PVr
PR #3388 added hardlink-based asset import/export (os.link with a
shutil.copy2 fallback) to avoid duplicating disk space. The Dockerfile
VOLUME instruction listed each /romm subdirectory (resources, library,
assets, config, sync) separately, which makes Docker create an
independent mount point for each one — even when the user bind-mounts a
single parent at /romm. Each mount point is its own st_dev, so every
cross-directory os.link() failed with EXDEV and silently fell back to a
full copy, defeating the optimization.
Declare the parent /romm directory instead so all subdirectories share a
single filesystem and hardlinks can succeed when paths reside on the same
underlying host filesystem.
The default Docker image symlinked /romm/assets into the nginx static web
root (/assets/romm/assets), where it was served by an unauthenticated
`location /assets { try_files ... }` block. /romm/assets holds private user
data (save files, save states, screenshots, avatars) that is meant to be
accessible only through the authenticated /api/raw/assets/{path} route
(Scope.ASSETS_READ). The static symlink bypassed that protection, letting any
unauthenticated caller read another user's files given a (guessable) path.
Avatar URLs leaked the hex user ID through the same static route, making path
construction straightforward.
Fix:
- Drop the /romm/assets symlink from the Docker image build and both
entrypoint scripts; only /romm/resources (public cover art, screenshots,
manuals) remains statically served.
- Point the frontend avatar URLs at the authenticated /api/raw/assets/ route
instead of /assets/romm/assets/. Browser <img> loads authenticate via the
existing session cookie.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
saves responses now include one device_syncs entry per device that has
synced a save, not just the caller's, so clients can tell which devices
hold a save. is_current is computed per entry and the caller's own entry
is ordered first for backward compatibility.
add a saves.origin_device_id column (migration 0081) recording the
device that created a save, set on initial upload only, surfaced as
origin_device_id on the save schema.
ScreenScraper matching skipped the stronger jeuInfos (romnom + systemeid)
lookup for any file without a hash, falling straight through to the weaker
jeuRecherche name search. Files are un-hashed for NON_HASHABLE_PLATFORMS
(PS3/4/5, Switch, Wii U, Xbox, etc.) and whenever SKIP_HASH_CALCULATION is
set, so those platforms matched worse than they could.
The transport already supports a hash-less jeuInfos?romnom=...&systemeid=...
request, so relax lookup_rom's early-return: only bail when there is neither
a hash nor a filename to match on. jeuRecherche stays the last-resort
fallback, keeping this quota-neutral.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scan loaders no longer eager-load `Rom.files` (#3425 + follow-ups), so
the hash-based metadata lookups can't rely on `rom.files` being populated —
`hasheous`/`ss` `lookup_rom` read `RomFile.is_top_level`, which dereferences
`RomFile.rom.full_path` and would raise `DetachedInstanceError` once the
session closed.
Add `DBRomsHandler.rom_files_for_rom_id`, which loads a ROM's files on demand
with the `RomFile.rom` backref eager-loaded (`load_only(fs_path, fs_name)`).
The scan path uses it as a fallback only when the filesystem walk yielded no
files (e.g. an unchanged rescan), behind a per-ROM `functools.cache` helper so
the playmatch/hasheous/ss lookups share a single DB fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Renaming a ROM gives its file a new internal id, but the EmulatorJS
player keeps a remembered file id ("disc") in localStorage and reuses
it on the next launch. After a rename that id is stale, so the content
download endpoint matched zero files and fell through to its multi-file
ZIP path, producing a download whose only entry was an empty .m3u
playlist. nginx's mod_zip decode step rejects the blank value (HTTP
400) and aborts the response, sending 0 bytes — which EmulatorJS
surfaces as a generic "network error" (issue #3470).
The frontend half (validating the remembered disc against the ROM's
current files) already landed on master in d1696cd04. This is the
backend half: when no files match the request, raise a clean 404
instead of building a broken empty-.m3u ZIP. This also covers a ROM
with zero files.
Add endpoint tests (auth, single-file, valid file id, stale file id
-> 404, missing rom -> 404) plus a `rom_file` fixture.
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The update_rom path already re-parses filename tags when fs_name
changes (master commit d7a896b5da), but the headline scenario from
issue #3471 — an untagged ROM renamed to add (Europe) so the region
flag appears — was never asserted; existing coverage only exercised
the tag-removal direction.
Add a test that renames the untagged rom fixture to "test_rom
(Europe).zip" and asserts regions == ["Europe"], locking down the
add-region direction described in the issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The five pkgj feed tests created a ROM but no RomFile, so the per-file
feeds emitted only a header and the "in response.text" assertions never
actually verified output (pre-existing failures, also red on master).
Add a top-level `.pkg` GAME file (games feeds) or a DLC-category file
(dlc feeds), mirroring the pkgi_ps3 test, so the feeds produce rows.
This also gives real coverage of the new `include_files=True` path that
these feeds rely on.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
filter_roms feeds both the gallery/list endpoint (SimpleRomSchema, no
files) and the feed endpoints (which iterate rom.files / is_top_level).
The cleanup commit's unconditional selectinload(Rom.files) + joinedload
made the gallery/list and filter-value paths pay for files they never
serialize.
Gate the files load behind a new `include_files` flag (default False),
mirroring the existing `include_file_stats` opt-in, and plumb it through
get_roms_scalar. The 9 feed endpoints that actually read rom.files opt
in; the gallery/list, filter-values, identifiers, smart-collection, and
the three feeds that don't touch files (webrcade, fpkgi, kekatsu) skip
the load entirely — keeping the gallery query at zero file cost.
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
The SS metadata handler pre-encoded the name-search term with quote()
before handing it to the service layer, which percent-encodes the query
again via yarl's with_query(). This double-encoded any character that
needs URL-encoding (e.g. "+" -> "%2B" -> "%252B"), so the request URL
carried a doubly-escaped term.
Pass the raw (unidecode-transliterated but un-percent-encoded) term to
search_games() in both _search_rom() and get_matched_roms_by_name() and
let the URL builder encode it exactly once. The scan now sends e.g.
recherche=...%2B... instead of ...%252B...
This is a request-correctness fix. It does not, on its own, make every
previously-unmatched title match: ScreenScraper's jeuRecherche normalizes
punctuation and applies its own relevance ranking, so some titles still
return no results for the full filename-derived term (verified directly
against the API). Improving name-search robustness is a separate concern.
Add TestSearchTermEncoding regression tests covering the un-pre-encoded
term, preserved unidecode transliteration, and a single-encoded request
URL (%2B, never %252B).
Written primarily by Claude Code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restore the "platform only" contract of `get_roms_by_fs_name` (per its
docstring) by dropping the `selectinload(Rom.files)` + `joinedload`. That
load only existed for `scan_rom`'s rare `fs_rom["files"] or rom.files`
fallback, but it forced files (and a per-file join back to roms) for every
ROM in a scan batch — expensive on large platforms, and only used when the
filesystem scan yielded no files.
Instead, fetch the persisted files on demand: `scan_rom` now resolves match
files via a small helper that returns the filesystem-scanned files, falling
back to `db_rom_handler.get_rom_files_by_rom_id(rom.id)` only when there are
none. The new getter eager-loads the `RomFile.rom` backref so `is_top_level`
keeps working on the detached results (the rare path was already latently
broken on master, which loaded files without the backref).
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
