Drop the migration and the multi_file / top_level_file_count columns on
roms; express both as deferred column_property correlated subqueries
against rom_files instead. The gallery list and detail queries opt in
via undefer, so they get the values computed in the same SELECT via
indexed subqueries (rom_id index already in place); other code paths
that don't read the flags pay nothing.
This keeps the gallery perf win (no rom_files load for cards) without
introducing schema state that has to stay in sync with rom_files at
write time.
ASGI spec only allows headers on the http.response.start message;
appending Set-Cookie to body messages is out-of-spec and may break on
some servers. Early-return for non-start messages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The gallery list endpoint was eager-loading every rom_file row for each
paginated ROM via selectinload, then re-joining each row back to its
parent rom for the is_top_level computation. For platforms with extracted
multi-file ROMs (Xbox 360 ~1394 files/ROM, Switch ~199 files/ROM), this
made /api/roms time out at 120s even with a rom_id index.
Cards never displayed individual files — only the has_simple_single_file
/ has_nested_single_file / has_multiple_files booleans that derive from
the file list. Denormalize the underlying state onto roms as multi_file
(folder-based vs single-file) and top_level_file_count, recompute the
booleans from those columns, drop the selectinload from filter_roms, and
move the files field from SimpleRomSchema to DetailedRomSchema so the
gallery payload no longer ships file rows.
Also drop the redundant joinedload(RomFile.rom) and switch the relation
to lazy="select" so subsequent file.rom accesses resolve from the
session identity map instead of re-JOINing the parent rom per file row.
ShowQRCode.vue's folder-based DS/3DS fallback now fetches the detailed
rom on demand, since SimpleRom no longer carries files.
Apply the same lazy-factory pattern to FSLaunchboxHandler and FSSyncHandler
that ssh_sync_handler now uses. With both opt-in features deferred to
first-use, the tolerate_missing_base escape hatch on FSHandler is no longer
needed — every handler now fails loudly on mkdir failure, which is the
right behavior for the always-on core paths (assets, library, resources).
Touched call sites:
- resources_handler._resolve_local_file_uri (launchbox)
- sync_watcher.py, endpoints/device.py, tasks/manual/sync_folder_scan.py
(fs_sync)
Net effect:
- Default installs never poke /romm/launchbox or /romm/sync at startup.
- Misconfigured opt-in users get a clear, actionable PermissionError at
the call site instead of a silent warning followed by mystery failures.
- tolerate_missing_base, its tests, and one stale log import are gone.
The module-level SSHSyncHandler() singleton ran filesystem side effects
(mkdir on SYNC_SSH_KEYS_PATH) at import time, which meant even default
installs with push-pull sync disabled would touch /romm/sync — and the
previous tolerate-and-warn fallback could leave users wondering why
sync silently does nothing.
Replace the eager singleton with a functools.cache'd factory. The
handler is now constructed on first use, so:
- Default-install users (ENABLE_SYNC_PUSH_PULL=false, no manual sync
triggered) never touch /romm/sync.
- Users opting in get a clear, actionable RuntimeError pointing at
the unwritable path and the env var to override, at the call site
rather than buried in a startup stack trace.
Also document in env.template that enabling either sync mode requires
a writable volume at $ROMM_BASE_PATH/sync.
The SSH sync handler is instantiated at module import time, so any
PermissionError on mkdir would crash the entire app rather than just
disabling the push-pull sync feature. This affected users whose /romm
mount didn't include a writable sync subdirectory (common on Unraid
and similar setups that mount specific subpaths).
Mirror the FSHandler pattern: log a warning and continue. Keys are
expected to be pre-mounted per the module docstring, and
_resolve_key_path already handles a missing directory gracefully.
Fixes#3419
Instead of smuggling an internal control flag through the SSRom dict,
lookup_rom now returns (SSRom, is_not_game: bool). scan_handler unpacks
the tuple and short-circuits the name-search fallback when either an
ss_id matched or the hash lookup flagged the entry as notgame.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit added a notgame skip-name-search check in scan_handler
but used a different key ("notgame") than what lookup_rom returned
("not_game"), so the fallback was never actually skipped. Align both on
SSRom.not_game, pop the internal flag before returning the SSRom to the
rest of the scan pipeline, and rename the helper to _is_not_game for
consistency with the field name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When jeuInfos.php returned a notgame entry (BIOS files, ZZZ hacks, etc.),
lookup_rom() had no notgame check, leading to two bugs:
- notgame entries with a real ID were stored as valid SS matches
- notgame entries with a falsy ID fell through to a jeuRecherche.php name
search that always returned nothing (pointless quota usage)
Adds _is_notgame() and NOTGAME_NAME_PREFIX, returns SSRom(notgame=True)
from lookup_rom() on a notgame hit, and guards the get_rom() fallback in
scan_handler so the name search is skipped entirely. Also adds the missing
notgame filter to _search_rom() so ZZZ entries can't match by name either.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SS media URLs are stripped of ssid/sspassword before DB storage (correct),
but downloads were issued against the credential-less stored URLs, causing
them to count against the anonymous IP quota instead of the user's account.
Adds restore_sensitive_query_params() as the principled complement to
strip_sensitive_query_params(), and add_ss_auth_to_url() in ss_handler
which re-attaches credentials at download time without storing them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A COMPLETE rescan never deleted previously downloaded asset files, and
the post-scan download steps skip work when a file already exists
(store_media_file) or when the source URL is unchanged (get_cover). This
meant covers, screenshots, manuals, and SS/gamelist/LaunchBox extended
media were reused even when a fresh fetch returned a different URL —
defeating the region-priority fix in #3396.
- scan socket: for a COMPLETE rescan of an existing ROM, remove the
cover, manual, screenshots, and all extended media directories before
re-fetching, so the download steps pull fresh files.
- scan_handler: reset url_cover/url_screenshots/url_manual and the
matching path_* fields for COMPLETE rescans before the priority loops
run, so stale DB values are nulled when no selected source supplies
them (clearing for deselected sources falls out as a subset).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a user manually sets metadata IDs (e.g., ra_id, launchbox_id, hasheous_id)
via the UI and then runs "Refresh Metadata" (which defaults to UNMATCHED scan type),
no metadata was fetched because UNMATCHED conditions checked `not rom.xxx_id` — so
if the ID was already set, the handler was skipped entirely.
Fix: Change each handler's UNMATCHED condition to also trigger when the ID is set
but the corresponding metadata dict is empty (i.e., `not rom.xxx_id or not rom.xxx_metadata`).
For handlers that support ID-based lookup (RA, Launchbox, IGDB, MobyGames, SS,
Flashpoint), also add the `get_rom_by_id` path inside the function.
For Hasheous: when hash lookup fails but `hasheous_id` is set on an existing ROM
(not newly added), return a partial HasheousRom built from the existing sub-IDs
(igdb_id, ra_id, tgdb_id) so the downstream get_igdb_game / get_ra_game proxy
calls can still enrich the ROM.
Add three targeted tests to validate:
- UNMATCHED scan fetches RA metadata when ra_id is set but ra_metadata is empty
- UNMATCHED scan skips RA when both ra_id and ra_metadata are already populated
- UNMATCHED scan passes existing sub-IDs to Hasheous proxies when hash lookup fails
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/098b482f-9f73-4f35-819a-b55004a79b13
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
- Add `STANDALONE_EXPANSION` game type to the `with_game_type` filter in
`_search_rom` so games like "Ecco: The Tides of Time" (which IGDB classifies
as a standalone expansion) are included in the first search pass and are not
confused with their parent game ("Ecco The Dolphin")
- Fix the expanded search fallback to fetch and compare ALL unique game IDs
returned by the IGDB search endpoint, instead of only the first result
- Add tests to verify both fixes
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/d6a0c1dd-e541-4d8e-a272-9e5511a2077e
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
For ROMs tagged with multiple regions (e.g. "(Japan, USA)"), filename order
previously decided which region's name and box art won. Now reorder the rom's
filename-tagged regions by SCAN_REGION_PRIORITY before prepending, so the
user's configured preference wins among the regions the file is actually
tagged as. Untagged priority regions still cannot outrank a filename-tagged
region.
Also tweak the Total Rescan → Complete Rescan label in en_GB/en_US scan
locales.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
