A ROM's name defaults to its raw filename (extension included) when it is
first discovered. On UNMATCHED and UPDATE rescans, the block that preserves
existing base fields kept that placeholder name even after a metadata
provider finally matched the ROM, leaving the filename (with its ".zip"
extension and region tags) as the title.
Treat a name equal to the ROM's filename as "no name" so a freshly matched
provider name replaces it, while still preserving user-edited names.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Shf4omKSP6dHC3dC2Jjg7b
Drop the name_sort_key_custom flag/migration in favour of a flagless rule: a
key is "custom" when it no longer equals compute(name). Apply that consistently
across all three write paths so a manual sort key survives renames while a
derived key keeps following the name:
- @validates re-derives on name assignment only when the stored key still
matches the derived value; direct name_sort_key assignment stores a
normalized custom key (or reverts to derived when cleared). Handles both
kwarg orders at construction.
- update_rom mirrors the same check for the bulk update() path it bypasses.
- The edit endpoint only writes the key when the user actually changed the
field, delegating the untouched case to update_rom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the separate `sort_name` column into `name_sort_key`, which is now
the single user-settable sort field: always normalized and indexed for fast
ordering, derived from `name` by default, and overridable. A new
`name_sort_key_custom` boolean marks user/metadata overrides so they survive
renames and rescans.
- Drop the `roms.sort_name` column; repurpose migration 0085 to add
`name_sort_key_custom`.
- Derive the key via `@validates("name")` unless pinned custom; the edit
dialog, unmatch flow, and ES-DE gamelist <sortname> set custom keys.
- update_rom / scan_rom keep the columns in sync explicitly (bulk update and
construction bypass / reorder the validator).
- Frontend: edit field drives name_sort_key (empty when auto), api sends the
override only when custom, regenerated types updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralize the *_no_tags / *_no_ext / *_extension columns (derived from a
file name) behind @validates hooks instead of computing them by hand at
every write site:
- Add pure helpers (compute_file_name_parts and friends) to models.base;
the filesystem base handler now delegates to them.
- Add @validates on Rom (fs_name), BaseAsset (file_name, inherited by all
asset subclasses), and Firmware (file_name).
- update_rom keeps the fs_name-derived columns in sync on bulk update(),
which also fixes the rename path never updating fs_extension.
- Drop the now-redundant computations at the scan/rename call sites.
Also fix the migration backfill loop and a pre-existing list[str | None]
type mismatch surfaced in scan_handler. Add tests for the helpers, the
validators, and the update_rom bulk-sync path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopt master's ROM schema design (sibling_roms + files, batched
get_files_for_roms / get_siblings_for_roms) while preserving the v2-branch
features master lacks: per-user is_main_sibling on siblings and audio_meta
on rom files.
Conflict resolution:
- responses/rom.py: keep master's sibling_roms/files fields; re-graft
is_main_sibling via SiblingRomSchema.from_rom(rom, is_main_sibling=...);
restore the eager-relationship fallback in
SimpleRomSchema.from_orm_with_request (None sentinel) so the v2
/{id}/simple endpoint still returns siblings/files.
- roms_handler.py: get_siblings_for_roms now left-joins RomUser and returns
(Rom, is_main_sibling) tuples; keep both branch and master file helpers.
- drop the redundant branch-only sibling_ids field and
get_sibling_data_for_roms.
- generated types resolved to match (sibling_roms + files; RomFileSchema
keeps audio_meta and gains archive_members).
- update v2 components and the RelatedGameCard mock to read sibling_roms.
- fix stale exclude={"siblings"} -> "sibling_roms" in scan emit payloads.
- re-chain the audio_meta migration as 0083 (after master's 0082) to keep a
single Alembic head.
- package.json: union of branch tooling + master dependency bumps; lock
regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scan loaders no longer eager-load `Rom.files` (#3425 + follow-ups), so
the hash-based metadata lookups can't rely on `rom.files` being populated —
`hasheous`/`ss` `lookup_rom` read `RomFile.is_top_level`, which dereferences
`RomFile.rom.full_path` and would raise `DetachedInstanceError` once the
session closed.
Add `DBRomsHandler.rom_files_for_rom_id`, which loads a ROM's files on demand
with the `RomFile.rom` backref eager-loaded (`load_only(fs_path, fs_name)`).
The scan path uses it as a fallback only when the filesystem walk yielded no
files (e.g. an unchanged rescan), behind a per-ROM `functools.cache` helper so
the playmatch/hasheous/ss lookups share a single DB fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore the "platform only" contract of `get_roms_by_fs_name` (per its
docstring) by dropping the `selectinload(Rom.files)` + `joinedload`. That
load only existed for `scan_rom`'s rare `fs_rom["files"] or rom.files`
fallback, but it forced files (and a per-file join back to roms) for every
ROM in a scan batch — expensive on large platforms, and only used when the
filesystem scan yielded no files.
Instead, fetch the persisted files on demand: `scan_rom` now resolves match
files via a small helper that returns the filesystem-scanned files, falling
back to `db_rom_handler.get_rom_files_by_rom_id(rom.id)` only when there are
none. The new getter eager-loads the `RomFile.rom` backref so `is_top_level`
keeps working on the detached results (the rare path was already latently
broken on master, which loaded files without the backref).
https://claude.ai/code/session_01PSXKmejPRzdxLFMN6P2QQ4
Standard media fields (url_cover, url_manual, url_screenshots) were downloaded
using the stored credential-less URLs, causing them to count against the anonymous
IP quota instead of the user's SS account. Apply add_ss_auth_to_url() at each
download call site in the scan and ROM update paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(screenscraper): guard add_ss_auth_to_url against non-SS URLs
Only inject ssid/sspassword into screenscraper.fr URLs to prevent
leaking user credentials to third-party sources (IGDB, LaunchBox, etc.)
when url_cover/url_manual/url_screenshots originate from other providers.
Add tests for the non-SS no-op and empty-string edge cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test(screenscraper): verify SS credentials injected for all media download paths
- TestAddSsAuthToUrl: add guards for non-SS URLs (IGDB, LaunchBox) and
empty string inputs
- test_update_rom: verify ssid/sspassword appear in url_cover and
url_manual args passed to get_cover/get_manual for screenscraper.fr
URLs; verify IGDB URLs are NOT decorated with SS credentials
- TestScanCredentialInjection: verify the scan-path ternary pattern
correctly applies add_ss_auth_to_url to cover and screenshot URLs,
and that a None cover URL passes through without error
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test(screenscraper): empirical audit — every SS request carries ssid/sspassword
Intercepts both HTTP clients at the transport/session level to verify
that every outgoing screenscraper.fr request is decorated with the user's
ssid and sspassword credentials:
aiohttp (API calls via auth_middleware):
- jeuInfos.php, jeuRecherche.php, ssinfraInfos.php, ssuserInfos.php
httpx (media downloads via FSResourcesHandler):
- get_cover → url_cover
- get_manual → url_manual
- get_rom_screenshots → url_screenshots (each URL)
- store_media_file → extra media (fanart, bezel, etc.)
Also verifies the domain guard: IGDB URLs passed through add_ss_auth_to_url
are NOT decorated with SS credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of smuggling an internal control flag through the SSRom dict,
lookup_rom now returns (SSRom, is_not_game: bool). scan_handler unpacks
the tuple and short-circuits the name-search fallback when either an
ss_id matched or the hash lookup flagged the entry as notgame.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit added a notgame skip-name-search check in scan_handler
but used a different key ("notgame") than what lookup_rom returned
("not_game"), so the fallback was never actually skipped. Align both on
SSRom.not_game, pop the internal flag before returning the SSRom to the
rest of the scan pipeline, and rename the helper to _is_not_game for
consistency with the field name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When jeuInfos.php returned a notgame entry (BIOS files, ZZZ hacks, etc.),
lookup_rom() had no notgame check, leading to two bugs:
- notgame entries with a real ID were stored as valid SS matches
- notgame entries with a falsy ID fell through to a jeuRecherche.php name
search that always returned nothing (pointless quota usage)
Adds _is_notgame() and NOTGAME_NAME_PREFIX, returns SSRom(notgame=True)
from lookup_rom() on a notgame hit, and guards the get_rom() fallback in
scan_handler so the name search is skipped entirely. Also adds the missing
notgame filter to _search_rom() so ZZZ entries can't match by name either.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A COMPLETE rescan never deleted previously downloaded asset files, and
the post-scan download steps skip work when a file already exists
(store_media_file) or when the source URL is unchanged (get_cover). This
meant covers, screenshots, manuals, and SS/gamelist/LaunchBox extended
media were reused even when a fresh fetch returned a different URL —
defeating the region-priority fix in #3396.
- scan socket: for a COMPLETE rescan of an existing ROM, remove the
cover, manual, screenshots, and all extended media directories before
re-fetching, so the download steps pull fresh files.
- scan_handler: reset url_cover/url_screenshots/url_manual and the
matching path_* fields for COMPLETE rescans before the priority loops
run, so stale DB values are nulled when no selected source supplies
them (clearing for deselected sources falls out as a subset).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a user manually sets metadata IDs (e.g., ra_id, launchbox_id, hasheous_id)
via the UI and then runs "Refresh Metadata" (which defaults to UNMATCHED scan type),
no metadata was fetched because UNMATCHED conditions checked `not rom.xxx_id` — so
if the ID was already set, the handler was skipped entirely.
Fix: Change each handler's UNMATCHED condition to also trigger when the ID is set
but the corresponding metadata dict is empty (i.e., `not rom.xxx_id or not rom.xxx_metadata`).
For handlers that support ID-based lookup (RA, Launchbox, IGDB, MobyGames, SS,
Flashpoint), also add the `get_rom_by_id` path inside the function.
For Hasheous: when hash lookup fails but `hasheous_id` is set on an existing ROM
(not newly added), return a partial HasheousRom built from the existing sub-IDs
(igdb_id, ra_id, tgdb_id) so the downstream get_igdb_game / get_ra_game proxy
calls can still enrich the ROM.
Add three targeted tests to validate:
- UNMATCHED scan fetches RA metadata when ra_id is set but ra_metadata is empty
- UNMATCHED scan skips RA when both ra_id and ra_metadata are already populated
- UNMATCHED scan passes existing sub-IDs to Hasheous proxies when hash lookup fails
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/098b482f-9f73-4f35-819a-b55004a79b13
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
CHD files now follow the same hash logic as all other file types — CRC32,
MD5, and SHA1 are computed from raw container bytes. This allows
ScreenScraper to log KO entries for unrecognised CHD files, which it
could not do when only the disc-data SHA1 was being computed.
The CHD header SHA1 (disc-data SHA1) is separately extracted and stored
in a new chd_sha1_hash field on RomFile, with a migration adding the
column to rom_files. Hasheous receives only this disc-data SHA1 (no
CRC/MD5) since it indexes disc-based games by disc-data SHA1, not raw
file hashes.
The RAHasher multi-file path now passes the largest CHD directly instead
of a /* wildcard, which RAHasher cannot expand. Hash computations are
wrapped in asyncio.to_thread to avoid blocking the event loop during
large reads.
Hash-lookup metadata handlers (ScreenScraper, Hasheous, Playmatch) now
fall back to rom.files (stored DB hashes) when fs_rom files are not
rehashed, fixing hash-based matching for UNMATCHED and UPDATE scan types.
The Disc SHA-1 is displayed in the ROM detail view for both single-file
(FileInfo.vue) and multi-file (FileSelectItem.vue) CHD games.
Replace the tuple+derived-dict pair with PLAYMATCH_TAG_TO_ATTR as the
canonical mapping. Rename enum members to UPPER_CASE, expand
PlaymatchRomMatch to cover all provider ids, and inline the fallback
match in place of the _empty_playmatch_rom_match helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a ROM filename carries a region tag (e.g. (Europe)), use that
region first when picking artwork and localized titles, falling back to
the configured scan.priority.region. Previously the configured priority
was the only signal, so a US-first config would force US covers onto
European ROMs even when an EU asset was available.
Adds a shared name->provider-shortcode map and threads the rom through
the IGDB and SS lookup APIs so the rom-aware locale/region selection
can run for both providers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UPDATE scans with a known launchbox_id were calling get_rom_by_id(),
which only returns remote data and bypassed get_rom()'s local-first
merge — so local-only fields like Notes were getting clobbered and
local media matching fidelity dropped. get_rom_by_id() now optionally
takes fs_name/platform_slug and merges the local entry when its
DatabaseID matches the requested id.
Also fixes populate_rom_specific_paths writing every video as
video.mp4 regardless of source extension; since store_media_file is a
byte copy, .mkv/.webm contents would be served as .mp4 and break MIME
sniffing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sample_metadata.zip test fixture used invented tag names
(<Description>, FileName ending in .zip). Real LaunchBox Mame.xml
(see backend/tasks/fixtures/launchbox/mame.xml) uses <Name> as the
full title and keys MameFile entries by the stem (e.g. wrlok_l3),
no extension. Read Name instead of Description, and fall back to
the stem when the raw fs_name lookup misses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LaunchBox stores videos under /romm/launchbox/Videos/<Platform>/<Game>.<ext>
but the handler only extracted YouTube IDs; LAUNCHBOX_VIDEOS_DIR was
defined and unused. Add _get_video() to surface local videos as
launchbox-file:// URLs, thread a populate_rom_specific_paths() through
the scan pipeline to set video_path once the rom is known, and mirror
the SS/gamelist branch in the scan socket so store_media_file actually
copies the video into the rom's resource directory. Rom.path_video now
also reads from launchbox_metadata.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SGDB's url_cover used to clobber whatever the artwork priority loop
picked, which meant user-set priorities (e.g. libretro > sgdb) and
manually uploaded covers were silently overridden. SGDB now only
replaces the current url_cover when it outranks every other source
that produced one under SCAN_ARTWORK_PRIORITY, and never over a
manual cover preserved on UPDATE/UNMATCHED scans. Default artwork
priority gains sgdb at the top so existing "SGDB wins" behavior is
preserved for default configs.
On the /search/roms endpoint, libretro is now an enrichment source
alongside SGDB instead of a primary match source: it decorates
entries resolved by IGDB/Moby/SS/Flashpoint/Launchbox with
libretro_id and libretro_url_cover, mirroring how SGDB works.
get_matched_roms_by_name is removed from the libretro handler since
nothing else calls it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds the libretro thumbnail repository as a first-class artwork source so
region-correct box art (PAL/Europe, Japan, etc.) can be matched directly
to ROM filenames, addressing rommapp/romm#3239.
Implementation follows the SGDB handler pattern (artwork-only, no game
metadata): MetadataSource enum entry, scan-time fetch wired into the
SCAN_ARTWORK_PRIORITY loop, /search/roms integration, MatchRom dialog
chip + cover selection, and a heartbeat flag.
Matching is exact case-insensitive against the directory listing first
(so a ROM named "(Europe)" lands on the (Europe) artwork), with a
JaroWinkler fuzzy fallback at 0.8 that strips parenthetical tags from
both sides. Listings are cached in Redis with a 24h TTL.
`libretro_id` is persisted on the Rom model as the SHA1 hex of the
matched libretro filename — stable across scans, distinct per region,
indexed for lookup. Migration 0077 adds the column.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
