For ROMs tagged with multiple regions (e.g. "(Japan, USA)"), filename order
previously decided which region's name and box art won. Now reorder the rom's
filename-tagged regions by SCAN_REGION_PRIORITY before prepending, so the
user's configured preference wins among the regions the file is actually
tagged as. Untagged priority regions still cannot outrank a filename-tagged
region.
Also tweak the Total Rescan → Complete Rescan label in en_GB/en_US scan
locales.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix several issues in ScreenScraper API request/response handling:
- Correctly handle SS-specific HTTP error codes (KO responses, 429, 431,
and the SS-quirk of returning 401 when server CPU >60%).
- Construct requests with proper parameter encoding so jeuInfos lookups
and search queries return the expected results.
- Store media URLs returned by SS as-is, preserving the dev credential
query parameters required for media playback. Removing them broke
downstream media fetches.
To keep dev credentials out of log output, add a redacting formatter in
the logger pipeline that scrubs ssid/sspassword/devid/devpassword query
parameters from any URL it sees.
Test coverage added for the new HTTP error paths and the as-is URL
storage behaviour.
Importer (gamelist/launchbox file:// flows) and exporters (gamelist.xml,
metadata.pegasus.txt local exports) now hardlink media assets when source
and destination share a filesystem, falling back transparently to a copy
on EXDEV / EPERM / EOPNOTSUPP / EMLINK / EACCES (cross-device, FAT32,
exFAT, network mounts, etc.). Saves disk space and is effectively
instantaneous on large files (videos, manuals, miximages).
Covers keep a real copy (allow_link=False) because _store_cover resizes
the small cover in place via PIL.Image.save, which would truncate the
shared inode and corrupt the user's source image.
Also makes FSSyncHandler tolerate a missing/unwritable /romm/sync at
startup: an OSError from mkdir now logs a warning instead of crashing
the whole app at module-import time. Sync calls still fail at use time
if the mount remains broken — the right place to surface the error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CHD files now follow the same hash logic as all other file types — CRC32,
MD5, and SHA1 are computed from raw container bytes. This allows
ScreenScraper to log KO entries for unrecognised CHD files, which it
could not do when only the disc-data SHA1 was being computed.
The CHD header SHA1 (disc-data SHA1) is separately extracted and stored
in a new chd_sha1_hash field on RomFile, with a migration adding the
column to rom_files. Hasheous receives only this disc-data SHA1 (no
CRC/MD5) since it indexes disc-based games by disc-data SHA1, not raw
file hashes.
The RAHasher multi-file path now passes the largest CHD directly instead
of a /* wildcard, which RAHasher cannot expand. Hash computations are
wrapped in asyncio.to_thread to avoid blocking the event loop during
large reads.
Hash-lookup metadata handlers (ScreenScraper, Hasheous, Playmatch) now
fall back to rom.files (stored DB hashes) when fs_rom files are not
rehashed, fixing hash-based matching for UNMATCHED and UPDATE scan types.
The Disc SHA-1 is displayed in the ROM detail view for both single-file
(FileInfo.vue) and multi-file (FileSelectItem.vue) CHD games.
When a ROM filename carries a region tag (e.g. (Europe)), use that
region first when picking artwork and localized titles, falling back to
the configured scan.priority.region. Previously the configured priority
was the only signal, so a US-first config would force US covers onto
European ROMs even when an EU asset was available.
Adds a shared name->provider-shortcode map and threads the rom through
the IGDB and SS lookup APIs so the rom-aware locale/region selection
can run for both providers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UPDATE scans with a known launchbox_id were calling get_rom_by_id(),
which only returns remote data and bypassed get_rom()'s local-first
merge — so local-only fields like Notes were getting clobbered and
local media matching fidelity dropped. get_rom_by_id() now optionally
takes fs_name/platform_slug and merges the local entry when its
DatabaseID matches the requested id.
Also fixes populate_rom_specific_paths writing every video as
video.mp4 regardless of source extension; since store_media_file is a
byte copy, .mkv/.webm contents would be served as .mp4 and break MIME
sniffing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the get_platform_fs_structure, get_firmware_fs_structure,
get_platforms_directory and get_roms_fs_structure tests from mocking
os.path.exists to mocking glob.glob, matching the new STRUCTURE_PATH_B
detection logic. Rename the existing high_priority/normal_structure
variants to structure_a/structure_b for clarity, and fix
test_platform_specific_behavior so its monkeypatch wraps the calls
instead of being applied after them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sample_metadata.zip test fixture used invented tag names
(<Description>, FileName ending in .zip). Real LaunchBox Mame.xml
(see backend/tasks/fixtures/launchbox/mame.xml) uses <Name> as the
full title and keys MameFile entries by the stem (e.g. wrlok_l3),
no extension. Read Name instead of Description, and fall back to
the stem when the raw fs_name lookup misses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
remote_media_req built a MediaRequest with platform_name=None, which
made _build_local_media_context bail and never search on-disk
Images/Manuals/Videos. Any rom that matched only via Metadata.xml (no
local XML entry) ended up with images: [] even though LaunchBox art
was sitting on disk. Thread platform_name and fs_name into
remote_media_req, falling back to the remote entry's Platform field
when no slug is available (e.g. lookups by database id).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LaunchBox stores videos under /romm/launchbox/Videos/<Platform>/<Game>.<ext>
but the handler only extracted YouTube IDs; LAUNCHBOX_VIDEOS_DIR was
defined and unused. Add _get_video() to surface local videos as
launchbox-file:// URLs, thread a populate_rom_specific_paths() through
the scan pipeline to set video_path once the rom is known, and mirror
the SS/gamelist branch in the scan socket so store_media_file actually
copies the video into the rom's resource directory. Rom.path_video now
also reads from launchbox_metadata.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The scheduled task populated LAUNCHBOX_MAME_KEY from Mame.xml but no
handler code ever read it, so arcade ROMs like pacman.zip never matched
against Metadata.xml (which keys on full titles like "Pac-Man"). Add
RemoteSource.get_mame_entry() and an ARCADE branch in get_rom() that
resolves the MAME filename to its Description before name lookup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
get_rom now also fetches Named_Snaps, Named_Titles, and Named_Logos
when the matching MetadataMediaType (SCREENSHOT, TITLE_SCREEN, LOGO)
is in SCAN_MEDIA. Box art is still fetched unconditionally — it drives
url_cover and libretro_id. Matching extras are appended to
url_screenshots so the scan_handler artwork priority loop picks them
up without further changes. All enabled listings are fetched
concurrently.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SGDB's url_cover used to clobber whatever the artwork priority loop
picked, which meant user-set priorities (e.g. libretro > sgdb) and
manually uploaded covers were silently overridden. SGDB now only
replaces the current url_cover when it outranks every other source
that produced one under SCAN_ARTWORK_PRIORITY, and never over a
manual cover preserved on UPDATE/UNMATCHED scans. Default artwork
priority gains sgdb at the top so existing "SGDB wins" behavior is
preserved for default configs.
On the /search/roms endpoint, libretro is now an enrichment source
alongside SGDB instead of a primary match source: it decorates
entries resolved by IGDB/Moby/SS/Flashpoint/Launchbox with
libretro_id and libretro_url_cover, mirroring how SGDB works.
get_matched_roms_by_name is removed from the libretro handler since
nothing else calls it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collapse the two parallel id lists and their mirrored chunked-update
loops into a `flips: dict[bool, list[int]]` keyed by desired state, and
drop unused rom assignments in the related tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds the libretro thumbnail repository as a first-class artwork source so
region-correct box art (PAL/Europe, Japan, etc.) can be matched directly
to ROM filenames, addressing rommapp/romm#3239.
Implementation follows the SGDB handler pattern (artwork-only, no game
metadata): MetadataSource enum entry, scan-time fetch wired into the
SCAN_ARTWORK_PRIORITY loop, /search/roms integration, MatchRom dialog
chip + cover selection, and a heartbeat flag.
Matching is exact case-insensitive against the directory listing first
(so a ROM named "(Europe)" lands on the (Europe) artwork), with a
JaroWinkler fuzzy fallback at 0.8 that strips parenthetical tags from
both sides. Listings are cached in Redis with a 24h TTL.
`libretro_id` is persisted on the Rom model as the SHA1 hex of the
matched libretro filename — stable across scans, distinct per region,
indexed for lookup. Migration 0077 adds the column.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The scan was spending excessive time on large platforms even when all ROMs
were already scanned. Root causes: per-ROM UPDATE queries for skipped ROMs
(10k individual writes), missing composite index on (platform_id, fs_name)
causing full table scans, NOT IN clauses with 10k+ values in
mark_missing_roms(), and redundant filesystem reads.
Changes:
- Add bulk_mark_present() for batch-updating skipped ROMs in one query
- Move skip detection from _identify_rom to the batch loop so skipped ROMs
never enter the async scan pipeline, and report progress for them
- Add composite index idx_roms_platform_id_fs_name via migration 0077
- Rewrite mark_missing_roms() with flip-based approach: mark all missing,
then un-mark present ones in chunks of 1000
- Cache filesystem reads in scan_platforms() to avoid double directory
traversal (precounting + scanning)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
