Commit Graph

1553 Commits

Author SHA1 Message Date
Georges-Antoine Assi
9bafd11993 Merge pull request #3381 from DevYukine/feat/playmatch-multi-provider
feat(playmatch): add SteamGridDB, ScreenScraper, MobyGames & Launchbox hash support
2026-05-16 15:53:50 -04:00
Georges-Antoine Assi
c3c6829962 refactor(playmatch): use dict as single source of truth for provider tags
Replace the tuple+derived-dict pair with PLAYMATCH_TAG_TO_ATTR as the
canonical mapping. Rename enum members to UPPER_CASE, expand
PlaymatchRomMatch to cover all provider ids, and inline the fallback
match in place of the _empty_playmatch_rom_match helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 15:41:33 -04:00
Georges-Antoine Assi
06eac10134 Merge pull request #3368 from rommapp/copilot/fix-ra-hashes-missing-ids
fix: include games without achievements in RA hash cache
2026-05-16 09:13:20 -04:00
Yukine
f307aeaca3 refactor(playmatch): move PLAYMATCH_SUPPORTED_SOURCES into playmatch_handler 2026-05-16 05:41:22 +02:00
Yukine
820595a160 refactor(playmatch): simplify comments 2026-05-16 05:21:22 +02:00
Yukine
563e1f4e76 feat(scan): lift IGDB gate from Playmatch and forward provider ids to fetch funcs 2026-05-16 05:17:37 +02:00
Yukine
d9427573dc feat(playmatch): align with upstream MetadataProvider enum and return all provider ids 2026-05-16 05:17:17 +02:00
Georges-Antoine Assi
1b164ee81b Merge pull request #3380 from EntropyEngineer/feature-gamelist-comma-separated-attributes
Add comma-separated metadata attribute support to gamelist
2026-05-15 21:15:08 -04:00
Georges-Antoine Assi
e7045446a8 run fmt 2026-05-15 21:01:33 -04:00
Entropy Engineer
4999214a92 Add comma-separated metadata attribute support to gamelist
Add comma-separated metadata attribute support to gamelist
2026-05-16 00:41:09 +05:00
Georges-Antoine Assi
3db3dd4434 run fmt 2026-05-14 15:41:32 -04:00
copilot-swe-agent[bot]
2c3335e8cf refactor: version RA hash cache and switch to O(1) hash-index format
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/f7097cd9-6bf4-42df-a14d-ff23e423291f

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-13 18:01:20 +00:00
Georges-Antoine Assi
c6a2f56fad Merge pull request #3367 from rommapp/regional-provider-tags
Prefer ROM's own region tag for ScreenScraper and IGDB artwork
2026-05-13 11:19:53 -04:00
copilot-swe-agent[bot]
157f719297 fix: include games without achievements in ra_hashes.json
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/d113d268-38fd-4564-892d-959fd5fddc68

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-13 15:17:47 +00:00
Georges-Antoine Assi
dad1250e15 case-insensitive region lookup for provider shortcode mapping
Rom.regions can contain raw filename text like "europe" or "EUROPE"
(filename parsing in roms_handler doesn't normalize casing), so the
direct dict lookup missed those tags and the locale silently fell back
to scan.priority.region. Replace the dict access with a helper that
lowercases both sides.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:34:10 -04:00
Georges-Antoine Assi
944514acc0 prefer rom's own region tag for ScreenScraper and IGDB artwork
When a ROM filename carries a region tag (e.g. (Europe)), use that
region first when picking artwork and localized titles, falling back to
the configured scan.priority.region. Previously the configured priority
was the only signal, so a US-first config would force US covers onto
European ROMs even when an EU asset was available.

Adds a shared name->provider-shortcode map and threads the rom through
the IGDB and SS lookup APIs so the rom-aware locale/region selection
can run for both providers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:06:11 -04:00
Georges-Antoine Assi
8999b66574 Preserve local LaunchBox data on UPDATE scans and video extensions
UPDATE scans with a known launchbox_id were calling get_rom_by_id(),
which only returns remote data and bypassed get_rom()'s local-first
merge — so local-only fields like Notes were getting clobbered and
local media matching fidelity dropped. get_rom_by_id() now optionally
takes fs_name/platform_slug and merges the local entry when its
DatabaseID matches the requested id.

Also fixes populate_rom_specific_paths writing every video as
video.mp4 regardless of source extension; since store_media_file is a
byte copy, .mkv/.webm contents would be served as .mp4 and break MIME
sniffing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:32:34 -04:00
Georges-Antoine Assi
d8ef6f0c05 Merge branch 'master' into local-lb-fix 2026-05-09 13:20:31 -04:00
Georges-Antoine Assi
547c64c4f5 update hltb api 2026-05-09 12:56:01 -04:00
Georges-Antoine Assi
e3aaa106a2 perf(backend): reuse libmagic instance for image upload validation
magic.Magic(mime=True) loads the magic database from disk on construction;
instantiating it per request was adding pointless overhead to every avatar
and artwork upload. Share a module-level instance guarded by a lock (the
underlying magic_t handle is not thread-safe), and surface MagicException
as a 400 so a sniffing failure fails closed instead of bubbling a 500.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 10:14:38 -04:00
Georges-Antoine Assi
53f14f5710 fix(backend): validate uploaded images with libmagic before storing
Avatar, ROM artwork, and collection artwork uploads now sniff the file
header with libmagic and reject anything that isn't PNG/JPEG/WebP/GIF,
saving the file with an extension derived from the detected MIME rather
than the user-supplied filename. Pairs with the raw asset endpoint,
which decides inline vs attachment from the on-disk extension.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:18:02 -04:00
Georges-Antoine Assi
5e3a2707b0 cleanup 2026-05-03 19:39:19 -04:00
copilot-swe-agent[bot]
da005cf81a Optimize fnmatch check and use consistent n64 filename in test
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8cbbc2ca-a3e3-4c61-9e47-f8544d59231a

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 23:36:23 +00:00
copilot-swe-agent[bot]
9593c30292 Address PR review: normalize exclusion sets, avoid duplicates, add multi-dot test for get_rom_files
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8cbbc2ca-a3e3-4c61-9e47-f8544d59231a

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 23:34:30 +00:00
copilot-swe-agent[bot]
101629628e Simplify extension exclusion to use ends-with check instead of sub-extension iteration
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/a81b2023-a243-4721-bc5e-c6fa1a473a79

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 22:46:21 +00:00
copilot-swe-agent[bot]
55cd0cfc4f Support compound suffix exclusions like "hash.txt" for multi-dot filenames
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/d1c69638-bfa0-480e-8050-d565b234ea44

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 01:29:04 +00:00
copilot-swe-agent[bot]
21de7e21f8 Fix file exclusion for multi-dot filenames (e.g. game.nds.hash.txt)
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/2f711770-100b-4e9e-a66e-ab1a74f025f8

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-02 16:50:30 +00:00
Georges-Antoine Assi
07af3d0f64 run fmt 2026-04-30 14:48:20 -04:00
Georges-Antoine Assi
962a9bfa7e one more 2026-04-30 14:39:52 -04:00
Georges-Antoine Assi
fc8d69dc0c update tests 2026-04-30 14:18:49 -04:00
Georges-Antoine Assi
96c3634b80 refactor: split HIGH_PRIO_STRUCTURE_PATH into STRUCTURE_PATH_A/B
Replace the single HIGH_PRIO_STRUCTURE_PATH config attribute with two
glob patterns (STRUCTURE_PATH_A = roms/*, STRUCTURE_PATH_B = */roms) and
update all call sites to detect Structure B via glob.glob, defaulting to
Structure A when no match is found.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 12:52:22 -04:00
Yukine
1591e0971a fix(playmatch): only fire suggestion on manual mapping updates 2026-04-21 11:07:26 +02:00
Yukine
5ecdc8fd13 fix(playmatch): filter falsy provider ids and surface non-2xx responses 2026-04-21 10:56:48 +02:00
Yukine
fee40dc355 feat(playmatch): send a suggestion to playmatch on manual match 2026-04-21 10:54:41 +02:00
Georges-Antoine Assi
91d6928281 Lift LaunchBox video extensions to module constant
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 14:40:36 -04:00
Georges-Antoine Assi
fad2e5c77d Use correct LaunchBox MameFile schema
The sample_metadata.zip test fixture used invented tag names
(<Description>, FileName ending in .zip). Real LaunchBox Mame.xml
(see backend/tasks/fixtures/launchbox/mame.xml) uses <Name> as the
full title and keys MameFile entries by the stem (e.g. wrlok_l3),
no extension. Read Name instead of Description, and fall back to
the stem when the raw fs_name lookup misses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 14:31:16 -04:00
Georges-Antoine Assi
f33844be40 Find local LaunchBox media for remote-only rom matches
remote_media_req built a MediaRequest with platform_name=None, which
made _build_local_media_context bail and never search on-disk
Images/Manuals/Videos. Any rom that matched only via Metadata.xml (no
local XML entry) ended up with images: [] even though LaunchBox art
was sitting on disk. Thread platform_name and fs_name into
remote_media_req, falling back to the remote entry's Platform field
when no slug is available (e.g. lookups by database id).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:32:54 -04:00
Georges-Antoine Assi
8fc426b35c Import local LaunchBox videos
LaunchBox stores videos under /romm/launchbox/Videos/<Platform>/<Game>.<ext>
but the handler only extracted YouTube IDs; LAUNCHBOX_VIDEOS_DIR was
defined and unused. Add _get_video() to surface local videos as
launchbox-file:// URLs, thread a populate_rom_specific_paths() through
the scan pipeline to set video_path once the rom is known, and mirror
the SS/gamelist branch in the scan socket so store_media_file actually
copies the video into the rom's resource directory. Rom.path_video now
also reads from launchbox_metadata.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:13:42 -04:00
Georges-Antoine Assi
a83f3bdca7 Wire up LaunchBox MAME arcade lookups
The scheduled task populated LAUNCHBOX_MAME_KEY from Mame.xml but no
handler code ever read it, so arcade ROMs like pacman.zip never matched
against Metadata.xml (which keys on full titles like "Pac-Man"). Add
RemoteSource.get_mame_entry() and an ARCADE branch in get_rom() that
resolves the MAME filename to its Description before name lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:00:26 -04:00
Georges-Antoine Assi
40689d7e39 Fix LaunchBox local-media file:// paths resolving under library root
LaunchBox produced file:// URIs relative to /romm/launchbox, but the
resources handler resolved them under /romm/library via fs_rom_handler,
so local images/manuals/screenshots were never found. Switch LaunchBox
to a distinct launchbox-file:// scheme and add FSLaunchboxHandler +
_resolve_local_file_uri to route each scheme to the correct root.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 22:59:00 -04:00
copilot-swe-agent[bot]
bf246f842f fix: add cus region to default ScreenScraper fallback regions
Games where only custom (cus) region artwork exists were not fetched
unless users explicitly added 'cus' to their region priority config.
Adding 'cus' to the default fallback list ensures custom artwork is
always considered as a fallback when no other regional artwork exists.

Agent-Logs-Url: https://github.com/rommapp/romm/sessions/75c3e455-f7dd-4bd6-bec7-0460987e40a0

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-04-16 14:28:51 +00:00
Spinnich
15b485b118 fix(collections): address reviewer feedback on types and concurrency
- Use CollectionSchema instead of ReturnType<typeof collectionsStore.getCollection>
  in AddRoms.vue and RemoveRoms.vue (simpler, per gantoine review)
- Wrap bulk INSERT in a savepoint so a concurrent duplicate-key violation
  is caught via IntegrityError and ignored rather than aborting the transaction
- Only bump Collection.updated_at in remove_roms_from_collection when rows
  were actually deleted (rowcount > 0), matching add_roms_to_collection behavior

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:46:12 -04:00
Spinnich
2ecefa3d3f Fix race condition in collection and favorite rom membership updates
Replace full rom_ids list replacement with atomic POST/DELETE endpoints
that add or remove individual ROMs from a collection. This prevents
concurrent rapid clicks from overwriting each other (last-write-wins).

Also fix missing session.flush() in add_rom_user() and add collection
endpoint tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 15:08:53 -04:00
Georges-Antoine Assi
105ed84e43 feat: show TheGamesDB link on platform drawer when tgdb_id is set
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 19:04:51 -04:00
Georges-Antoine Assi
e8a6e9f01d final fixes 2026-04-12 18:43:24 -04:00
Georges-Antoine Assi
d45afb5dde more fixes 2026-04-12 18:32:15 -04:00
Georges-Antoine Assi
f2df03361e fix: cast spread-dict back to RAUserGameProgression to preserve typing
Pylance infers dict[str, Unknown] from the {**old, "highest_award_kind": ...}
spread, which then fails to assign to list[RAUserGameProgression]. Wrap in
typing.cast so the TypedDict type survives the reassignment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:08:18 -04:00
Georges-Antoine Assi
628d8d8bae refactor: pass RAGamesPlatform dict into calculate_hash, normalize extension
Callers now pass the full platform dict and rom.fs_extension; the service
normalizes the extension (optional leading dot, case-insensitive) before
checking the compressed-archive skip set, so ROMs stored with bare
extensions like "zip" correctly hit the skip path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:05:44 -04:00
Georges-Antoine Assi
8f1b8f41d7 perf: skip RAHasher subprocess for archived disc-platform ROMs
RAHasher was being spawned for every hashable ROM regardless of file
type. When the source file is a zip/7z/tar and the RA platform needs
an on-disk disc image (PSX, PS2, PSP, Saturn, Dreamcast, Sega CD,
3DO, PC-FX, Neo Geo CD, TurboGrafx CD, Atari Jaguar CD, Wii), the
subprocess fails with "Unsupported console for buffer hash: {id}"
after paying full process-spawn overhead per ROM — a serious slowdown
when indexing large zipped collections (e.g. myrient PS2/PSP sets).

calculate_hash now short-circuits those combinations with a debug log
and no subprocess. Raw disc images (.iso, .chd, .cue/.bin) and
archives on cartridge platforms still go through RAHasher as before.

Also centralize COMPRESSED_FILE_EXTENSIONS in utils/filesystem.py so
roms_handler (is_compressed_file / hashing), rahasher (skip logic),
and feeds (PKGi passthrough) share one source of truth. The shared
set adds .rar, which is_compressed_file now recognizes too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:18:14 -04:00
Georges-Antoine Assi
3ce0873931 fix 2026-04-12 16:09:36 -04:00