Commit Graph

3586 Commits

Author SHA1 Message Date
Georges-Antoine Assi
8210bab806 Tolerate forward-compat values and malformed YAML in config loader
Sample configs shipped with newer releases can reference media types or
options unknown to older versions; previously the loader called
sys.exit(3) and refused to boot. Now drop unknown scan.media entries,
fall back to defaults for unknown gamelist media thumbnail/image values,
and recover from YAML parse errors with a clear critical log instead of
a traceback. Type-mismatch validation still hard-fails since those are
real misconfigurations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:26:43 -04:00
Georges-Antoine Assi
8999b66574 Preserve local LaunchBox data on UPDATE scans and video extensions
UPDATE scans with a known launchbox_id were calling get_rom_by_id(),
which only returns remote data and bypassed get_rom()'s local-first
merge — so local-only fields like Notes were getting clobbered and
local media matching fidelity dropped. get_rom_by_id() now optionally
takes fs_name/platform_slug and merges the local entry when its
DatabaseID matches the requested id.

Also fixes populate_rom_specific_paths writing every video as
video.mp4 regardless of source extension; since store_media_file is a
byte copy, .mkv/.webm contents would be served as .mp4 and break MIME
sniffing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:32:34 -04:00
Georges-Antoine Assi
d8ef6f0c05 Merge branch 'master' into local-lb-fix 2026-05-09 13:20:31 -04:00
Georges-Antoine Assi
547c64c4f5 update hltb api 2026-05-09 12:56:01 -04:00
Georges-Antoine Assi
d009662499 Merge pull request #3342 from rommapp/copilot/add-copy-default-config-yml
Auto-create missing `config.yml` on startup in mounted config volume
2026-05-09 11:00:22 -04:00
Georges-Antoine Assi
e3aaa106a2 perf(backend): reuse libmagic instance for image upload validation
magic.Magic(mime=True) loads the magic database from disk on construction;
instantiating it per request was adding pointless overhead to every avatar
and artwork upload. Share a module-level instance guarded by a lock (the
underlying magic_t handle is not thread-safe), and surface MagicException
as a 400 so a sniffing failure fails closed instead of bubbling a 500.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 10:14:38 -04:00
Georges-Antoine Assi
783d9a257e test(backend): cover artwork upload validation for roms and collections
Adds rejection + acceptance tests for update_rom, add_collection, and
update_collection artwork uploads, mirroring the existing avatar tests:
non-image content returns 400, and a real PNG uploaded under a misleading
filename like payload.html is stored with the trusted .png extension.

Also fixes two `return HTTPException(...)` → `raise` in raw.py so the 404
path actually surfaces instead of silently returning the exception object.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:37:44 -04:00
Georges-Antoine Assi
53f14f5710 fix(backend): validate uploaded images with libmagic before storing
Avatar, ROM artwork, and collection artwork uploads now sniff the file
header with libmagic and reject anything that isn't PNG/JPEG/WebP/GIF,
saving the file with an extension derived from the detected MIME rather
than the user-supplied filename. Pairs with the raw asset endpoint,
which decides inline vs attachment from the on-disk extension.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:18:02 -04:00
Georges-Antoine Assi
5ea5aecd25 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-08 14:26:11 -04:00
copilot-swe-agent[bot]
c0910dde1f fix(config): initialize state when config creation fails
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/0c94e1c8-0f79-4f5f-a72b-e45832906d00

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-08 13:56:51 +00:00
copilot-swe-agent[bot]
eae577f74c refactor(config): use pathlib for missing config creation
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/0c94e1c8-0f79-4f5f-a72b-e45832906d00

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-08 13:55:38 +00:00
copilot-swe-agent[bot]
9ef217339f test(config): cover auto-created config directory
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/0c94e1c8-0f79-4f5f-a72b-e45832906d00

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-08 13:54:19 +00:00
copilot-swe-agent[bot]
fdd9047b37 feat(config): auto-create missing config.yml on startup
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/0c94e1c8-0f79-4f5f-a72b-e45832906d00

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-08 13:52:53 +00:00
Georges-Antoine Assi
5e3a2707b0 cleanup 2026-05-03 19:39:19 -04:00
copilot-swe-agent[bot]
da005cf81a Optimize fnmatch check and use consistent n64 filename in test
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8cbbc2ca-a3e3-4c61-9e47-f8544d59231a

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 23:36:23 +00:00
copilot-swe-agent[bot]
78c0b5b894 Fix broken test_rename_fs_rom_same_name function definition after new test insertion
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8cbbc2ca-a3e3-4c61-9e47-f8544d59231a

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 23:35:19 +00:00
copilot-swe-agent[bot]
9593c30292 Address PR review: normalize exclusion sets, avoid duplicates, add multi-dot test for get_rom_files
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8cbbc2ca-a3e3-4c61-9e47-f8544d59231a

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 23:34:30 +00:00
copilot-swe-agent[bot]
101629628e Simplify extension exclusion to use ends-with check instead of sub-extension iteration
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/a81b2023-a243-4721-bc5e-c6fa1a473a79

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 22:46:21 +00:00
copilot-swe-agent[bot]
55cd0cfc4f Support compound suffix exclusions like "hash.txt" for multi-dot filenames
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/d1c69638-bfa0-480e-8050-d565b234ea44

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-03 01:29:04 +00:00
copilot-swe-agent[bot]
21de7e21f8 Fix file exclusion for multi-dot filenames (e.g. game.nds.hash.txt)
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/2f711770-100b-4e9e-a66e-ab1a74f025f8

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-05-02 16:50:30 +00:00
Georges-Antoine Assi
580ee4c694 whatevefr 2026-04-30 21:01:22 -04:00
Georges-Antoine Assi
699466b37f cleanup 2026-04-30 19:31:13 -04:00
Georges-Antoine Assi
c4431e0ae4 cleanup 2026-04-30 15:22:22 -04:00
Georges-Antoine Assi
07af3d0f64 run fmt 2026-04-30 14:48:20 -04:00
Georges-Antoine Assi
962a9bfa7e one more 2026-04-30 14:39:52 -04:00
Georges-Antoine Assi
fc8d69dc0c update tests 2026-04-30 14:18:49 -04:00
Georges-Antoine Assi
0e7359132b test: update fs handler tests for STRUCTURE_PATH_A/B refactor
Switch the get_platform_fs_structure, get_firmware_fs_structure,
get_platforms_directory and get_roms_fs_structure tests from mocking
os.path.exists to mocking glob.glob, matching the new STRUCTURE_PATH_B
detection logic. Rename the existing high_priority/normal_structure
variants to structure_a/structure_b for clarity, and fix
test_platform_specific_behavior so its monkeypatch wraps the calls
instead of being applied after them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 14:00:53 -04:00
Georges-Antoine Assi
96c3634b80 refactor: split HIGH_PRIO_STRUCTURE_PATH into STRUCTURE_PATH_A/B
Replace the single HIGH_PRIO_STRUCTURE_PATH config attribute with two
glob patterns (STRUCTURE_PATH_A = roms/*, STRUCTURE_PATH_B = */roms) and
update all call sites to detect Structure B via glob.glob, defaulting to
Structure A when no match is found.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 12:52:22 -04:00
copilot-swe-agent[bot]
ee1faf363c fix: update test mocks to use job.id attribute instead of get_id()
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/f053983a-ae39-44b3-a59d-083d65f2055d

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-04-29 19:57:57 +00:00
copilot-swe-agent[bot]
af870e0278 fix: replace deprecated job.get_id() with job.id for rq >= 2.7.0 compatibility
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/4326c200-57f5-4a27-96e8-570df82968e5

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-04-29 19:31:13 +00:00
Georges-Antoine Assi
aee17e51d5 fix 2026-04-29 14:43:37 -04:00
Georges-Antoine Assi
832447bbb1 rnu fmt 2026-04-29 14:30:48 -04:00
copilot-swe-agent[bot]
80b5fdf21f fix: add periodic janitor to clean up orphaned upload tmp directories
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/2e55dd7f-d99d-48a2-ae15-0b8c4817cc6e

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-04-29 16:42:12 +00:00
copilot-swe-agent[bot]
2498ecf729 fix: store chunk uploads under ROMM_BASE_PATH and show upload speed in UI
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/2f626cb9-e564-4764-b9e9-3de396e3e7ca

Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
2026-04-29 12:39:47 +00:00
Yukine
a00bb281e4 refactor(utils): move fire_and_forget helper into utils.background_tasks 2026-04-21 12:50:10 +02:00
Yukine
1591e0971a fix(playmatch): only fire suggestion on manual mapping updates 2026-04-21 11:07:26 +02:00
Yukine
5ecdc8fd13 fix(playmatch): filter falsy provider ids and surface non-2xx responses 2026-04-21 10:56:48 +02:00
Yukine
fee40dc355 feat(playmatch): send a suggestion to playmatch on manual match 2026-04-21 10:54:41 +02:00
Georges-Antoine Assi
c2d6b08a6b Merge pull request #3293 from rommapp/copilot/fix-update-tags-metadata
feat: open refresh metadata dialog with scan type and metadata source selection
2026-04-20 21:57:29 -04:00
Georges-Antoine Assi
ac45d1a32f changes from bot review 2026-04-20 21:51:30 -04:00
Georges-Antoine Assi
98f1d65a7c fixes 2026-04-20 21:36:59 -04:00
zurdi
d7a896b5da fix: re-parse tags from filename when renaming a rom
Renaming a rom via PUT /roms/{id} only updated fs_name and its
derivatives, leaving regions/languages/tags/revision/version stale
against the new filename. Re-parse them whenever fs_name changes so
edits like "patapon (Fr En)" -> "Patapon (Fr, En)" are reflected in
the database.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 22:23:33 +00:00
Georges-Antoine Assi
91d6928281 Lift LaunchBox video extensions to module constant
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 14:40:36 -04:00
Georges-Antoine Assi
fad2e5c77d Use correct LaunchBox MameFile schema
The sample_metadata.zip test fixture used invented tag names
(<Description>, FileName ending in .zip). Real LaunchBox Mame.xml
(see backend/tasks/fixtures/launchbox/mame.xml) uses <Name> as the
full title and keys MameFile entries by the stem (e.g. wrlok_l3),
no extension. Read Name instead of Description, and fall back to
the stem when the raw fs_name lookup misses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 14:31:16 -04:00
Georges-Antoine Assi
f33844be40 Find local LaunchBox media for remote-only rom matches
remote_media_req built a MediaRequest with platform_name=None, which
made _build_local_media_context bail and never search on-disk
Images/Manuals/Videos. Any rom that matched only via Metadata.xml (no
local XML entry) ended up with images: [] even though LaunchBox art
was sitting on disk. Thread platform_name and fs_name into
remote_media_req, falling back to the remote entry's Platform field
when no slug is available (e.g. lookups by database id).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:32:54 -04:00
Georges-Antoine Assi
8fc426b35c Import local LaunchBox videos
LaunchBox stores videos under /romm/launchbox/Videos/<Platform>/<Game>.<ext>
but the handler only extracted YouTube IDs; LAUNCHBOX_VIDEOS_DIR was
defined and unused. Add _get_video() to surface local videos as
launchbox-file:// URLs, thread a populate_rom_specific_paths() through
the scan pipeline to set video_path once the rom is known, and mirror
the SS/gamelist branch in the scan socket so store_media_file actually
copies the video into the rom's resource directory. Rom.path_video now
also reads from launchbox_metadata.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:13:42 -04:00
Georges-Antoine Assi
a83f3bdca7 Wire up LaunchBox MAME arcade lookups
The scheduled task populated LAUNCHBOX_MAME_KEY from Mame.xml but no
handler code ever read it, so arcade ROMs like pacman.zip never matched
against Metadata.xml (which keys on full titles like "Pac-Man"). Add
RemoteSource.get_mame_entry() and an ARCADE branch in get_rom() that
resolves the MAME filename to its Description before name lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:00:26 -04:00
Georges-Antoine Assi
40689d7e39 Fix LaunchBox local-media file:// paths resolving under library root
LaunchBox produced file:// URIs relative to /romm/launchbox, but the
resources handler resolved them under /romm/library via fs_rom_handler,
so local images/manuals/screenshots were never found. Switch LaunchBox
to a distinct launchbox-file:// scheme and add FSLaunchboxHandler +
_resolve_local_file_uri to route each scheme to the correct root.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 22:59:00 -04:00
Georges-Antoine Assi
49fd1280af Merge pull request #3273 from rommapp/copilot/fix-custom-boxart-download
fix: include `cus` region in default ScreenScraper fallback regions
2026-04-16 21:37:32 -04:00
Georges-Antoine Assi
7b20101d50 Merge pull request #3270 from Namaneo/home-boxarts
A bunch of minor bug fixes
2026-04-16 11:16:16 -04:00