Internal members of multi-file archives (zip/tar/7z/rar) are now hashed
individually (crc/md5/sha1) and stored in a new `archive_members` JSON
column on the archive's RomFile, alongside the existing composite hash
used for hash-database matching. Only the archive itself is surfaced as
a RomFile so full_path keeps pointing at a file that exists on disk,
which is the constraint that previously forced us to choose between
composite-only or broken downloads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidate all archive readers (zip/tar/7z/rar) and 7z-internal helpers
into a single utils/archives.py module to keep the archive surface area
in one place.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-internal-member RomFiles produced full_paths that didn't exist on
disk, breaking downloads and zip-building. Stream entries into the
composite hash only and emit one RomFile pointing at the archive itself.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apply the helper to the three other per-file FileHash constructions
(folder-walk hash, empty-archive fallback, single-file hash). The
all-empty FileHash literals are left alone since the helper would be
strictly more obscure for that case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add read_rar_archive_files via the existing 7zz binary (which natively
handles RAR3/RAR5 read), and collapse the per-extension reader dispatch
into an ARCHIVE_READERS dict so future formats are one entry away. Also
extract a small _make_file_hash helper to remove the repeated nested
ternaries in the inner loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pull file/archive readers (zip/tar/gz/bz2/7z), CHD parsing, and the
shared libmagic MIME detector out of roms_handler.py into a new
utils/archives.py. Rename the previously underscore-prefixed
read_zip_archive_files / read_tar_archive_files to match the existing
read_7z_archive_files convention, and consolidate the duplicated
"with lock: detector.from_file()" pattern into a detect_mime_type helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User-configured EXCLUDED_MULTI_PARTS_EXT/FILES are intentionally not
applied to archive internal files. Archives are curated ROM sets where
every file is relevant — user custom exclusions (e.g. "bin") could
silently produce incorrect composite hashes. Only the hardcoded
DEFAULT_EXCLUDED_FILES/EXTENSIONS (junk like .DS_Store, gamelist.xml)
are applied.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use AnyioPath.stat() instead of os.path.getmtime in async context (ASYNC240)
- Add assert to narrow rom_md5_h/rom_sha1_h from HASH|None to HASH (mypy/union-attr)
- Auto-formatted long log.error calls in archive_7zip.py (ruff)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a user manually sets metadata IDs (e.g., ra_id, launchbox_id, hasheous_id)
via the UI and then runs "Refresh Metadata" (which defaults to UNMATCHED scan type),
no metadata was fetched because UNMATCHED conditions checked `not rom.xxx_id` — so
if the ID was already set, the handler was skipped entirely.
Fix: Change each handler's UNMATCHED condition to also trigger when the ID is set
but the corresponding metadata dict is empty (i.e., `not rom.xxx_id or not rom.xxx_metadata`).
For handlers that support ID-based lookup (RA, Launchbox, IGDB, MobyGames, SS,
Flashpoint), also add the `get_rom_by_id` path inside the function.
For Hasheous: when hash lookup fails but `hasheous_id` is set on an existing ROM
(not newly added), return a partial HasheousRom built from the existing sub-IDs
(igdb_id, ra_id, tgdb_id) so the downstream get_igdb_game / get_ra_game proxy
calls can still enrich the ROM.
Add three targeted tests to validate:
- UNMATCHED scan fetches RA metadata when ra_id is set but ra_metadata is empty
- UNMATCHED scan skips RA when both ra_id and ra_metadata are already populated
- UNMATCHED scan passes existing sub-IDs to Hasheous proxies when hash lookup fails
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/098b482f-9f73-4f35-819a-b55004a79b13
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
- Add `STANDALONE_EXPANSION` game type to the `with_game_type` filter in
`_search_rom` so games like "Ecco: The Tides of Time" (which IGDB classifies
as a standalone expansion) are included in the first search pass and are not
confused with their parent game ("Ecco The Dolphin")
- Fix the expanded search fallback to fetch and compare ALL unique game IDs
returned by the IGDB search endpoint, instead of only the first result
- Add tests to verify both fixes
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/d6a0c1dd-e541-4d8e-a272-9e5511a2077e
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
For ROMs tagged with multiple regions (e.g. "(Japan, USA)"), filename order
previously decided which region's name and box art won. Now reorder the rom's
filename-tagged regions by SCAN_REGION_PRIORITY before prepending, so the
user's configured preference wins among the regions the file is actually
tagged as. Untagged priority regions still cannot outrank a filename-tagged
region.
Also tweak the Total Rescan → Complete Rescan label in en_GB/en_US scan
locales.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix several issues in ScreenScraper API request/response handling:
- Correctly handle SS-specific HTTP error codes (KO responses, 429, 431,
and the SS-quirk of returning 401 when server CPU >60%).
- Construct requests with proper parameter encoding so jeuInfos lookups
and search queries return the expected results.
- Store media URLs returned by SS as-is, preserving the dev credential
query parameters required for media playback. Removing them broke
downstream media fetches.
To keep dev credentials out of log output, add a redacting formatter in
the logger pipeline that scrubs ssid/sspassword/devid/devpassword query
parameters from any URL it sees.
Test coverage added for the new HTTP error paths and the as-is URL
storage behaviour.
Importer (gamelist/launchbox file:// flows) and exporters (gamelist.xml,
metadata.pegasus.txt local exports) now hardlink media assets when source
and destination share a filesystem, falling back transparently to a copy
on EXDEV / EPERM / EOPNOTSUPP / EMLINK / EACCES (cross-device, FAT32,
exFAT, network mounts, etc.). Saves disk space and is effectively
instantaneous on large files (videos, manuals, miximages).
Covers keep a real copy (allow_link=False) because _store_cover resizes
the small cover in place via PIL.Image.save, which would truncate the
shared inode and corrupt the user's source image.
Also makes FSSyncHandler tolerate a missing/unwritable /romm/sync at
startup: an OSError from mkdir now logs a warning instead of crashing
the whole app at module-import time. Sync calls still fail at use time
if the mount remains broken — the right place to surface the error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI runs pytest without root, so the new /var/lib/romm/sync default
fails when FSSyncHandler tries to mkdir its base path at import time.
The other handlers stay writable in tests because they derive from
ROMM_BASE_PATH=romm_test (relative); pin SYNC_BASE_PATH the same way.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
