Commit Graph

353 Commits

Author SHA1 Message Date
zurdi
2a35ca04e7 feat: refactor sibling data handling in RomSchema and update related methods 2026-05-26 08:21:45 +00:00
zurdi
8f48c92cbf Merge remote-tracking branch 'origin/master' into chore/frontend-v2 2026-05-26 07:44:59 +00:00
zurdi
bd4bf4f406 feat: enhance collection and game details UI
- Adjust padding in CollectionPickerRow and NewCollectionRow for a more compact layout.
- Introduce GameCard component in ManageCollectionsDialog for better visual representation of single ROMs.
- Implement SiblingBadge in GameCard to display sibling versions of ROMs.
- Add MainSiblingToggle and VersionSwitcher components to GameHeader for managing default versions and switching between ROM versions.
- Refactor CollectionSettingsDrawer to remove delete functionality, moving it to Collection.vue for consistency with other gallery surfaces.
- Enhance CollectionsIndex with specific empty state messages based on search and filter criteria.
- Update UserInterface settings to remove showSiblings toggle, integrating sibling display into groupRoms functionality.
2026-05-25 16:25:33 +00:00
Georges-Antoine Assi
fb3cc1da87 perf 2026-05-25 12:00:14 -04:00
Georges-Antoine Assi
9e9a282286 fix(roms): dedupe and sort sibling IDs for stable API output
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 11:35:16 -04:00
Georges-Antoine Assi
5e8fac36a7 Merge branch 'master' into claude/awesome-gates-NUpke 2026-05-25 11:10:04 -04:00
Georges-Antoine Assi
b9b82c751b more cleanup 2026-05-25 11:07:03 -04:00
Georges-Antoine Assi
9a87348c22 cleanup 2026-05-25 11:02:35 -04:00
zurdi
123a397c8a feat(scan): enhance ROM scanning interface with virtual scrolling and improved metadata selection
- Refactor ScanPlatform.vue to always use RVirtualScroller for listing ROMs, improving performance and layout consistency.
- Introduce a maximum viewport height for the virtual scroller to prevent excessive DOM growth.
- Update ScanPlatformRow.vue to make each ROM row a clickable router link, enhancing navigation to ROM details.
- Add a synthetic "All" option in RSelect for metadata sources, allowing users to toggle all items in multi-select mode.
- Split metadata providers into general and specific categories for clearer organization in the Scan view.
- Improve styling and layout of the scan configuration card for better user experience.
2026-05-25 12:32:12 +00:00
zurdi
17ea8da23d Merge remote-tracking branch 'origin/master' into chore/frontend-v2 2026-05-25 07:56:38 +00:00
Claude
95b1a99f2a perf(roms): avoid hydrating full Rom rows for siblings on list endpoint
The paginated ROM list eager-loaded sibling_roms via selectinload, which
hydrated full Rom ORM instances (including heavy JSON metadata columns)
for every sibling even though only an existence/count check was needed
on the frontend. On large collections this dominated request latency.

Split sibling handling by response shape:
- SimpleRomSchema (list): siblings is now list[int]; populated per page
  by a single SELECT against the sibling_roms view projecting only
  (rom_id, sibling_rom_id) — no Rom row hydration.
- DetailedRomSchema (detail): keeps full SiblingRomSchema objects, with
  load_only on (id, name, fs_name_no_tags, fs_name_no_ext) so sibling
  rows stop dragging in JSON metadata.

Frontend usage already only consumes siblings.length on list views; the
detail-page VersionSwitcher continues to receive the richer schema.
2026-05-24 23:17:34 +00:00
Georges-Antoine Assi
1a560d3660 cleanup 2026-05-24 18:57:38 -04:00
Georges-Antoine Assi
fb13f54f48 cleanup 2026-05-24 17:43:01 -04:00
Claude
8fcc16bad2 refactor(roms): replace denormalized columns with deferred column_property
Drop the migration and the multi_file / top_level_file_count columns on
roms; express both as deferred column_property correlated subqueries
against rom_files instead. The gallery list and detail queries opt in
via undefer, so they get the values computed in the same SELECT via
indexed subqueries (rom_id index already in place); other code paths
that don't read the flags pay nothing.

This keeps the gallery perf win (no rom_files load for cards) without
introducing schema state that has to stay in sync with rom_files at
write time.
2026-05-24 20:41:44 +00:00
Claude
5adffeca71 perf(roms): skip rom_files load on gallery list endpoint
The gallery list endpoint was eager-loading every rom_file row for each
paginated ROM via selectinload, then re-joining each row back to its
parent rom for the is_top_level computation. For platforms with extracted
multi-file ROMs (Xbox 360 ~1394 files/ROM, Switch ~199 files/ROM), this
made /api/roms time out at 120s even with a rom_id index.

Cards never displayed individual files — only the has_simple_single_file
/ has_nested_single_file / has_multiple_files booleans that derive from
the file list. Denormalize the underlying state onto roms as multi_file
(folder-based vs single-file) and top_level_file_count, recompute the
booleans from those columns, drop the selectinload from filter_roms, and
move the files field from SimpleRomSchema to DetailedRomSchema so the
gallery payload no longer ships file rows.

Also drop the redundant joinedload(RomFile.rom) and switch the relation
to lazy="select" so subsequent file.rom accesses resolve from the
session identity map instead of re-JOINing the parent rom per file row.

ShowQRCode.vue's folder-based DS/3DS fallback now fetches the detailed
rom on demand, since SimpleRom no longer carries files.
2026-05-24 15:02:02 +00:00
zurdi
a9b19ebde7 refactor: remove MetadataSections component and update gallery sorting logic
- Deleted the MetadataSections.vue component, which handled metadata display for various providers.
- Updated AlphaStrip.vue to include a direction prop for sorting letters in ascending or descending order.
- Enhanced GalleryShell.vue to manage grid sorting direction and integrate it with the toolbar.
- Modified GalleryToolbar.vue to add sorting direction controls for grid mode.
- Adjusted PlatformTile.vue to simplify the playable indicator display.
- Refactored useGalleryVirtualItems to normalize letter sorting with new bucket logic.
- Added subtitle support to RTextField for enhanced context display.
- Updated CollectionsIndex.vue to implement grid sorting functionality.
- Refined PlatformsIndex.vue to separate list and grid sorting states, improving overall sorting logic.
2026-05-21 14:31:30 +00:00
Georges-Antoine Assi
90945685e4 Stuff 2026-05-17 12:43:33 -04:00
zurdi
bd38eddd11 feat: add multi-file ROM fixture for testing and enhance soundtrack player functionality
- Introduced a new fixture `multi_file_rom` in `conftest.py` to create a ROM with multiple files for testing purposes.
- Updated `test_manual.py` and `test_soundtrack.py` to utilize the new fixture for testing manual uploads and soundtrack functionalities.
- Enhanced the audio tag extraction logic in `audio_tags.py` to handle oversized audio files and added a function to check allowed audio file extensions.
- Modified `nginx.py` to support inline file serving for audio files, allowing for better streaming capabilities.
- Improved error handling in the Vue components for soundtrack management, including user feedback for playback errors and metadata loading issues.
- Refactored the soundtrack player store to use local storage for volume and mute settings, simplifying state management.
- Added new localization strings for soundtrack player actions and error messages in both English (US and GB) locale files.
2026-04-20 08:33:35 +00:00
zurdi
916e8945e7 media resources revamp (manuals + soundtrack) 2026-04-17 16:02:37 +00:00
Spinnich
15b485b118 fix(collections): address reviewer feedback on types and concurrency
- Use CollectionSchema instead of ReturnType<typeof collectionsStore.getCollection>
  in AddRoms.vue and RemoveRoms.vue (simpler, per gantoine review)
- Wrap bulk INSERT in a savepoint so a concurrent duplicate-key violation
  is caught via IntegrityError and ignored rather than aborting the transaction
- Only bump Collection.updated_at in remove_roms_from_collection when rows
  were actually deleted (rowcount > 0), matching add_roms_to_collection behavior

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:46:12 -04:00
Spinnich
2ecefa3d3f Fix race condition in collection and favorite rom membership updates
Replace full rom_ids list replacement with atomic POST/DELETE endpoints
that add or remove individual ROMs from a collection. This prevents
concurrent rapid clicks from overwriting each other (last-write-wins).

Also fix missing session.flush() in add_rom_user() and add collection
endpoint tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 15:08:53 -04:00
Georges-Antoine Assi
abc69c790f fix scanning 2026-04-12 09:35:34 -04:00
Georges-Antoine Assi
85f9444b57 fix: restore get_roms_by_fs_name after stripped @with_details
Commit 3991e1b6e removed `@with_details` from `get_roms_by_fs_name` but
left the body using the `query` parameter that decorator was supposed
to inject, so every scan hit `'NoneType' object has no attribute
'filter'` and crashed the platform identification task.

Make the function self-contained: build `select(Rom)` directly and
eager-load only `Rom.platform`, the one relationship the scan loop
actually needs (via `rom.platform_slug` / `rom.platform.fs_slug`).
Keeps the prior commit's intent of avoiding the heavy `with_details`
eager-load on every batch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 09:10:27 -04:00
Georges-Antoine Assi
de9efb3da8 refactor: simplify mark_missing_roms with a single flips dict
Collapse the two parallel id lists and their mirrored chunked-update
loops into a `flips: dict[bool, list[int]]` keyed by desired state, and
drop unused rom assignments in the related tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 23:13:33 -04:00
Georges-Antoine Assi
3991e1b6ed fix: scan stalls on platforms with 10k+ already-scanned ROMs
The scan was spending excessive time on large platforms even when all ROMs
were already scanned. Root causes: per-ROM UPDATE queries for skipped ROMs
(10k individual writes), missing composite index on (platform_id, fs_name)
causing full table scans, NOT IN clauses with 10k+ values in
mark_missing_roms(), and redundant filesystem reads.

Changes:
- Add bulk_mark_present() for batch-updating skipped ROMs in one query
- Move skip detection from _identify_rom to the batch loop so skipped ROMs
  never enter the async scan pipeline, and report progress for them
- Add composite index idx_roms_platform_id_fs_name via migration 0077
- Rewrite mark_missing_roms() with flip-based approach: mark all missing,
  then un-mark present ones in chunks of 1000
- Cache filesystem reads in scan_platforms() to avoid double directory
  traversal (precounting + scanning)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:20:40 -04:00
Georges-Antoine Assi
4e5ee69343 more simplify 2026-04-06 22:29:55 -04:00
Georges-Antoine Assi
b2e1e094f7 some simplification 2026-04-06 21:53:49 -04:00
Georges-Antoine Assi
e5cb29ae80 Merge branch 'master' into feature/play-session-ingest 2026-04-06 12:42:15 -04:00
Georges-Antoine Assi
f2e8e337b2 Merge branch 'master' into save-sync 2026-04-05 21:47:53 -04:00
Georges-Antoine Assi
8664f6203f bad removal 2026-04-05 19:19:06 -04:00
Georges-Antoine Assi
e0214f100d more bot cleanup 2026-04-05 19:15:33 -04:00
Georges-Antoine Assi
7c41fb5bac revert fs_name sibling roms 2026-04-05 17:57:48 -04:00
Georges-Antoine Assi
a3ebe16a39 run fmt 2026-03-24 15:32:48 -04:00
copilot-swe-agent[bot]
a5954590ec Fix IntegrityError when updating favourites with stale ROM IDs
Co-authored-by: gantoine <3247106+gantoine@users.noreply.github.com>
Agent-Logs-Url: https://github.com/rommapp/romm/sessions/8a6fd3fe-77ab-4f7b-a0e9-1c55e7096e88
2026-03-23 22:19:06 +00:00
Georges-Antoine Assi
f107dc2752 changes from bot rview 2026-03-22 17:17:14 -04:00
nendo
719b98faaf Batch session inserts into single transaction
Replace per-item add_session with add_sessions using add_all.
No fallback on IntegrityError -- duplicate concurrent submissions
are the client's responsibility.
2026-03-22 21:21:27 +09:00
nendo
75302ed59a Add play session ingest for game time tracking
Backend API for collecting and querying play sessions, modeled after
the Argosy session data format. Clients submit batches per device,
recording both the session window and screen-on time.
2026-03-22 20:22:55 +09:00
nendo
4edb1710a5 fix: address review feedback on session handler and counter
- Restore NoResultFound behavior on update_session, complete_session,
  fail_session when row is missing (scalar returns None, old .one()
  raised -- silent None is a semantic regression)
- Remove redundant get_session call from _increment_session_counter;
  the atomic SQL increment is already a no-op on missing rows
- Log warning when passed session_id is not found in _sync_device
  instead of silently creating an orphan session
2026-03-16 10:59:49 +09:00
nendo
55638d15dc fix: address bugs, security issues, and convention violations in save-sync
- Fix broken path construction in FSSyncHandler: build_* methods now
  return relative paths; sync_watcher uses paths relative to sync base
  instead of CWD (was completely non-functional in production)
- Fix SSH connection leak in push-pull task: conn.close() now in finally
- Add log.warning for disabled SSH host key verification
- Fix race condition in session operation counter: use atomic SQL
  increment instead of read-then-write
- Extract _increment_session_counter helper, add exc_info to warnings
- Replace legacy session.query() with select() in sync_sessions_handler
- Fix orphaned session: trigger_push_pull now passes session_id to job
- Fix wasteful SSH download when no matched_save exists
- Fix BaseModel import collision in sync.py (pydantic -> project base)
- Fix ORM mutation in UserSchema.from_orm_with_request: set field on
  schema instance instead of mutating live ORM object
- Mask ssh_password and ssh_key_path in DeviceSchema API response
- Fix migration PostgreSQL compatibility: condition ON UPDATE clause
  on MySQL, drop enum in downgrade
- Rename copy-paste artifact rom_user_status_enum
2026-03-16 10:56:43 +09:00
Georges-Antoine Assi
e1b07cacfc make devices actually unique 2026-03-15 20:09:33 -04:00
Georges-Antoine Assi
e6ddc5da11 bot attempt at save sync 2026-03-14 22:13:38 -04:00
Georges-Antoine Assi
997e2c44aa start pre-4.8 cleanup 2026-03-12 23:02:12 -04:00
Zurdi
4c680f4919 Merge pull request #3121 from Bergbok/playable-filter-fix
fix: add missing platforms to playable game filter
2026-03-11 16:24:00 +01:00
Bergbok
99e41cecc2 refactor: create list for playable platforms that don't use EJS 2026-03-11 17:18:13 +02:00
Bergbok
ed61158aee fix: include DOS, Game Boy Color and Sega 32X games in playable filter 2026-03-11 16:40:39 +02:00
Bergbok
cad510a0b0 refactor: remove duplicate C64 entry from EJS_SUPPORTED_PLATFORMS 2026-03-11 16:37:49 +02:00
Bergbok
8517fb9aca fix: include browser games in playable filter 2026-03-11 16:11:08 +02:00
Georges-Antoine Assi
6f02a4beff Merge pull request #3114 from tmgast/feature/client-api-tokens
Add client API tokens with QR pairing flow
2026-03-10 22:17:05 -04:00
nendo
e0b25fbc6c feat(client-tokens): add client API tokens with QR pairing flow
Long-lived, revocable, scope-restricted tokens for external clients
(mobile apps, retro handhelds, third-party tools). Includes:

- Backend: model, migration, DB handler, auth integration (rmm_ prefix
  routing in HybridAuthBackend), CRUD + pairing + exchange endpoints,
  rate limiting, scope intersection enforcement, admin oversight
- Frontend: settings page with token management table, stepped
  create/deliver dialog (config -> copy/pair), QR code with RomM logo,
  admin token table, standalone /pair page for QR scan landing
- /pair page supports custom-scheme callbacks for app deep linking,
  falls back to displaying code for manual entry
- 33 backend tests across 5 classes (CRUD, auth, isolation, pairing,
  admin)
2026-03-11 10:56:35 +09:00
Georges-Antoine Assi
0c18c2588c run fmt 2026-03-10 21:47:27 -04:00