Addresses review feedback: `db_name` is interpolated into CREATE DATABASE
statements via f-strings (identifiers can't be bind parameters), so validate
it up-front against a plain-identifier allowlist ([A-Za-z0-9_]+) and refuse
anything containing quoting/other characters, rather than rely on quoting.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses review feedback: granting ALL PRIVILEGES on *.* is overly broad,
especially against a shared DB instance. A database-level grant on the
`romm\_test%` wildcard pattern still lets the user CREATE the per-worker
databases (romm_test_gw0, ...) needed by pytest-xdist, while confining it to
that namespace — verified that out-of-namespace CREATE DATABASE is denied.
PostgreSQL needs no equivalent change: its CI service user (POSTGRES_USER)
is the container superuser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add pytest-xdist and run the backend test suite across multiple workers
(`-n 4` in CI). Each worker gets its own database so the autouse
`clear_database` fixture can't wipe rows another worker is mid-test with:
- Rootdir `backend/conftest.py` sets a per-worker `DB_NAME`
(`romm_test_gw0`, ...) before any app module is imported, so each
worker's engine binds to its own database.
- `tests/conftest.py` creates the per-worker database on demand (mariadb/
mysql and postgresql paths) just before migrations run.
- The test user's grant is widened to `*.*` (setup.sql + CI) so it can
`CREATE DATABASE` for the workers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- main.py: await the cancelled log-forwarder task (suppressing
CancelledError) so its pubsub/lock cleanup finishes before shutdown.
- forwarder: only heartbeat the Redis lock while we still own it; if a
stall let another worker take it, relinquish forwarding to avoid
duplicate lines (the outer loop re-contends).
- endpoints/logs.py: derive MAX_LOG_LIMIT from LOG_BUFFER_SIZE so the
REST backfill never drifts from the producer's ring buffer.
- Logs.vue: append the download <a> to the DOM before click() (matches
the Patcher pattern) for cross-browser reliability.
- Add tests/endpoints/test_logs.py: non-admin 403, limit clamping to
[1, MAX_LOG_LIMIT], oldest-first ordering, and malformed-entry skip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The log-stream handler and forwarder deliberately swallow exceptions: the
handler is a best-effort mirror that must never raise into the app, and the
forwarder can't log its own failures without feeding back into the stream.
Annotate these with `# nosec` (with justification) so Trunk's bandit check
passes, keeping lines within black's width.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Introduced a new logs view for admin users, allowing real-time monitoring of backend logs.
- Implemented a log entry streaming mechanism using Socket.IO.
- Added filtering and searching capabilities for log entries.
- Created localized log messages in Spanish, French, Hungarian, Italian, Japanese, Korean, Polish, Portuguese, Romanian, Russian, Simplified Chinese, and Traditional Chinese.
- Updated router and sidebar components to include the new logs route.
- Enhanced user interface with tooltips and buttons for copying and downloading logs.
Drop the name_sort_key_custom flag/migration in favour of a flagless rule: a
key is "custom" when it no longer equals compute(name). Apply that consistently
across all three write paths so a manual sort key survives renames while a
derived key keeps following the name:
- @validates re-derives on name assignment only when the stored key still
matches the derived value; direct name_sort_key assignment stores a
normalized custom key (or reverts to derived when cleared). Handles both
kwarg orders at construction.
- update_rom mirrors the same check for the bulk update() path it bypasses.
- The edit endpoint only writes the key when the user actually changed the
field, delegating the untouched case to update_rom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the separate `sort_name` column into `name_sort_key`, which is now
the single user-settable sort field: always normalized and indexed for fast
ordering, derived from `name` by default, and overridable. A new
`name_sort_key_custom` boolean marks user/metadata overrides so they survive
renames and rescans.
- Drop the `roms.sort_name` column; repurpose migration 0085 to add
`name_sort_key_custom`.
- Derive the key via `@validates("name")` unless pinned custom; the edit
dialog, unmatch flow, and ES-DE gamelist <sortname> set custom keys.
- update_rom / scan_rom keep the columns in sync explicitly (bulk update and
construction bypass / reorder the validator).
- Frontend: edit field drives name_sort_key (empty when auto), api sends the
override only when custom, regenerated types updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The revision id '0084_add_roms_search_and_sort_indexes' (37 chars)
exceeded alembic_version.version_num VARCHAR(32), so every migration
run and test setup failed with 'Data too long for column version_num'
when stamping the revision. Rename it to '0084_roms_search_sort_indexes'
(29 chars) and update the 0085 down_revision link to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Added RDropzone component for handling file uploads with a customizable interface.
- Integrated RDropzone into Patcher and Upload views, replacing previous drop zone implementations.
- Enhanced ScreenshotsTab with additional functionality for community screenshots, including visibility toggles and owner display.
- Updated styles for improved user experience and responsiveness.
- Created Storybook stories for RDropzone to demonstrate its usage and interaction.
- Implemented screenshot upload feature in the ROM management section.
- Added new API methods for uploading and removing screenshots.
- Enhanced UI to support drag-and-drop for screenshots and display uploaded images.
- Updated localization files for Russian, Simplified Chinese, and Traditional Chinese to include new screenshot-related strings.
- Improved the FilesTab and MediaTab components to manage screenshots effectively.
- Added delete functionality for user-uploaded screenshots with confirmation prompts.
- Updated styling for screenshot elements to improve user experience.
- Include order_by / order_dir / group_by_meta_id in the char-index cache
key. The computed AlphaStrip positions depend on ordering and grouping,
so keying only by user reused a stale index when those changed.
- Gate the relevance-ranking test assertions to MySQL/MariaDB. Relevance
ordering uses MATCH ... AGAINST; PostgreSQL (also in the CI matrix) falls
back to name ordering, where those assertions don't hold.
- Clarify the order_by API docstring: relevance ordering applies on
MySQL/MariaDB; other databases fall back to name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralize the *_no_tags / *_no_ext / *_extension columns (derived from a
file name) behind @validates hooks instead of computing them by hand at
every write site:
- Add pure helpers (compute_file_name_parts and friends) to models.base;
the filesystem base handler now delegates to them.
- Add @validates on Rom (fs_name), BaseAsset (file_name, inherited by all
asset subclasses), and Firmware (file_name).
- update_rom keeps the fs_name-derived columns in sync on bulk update(),
which also fixes the rename path never updating fs_extension.
- Drop the now-redundant computations at the scan/rename call sites.
Also fix the migration backfill loop and a pre-existing list[str | None]
type mismatch surfaced in scan_handler. Add tests for the helpers, the
validators, and the update_rom bulk-sync path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the before_insert/before_update event listener with a
@validates("name") method on Rom. The sort key is now recomputed exactly
when name is assigned (ORM construction or mutation) rather than on every
flush of any field. Bulk update() still sets it explicitly in update_rom.
Also linearize the merged 0084 migration onto 0083_rom_category_soundtrack
so there is a single alembic head.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Combine 0083_add_roms_search_indexes, 0084_add_roms_name_index, and
0085_add_roms_name_sort_key into one migration (revision 0084) off
0082_save_origin_device.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a few new indexes to handle full-text searches instead of doing
`ILIKE` matching, improving performance substantially.
Alongside that, a few other things were done in order to improve search
performance, such as caching filter values so they're not computed on
each request to /api/roms. Overall, this should have a very noticeable
impact on large collections when using the search feature.
