Commit Graph

6900 Commits

Author SHA1 Message Date
Michael Manganiello
80291f4be1 misc: Move cache initialization to startup script
Guarantee that cache is initialized during startup, and only once,
instead of every time a `MetadataHandler` object is instantiated.

Also, improve logic to determine `fixtures` paths.
2025-08-13 12:26:15 -03:00
Michael Manganiello
57e9a9f0fd Merge pull request #2258 from rommapp/misc/create-startup-script
misc: Create startup script to run initial tasks before main application
2025-08-13 11:16:00 -03:00
Michael Manganiello
63f84b78d5 misc: Create startup script to run initial tasks before main application
For steps that need to run before the web application starts, such as
scheduling tasks, this new `startup.py` script is introduced.

This fixes a recently introduced issue where task scheduling was not
being triggered, because of it being included in the
`if __name__ == "__main__":` block, which is not executed when
the application is run by Gunicorn in production environments.

We do not include this logic as part of FastAPI's lifespan
implementation, as running multiple workers with Gunicorn would
cause this logic to be executed multiple times.
2025-08-12 23:14:26 -03:00
Georges-Antoine Assi
6b31a64347 Merge pull request #2256 from rommapp/romm-2163
[ROMM-2163] Fix excluding single roms without extensions
2025-08-12 17:35:21 -05:00
Georges-Antoine Assi
47e9138cbb lower the extension in config manager 2025-08-12 18:24:21 -04:00
Georges-Antoine Assi
774be0cbde [ROMM-2163] Fix excluding single roms without extensions 2025-08-12 18:18:29 -04:00
Georges-Antoine Assi
8dfdcc923e fix select on platform picker scan 2025-08-12 15:44:05 -04:00
Michael Manganiello
83b893723e Merge pull request #2254 from rommapp/fix/upload-manual-path
fix: Correctly upload manual to resources path
2025-08-12 16:27:10 -03:00
Michael Manganiello
1d2403d7cb fix: Correctly upload manual to resources path
Use the right filesystem handler for the manual to be uploaded to the
`resources` path instead of the `rom` path.
2025-08-12 16:17:20 -03:00
Michael Manganiello
6aca53fa38 Merge pull request #2252 from rommapp/feat/opentelemetry-integration-watcher
feat: Add OpenTelemetry integration to file watcher
2025-08-12 12:34:18 -03:00
Michael Manganiello
f95e0edff4 feat: Add OpenTelemetry integration to file watcher
Run file watcher using `opentelemetry-instrument` to enable tracing for
the watcher service.
2025-08-12 11:25:40 -03:00
Michael Manganiello
92e11beb8b Merge pull request #2251 from rommapp/feat/opentelemetry-integration
feat: Add OpenTelemetry integration for API service
2025-08-12 09:56:56 -03:00
Michael Manganiello
652df5d4cd feat: Add OpenTelemetry integration for API service
This change introduces OpenTelemetry dependencies, and uses
`opentelemetry-instrument` auto-instrumentation to allow users to
configure OpenTelemetry settings via environment variables [1].

The only custom environment variable added is
`OTEL_SERVICE_NAME_PREFIX`, which allows users to set a prefix to the
service names included by RomM to differentiate between `api`, `worker`,
and `watcher` services.

The instrumentation of RQ workers and file watcher will be added in
subsequent pull requests.

[1] https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
2025-08-12 09:18:59 -03:00
Georges-Antoine Assi
649a6dc7f7 Remove link from aikido report 2025-08-11 09:18:39 -04:00
Michael Manganiello
edbff6b7e8 Merge pull request #2248 from rommapp/fix/performance-asset-screenshot
fix: Improve performance on asset screenshot retrieval
2025-08-10 16:05:35 -03:00
Georges-Antoine Assi
c135847889 Merge pull request #2249 from rommapp/romm-2247
[ROMM-2247] Return early if cover write fails
2025-08-10 15:01:21 -04:00
Georges-Antoine Assi
4728e5da67 [ROMM-2247] Return early if cover write fails 2025-08-10 14:51:06 -04:00
Michael Manganiello
00d7815889 fix: Improve performance on asset screenshot retrieval
When retrieving the related screenshot for a `Save` or `State`, we were
retrieving a very heavy representation of the associated `Rom` object,
only to iterate through its screenshots to find the one we needed.

This change modifies the `Save` and `State` models to directly query the
`Screenshot` model, which is much faster and more efficient. The
`DBScreenshotsHandler` has been updated to include a new `filter` method
that will simplify building queries using SQLAlchemy, something we can
extend to other handlers in the future.

Fixes #1925.
2025-08-10 15:33:13 -03:00
Michael Manganiello
40e824db2c Merge pull request #2244 from rommapp/feat/database-query-string
feat: Add support for database query string using DB_QUERY_JSON variable
2025-08-10 10:54:10 -03:00
Michael Manganiello
889c71444b feat: Add support for database query string using DB_QUERY_JSON variable
This change adds a new environment variable `DB_QUERY_JSON` that allows
users to specify querystring values for additional database connection
parameters. This is useful for passing custom query parameters to the
database connection URL.

The `DB_QUERY_JSON` variable should contain a JSON string with key-value
pairs of strings, which is the required format for the SQLAlchemy URL
`query` parameter. If the variable is not set, no additional query
parameters will be added to the database connection URL.

Closes #2093.
2025-08-10 00:31:03 -03:00
Michael Manganiello
0e9f977e60 Merge pull request #2243 from rommapp/misc/no-warning-on-redis-host
misc: Do not warn when REDIS_HOST is set
2025-08-09 21:31:08 -03:00
Michael Manganiello
4eea761d87 misc: Do not warn when REDIS_HOST is set
Change log level to info when REDIS_HOST is set, as this is a valid
and common configuration.
2025-08-09 21:21:24 -03:00
Michael Manganiello
c0483832ac Merge pull request #2240 from rommapp/fix/retroachievements-case-insensitive-hash-match
fix: Make RetroAchievements hash matching case-insensitive
4.1.0-alpha.1
2025-08-09 16:13:41 -03:00
Georges-Antoine Assi
e164b8581b Merge pull request #2231 from rommapp/find-best-match
Use best-match algo for more providers
2025-08-09 15:11:21 -04:00
Georges-Antoine Assi
29ddcb45a4 fix all i18n locales 2025-08-09 15:04:49 -04:00
Georges-Antoine Assi
84ded3bc24 pick largest file from the list 2025-08-09 14:47:34 -04:00
Michael Manganiello
f13d2198db fix: Make RetroAchievements hash matching case-insensitive
Fixes #2182.
2025-08-09 12:48:28 -03:00
Georges-Antoine Assi
d005dba90b fix a bunch of shit 2025-08-09 11:16:12 -04:00
Georges-Antoine Assi
afaef09c84 cleanup 2025-08-08 17:08:16 -04:00
Georges-Antoine Assi
16c8c9f993 only init JW once 2025-08-08 17:03:28 -04:00
Georges-Antoine Assi
69079b2a90 Merge branch 'master' into find-best-match 2025-08-08 16:55:39 -04:00
Georges-Antoine Assi
659a3eb104 use jaro-winkler algo 2025-08-08 16:54:26 -04:00
Georges-Antoine Assi
b1609096de fix file_type typo 2025-08-08 15:00:18 -04:00
Michael Manganiello
86b7d021b7 Merge pull request #2238 from rommapp/fix/docker-build-fixes
fix: Docker build and init fixes
2025-08-08 15:35:57 -03:00
Michael Manganiello
edece77ffd Merge pull request #2237 from rommapp/fix/dockerignore
fix: Correctly set up dockerignore file
2025-08-08 15:35:27 -03:00
Michael Manganiello
676079f1e5 fix: Docker build and init fixes
* Added `linux-headers` back, but only for development stage.
* Fixed initialization script, as `uv` is not included in the final
  Docker image.
* Initialize variable `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA`.
2025-08-08 15:09:27 -03:00
Michael Manganiello
9c0df02296 fix: Correctly set up dockerignore file
* Rename file to `Dockerfile.dockerignore`. As the `Dockerfile` is not
  in the root directory, the Docker spec requires the file to be
  named `<dockerfile_name>.dockerignore`.
* Update ignore rules, and include `.mypy_cache/` directory.
2025-08-08 14:38:11 -03:00
Michael Manganiello
6d9c6143cb Merge pull request #2234 from rommapp/misc/move-tests-folder
misc: Separate tests folder from backend code
2025-08-08 13:11:09 -03:00
Michael Manganiello
ba21cbc1e1 misc: Separate tests folder from backend code
Create separate `tests/` folder for all tests. This will also simplify
not copying tests code into the Docker image.
2025-08-08 12:49:13 -03:00
Michael Manganiello
ee44a5c788 Merge pull request #2233 from rommapp/misc/remove-indirect-dependency-freezegun
misc: Remove indirect dependency freezegun
2025-08-08 12:42:32 -03:00
Michael Manganiello
26177eb95e misc: Remove indirect dependency freezegun
Update `rq-scheduler` fork to remove the unnecessary dependency on
`freezegun`.
2025-08-08 11:37:49 -03:00
Georges-Antoine Assi
de42333f2f Merge pull request #2232 from rommapp/native-7z-parse
Use native 7z to extract files and calculate hashes
2025-08-08 09:23:29 -04:00
Georges-Antoine Assi
c47be7240b changs from bot review 2025-08-07 18:15:49 -04:00
Georges-Antoine Assi
9e37d87610 fix complex game names 2025-08-07 18:02:04 -04:00
Georges-Antoine Assi
b247a5c5bb small tweaks 2025-08-07 17:39:57 -04:00
Georges-Antoine Assi
1a2944806b Use best-match algo for more providers 2025-08-07 16:57:29 -04:00
Michael Manganiello
3f98ef6b00 Merge pull request #2230 from rommapp/feat/watcher-watchfiles
feat: Migrate watcher to use watchfiles library
2025-08-07 15:36:01 -03:00
Georges-Antoine Assi
926fe26fd9 [exp] use cli 7z to cacl hash files 2025-08-07 13:12:32 -04:00
Michael Manganiello
76db0ab98c feat: Migrate watcher to use watchfiles library
The `watchfiles` library supports event batching, which allows us to
process multiple filesystem changes in a single run.

This change also avoids database calls in the watcher as much as
possible.
2025-08-07 13:28:49 -03:00
Michael Manganiello
1e6bfffe92 Merge pull request #2229 from rommapp/misc/move-pytest-cov-to-test-extra
misc: Move pytest-cov dependency to test extra
2025-08-07 12:45:33 -03:00