For steps that need to run before the web application starts, such as
scheduling tasks, this new `startup.py` script is introduced.
This fixes a recently introduced issue where task scheduling was not
being triggered, because of it being included in the
`if __name__ == "__main__":` block, which is not executed when
the application is run by Gunicorn in production environments.
We do not include this logic as part of FastAPI's lifespan
implementation, as running multiple workers with Gunicorn would
cause this logic to be executed multiple times.
When retrieving the related screenshot for a `Save` or `State`, we were
retrieving a very heavy representation of the associated `Rom` object,
only to iterate through its screenshots to find the one we needed.
This change modifies the `Save` and `State` models to directly query the
`Screenshot` model, which is much faster and more efficient. The
`DBScreenshotsHandler` has been updated to include a new `filter` method
that will simplify building queries using SQLAlchemy, something we can
extend to other handlers in the future.
Fixes#1925.
This change adds a new environment variable `DB_QUERY_JSON` that allows
users to specify querystring values for additional database connection
parameters. This is useful for passing custom query parameters to the
database connection URL.
The `DB_QUERY_JSON` variable should contain a JSON string with key-value
pairs of strings, which is the required format for the SQLAlchemy URL
`query` parameter. If the variable is not set, no additional query
parameters will be added to the database connection URL.
Closes#2093.
The `watchfiles` library supports event batching, which allows us to
process multiple filesystem changes in a single run.
This change also avoids database calls in the watcher as much as
possible.
The `emoji` library has been removed, in favor of using constants for
the few emojis used in the codebase. This reduces memory usage, and
avoids calling `emojize` for Python to discover where to replace emojis
in pre-defined strings.
This change replaces our custom `scheduler.py` script with the
`rqscheduler` command, allowing us to run the RQ scheduler as a
separate, low-memory process, by avoiding the need to maintain
the Python app in memory.
* Remove `scheduler.py` script.
* Move initialization of scheduled tasks to `worker.py`.
* Update `docker/init_scripts/init` to start the `rqscheduler`
command instead of the custom script.
* Fix scheduled tasks' `func` paths to the new project structure.
* Temporarily use a fork of `rq-scheduler` to support
username and SSL settings in the `rqscheduler` command.
The existing code incorrectly maps boolean flags to metadata sources,
leading to colliding `True` keys in the dictionary. This caused only one
metadata source to be recognized during scheduled scans, even when
multiple sources were enabled.
