As recommended by SQLAlchemy [1], this change makes a single
instantiation of the database engine and session maker, instead of one
entity per handler.
It also uses the provided `URL` constructor to better define the
database URL structure.
[1] https://docs.sqlalchemy.org/en/20/core/connections.html#basic-usage
This change installs and configures the `mod_zip` nginx module [1],
which allows nginx to stream ZIP files directly.
It includes a workaround needed to correctly calculate CRC-32 values for
included files, by including a new `server` section listening at port
8081, only used for the file requests to be upstream subrequests that
correctly trigger the CRC-32 calculation logic.
Also, to be able to provide a `m3u` file generated on the fly, we add a
`/decode` endpoint fully implemented in nginx using NJS, which receives
a `value` URL param, and decodes it using base64. The decoded value is
returned as the response.
That way, the contents of the `m3u` file is base64-encoded, and set as
part of the response, for `mod_zip` to include it in the ZIP file.
[1] https://github.com/evanmiller/mod_zip
According to multiple FastAPI discussions [1], FastAPI only includes a
built-in mechanism to redirect requests including a trailing slash, to
its variation without slash, using a `307` status code.
This can be an issue when certain clients do not send the same headers
on the redirected request.
This change adds a custom FastAPI `APIRouter`, that registers both route
path variations (with and without trailing slash), while only marking
the path without slash for being included in the OpenAPI schema.
[1] https://github.com/fastapi/fastapi/discussions/7298
Pytest v8.2 introduced the `PYTEST_VERSION` environment variable [1],
that can be used to check if code is running from within a pytest run.
This way, we can avoid checking the loaded `sys` modules.
[1] https://docs.pytest.org/en/stable/changelog.html#id57
This improvement avoids extra IGDB API requests when a received Rom is
an exact match. It avoids up to 2 requests per Rom, when an exact match
is found.
Convert `IGDBBaseHandler` methods to be asynchronous, and use an `httpx`
async client, instead of `requests` sync client.
This change also removes the direct dependency with `requests`, as the
project no longer uses it, preferring `httpx` instead.
This change mainly refactors the `scan_platforms` function, moving part
of its logic to `_identify_platform`, `_identify_firmware`, and
`_identify_rom`.
The logic is simpler this way, each smaller function returns `ScanStats`
that can be merged by the caller, and it simplifies future performance
improvements.
For filesystem resource handler, `requests` calls have been replaced
with `httpx`, and file I/O has been replaced with `anyio` utils.
The existing approach to save covers and screenshots, by calling
`shutil.copyfileobj` with the raw response is no longer needed. `httpx`
does not provide a file-like object when streaming [1], so there's no
easy drop-in replacement.
However, the applied solution correctly builds the file iteratively, by
consuming the response in chunks.
[1] https://github.com/encode/httpx/discussions/2296
Introduce an asynchronous Redis instance to be used in async functions.
Also, this change migrates most of the sync cache usage to the new async
cache.
`os.walk` is a generator that can iteratively navigate from the
specified path, top-bottom. However, most of the calls to `os.walk` in
the project cast the call to `list()`, which makes it traverse the path
and recursively find all nested directories.
This is commonly not needed, as we end up just using a `[0]` index to
only access the root path.
This change adds a few utils that simplifies listing files/directories,
and by default does it non-recursively. Performance gains shouldn't be
noticeable in systems with high-speed storage, but we can avoid the edge
cases of users having too many nested directories, by avoiding unneeded
I/O.
