This change mainly refactors the `scan_platforms` function, moving part
of its logic to `_identify_platform`, `_identify_firmware`, and
`_identify_rom`.
The logic is simpler this way, each smaller function returns `ScanStats`
that can be merged by the caller, and it simplifies future performance
improvements.
For filesystem resource handler, `requests` calls have been replaced
with `httpx`, and file I/O has been replaced with `anyio` utils.
The existing approach to save covers and screenshots, by calling
`shutil.copyfileobj` with the raw response is no longer needed. `httpx`
does not provide a file-like object when streaming [1], so there's no
easy drop-in replacement.
However, the applied solution correctly builds the file iteratively, by
consuming the response in chunks.
[1] https://github.com/encode/httpx/discussions/2296
Creates an example config file for Batocera/RetroBat-based mappings, as popular distributions so should hopefully save people a few hours.
Tested with a setup using RetroBat v6.0.3 folder structures, and only keeping matches for which I'm certain.
Introduce an asynchronous Redis instance to be used in async functions.
Also, this change migrates most of the sync cache usage to the new async
cache.
With the introduction of non-paginated responses for ROMs, JSON
responses for big collection could easily be at >1MiB.
This change adds a FastAPI-provided middleware to enable GZip
compression for responses greater than 1KiB.
`os.walk` is a generator that can iteratively navigate from the
specified path, top-bottom. However, most of the calls to `os.walk` in
the project cast the call to `list()`, which makes it traverse the path
and recursively find all nested directories.
This is commonly not needed, as we end up just using a `[0]` index to
only access the root path.
This change adds a few utils that simplifies listing files/directories,
and by default does it non-recursively. Performance gains shouldn't be
noticeable in systems with high-speed storage, but we can avoid the edge
cases of users having too many nested directories, by avoiding unneeded
I/O.
This change avoids blocking requests when retrieving covers from
SteamGridDB, which is the main bottleneck as the current implementation
iterates over paginated results for multiple games.
Using an asynchronous client like `httpx` provides a good performance
improvement, and reduces the latency when calling this endpoint.
Also, the inclusion of FastAPI `lifespan` allows instantiating a single
client on startup.
When testing with "Final Fantasy V Advance", the endpoint goes from ~9s
to ~1.5s to retrieve all covers.
