Commit Graph

193 Commits

Author SHA1 Message Date
Georges-Antoine Assi
3a9cef24e0 get it all wokring 2024-07-27 13:30:52 -04:00
Georges-Antoine Assi
ecc021e824 Merge branch 'master' into calc-file-hashes-roms 2024-07-27 12:17:18 -04:00
Michael Manganiello
96699a2f1b fix: Catch ProtocolError when writing screenshots/covers 2024-07-26 21:19:18 -03:00
Michael Manganiello
aced1f1ff2 misc: Migrate IGDBBaseHandler to async
Convert `IGDBBaseHandler` methods to be asynchronous, and use an `httpx`
async client, instead of `requests` sync client.

This change also removes the direct dependency with `requests`, as the
project no longer uses it, preferring `httpx` instead.
2024-07-26 21:19:01 -03:00
Georges-Antoine Assi
efc0b75cdb Merge branch 'master' into calc-file-hashes-roms 2024-07-25 22:42:46 -04:00
Michael Manganiello
959e896aa3 misc: Replace pytest-vcr dependency with pytest-recording
`pytest-vcr` [1] is no longer maintained (last commit was in 2020), and
`VCR.py` recommends [2] `pytest-recording` instead.

Also, `pytest-vcr` does not correctly support `httpx` [3].

This change also removes unused cassettes.

[1] https://github.com/ktosiek/pytest-vcr
[2] https://vcrpy.readthedocs.io/en/latest/usage.html#pytest-integration
[3] https://github.com/ktosiek/pytest-vcr/issues/46
2024-07-24 12:39:41 -03:00
Michael Manganiello
56066d6765 misc: Migrate filesystem resource handler to async
For filesystem resource handler, `requests` calls have been replaced
with `httpx`, and file I/O has been replaced with `anyio` utils.

The existing approach to save covers and screenshots, by calling
`shutil.copyfileobj` with the raw response is no longer needed. `httpx`
does not provide a file-like object when streaming [1], so there's no
easy drop-in replacement.

However, the applied solution correctly builds the file iteratively, by
consuming the response in chunks.

[1] https://github.com/encode/httpx/discussions/2296
2024-07-24 00:18:17 -03:00
Georges-Antoine Assi
db1787dff4 fix trunk issues 2024-07-18 22:03:08 -04:00
Georges-Antoine Assi
77afe55625 add support for extracting content in compressed files 2024-07-18 21:50:24 -04:00
Georges-Antoine Assi
591e41a4d5 start checking for specific file exts 2024-07-18 20:51:48 -04:00
Georges-Antoine Assi
2e91c440e3 Read into mem in chunks 2024-07-16 17:42:25 -04:00
Georges-Antoine Assi
2b2ff875ee Calculate hashes for rom files 2024-07-15 18:27:29 -04:00
Georges-Antoine Assi
d05a539871 Fix covers deleted when uploading extension other then PNG 2024-07-14 17:30:05 -04:00
Michael Manganiello
f20a9ffe34 fix: Avoid recursive os.walk calls
`os.walk` is a generator that can iteratively navigate from the
specified path, top-bottom. However, most of the calls to `os.walk` in
the project cast the call to `list()`, which makes it traverse the path
and recursively find all nested directories.

This is commonly not needed, as we end up just using a `[0]` index to
only access the root path.

This change adds a few utils that simplifies listing files/directories,
and by default does it non-recursively. Performance gains shouldn't be
noticeable in systems with high-speed storage, but we can avoid the edge
cases of users having too many nested directories, by avoiding unneeded
I/O.
2024-07-13 15:30:04 -03:00
zurdi
884e7a35ac fixed re-download lost covers 2024-07-11 08:47:59 +02:00
zurdi
59c2a51a6f added collection cover management 2024-07-03 14:31:17 +02:00
zurdi
7dfbcc9a0b dinamic ratio for small artwork 2024-06-26 19:37:50 +02:00
Michael Manganiello
fc53d77a58 misc: Compile constant regular expressions
Improve efficiency on reusable regular expressions, by compiling them.
2024-06-23 00:36:22 -03:00
zurdi
8de804ff27 redesigned select source on match rom dialog 2024-06-20 13:41:46 +02:00
zurdi
8386625f7f bigger small size cover 2024-06-20 11:36:13 +02:00
Georges-Antoine Assi
057fef7ce9 better error handling when image fetch 503 2024-06-19 23:00:56 -04:00
Georges-Antoine Assi
6e049c1012 can accept nulls 2024-06-19 10:19:00 -04:00
Georges-Antoine Assi
e2c5ba68cb backend tweaks + refactor 2024-06-18 23:21:17 -04:00
zurdi
d02d416f40 generate small version of custom cover 2024-06-18 14:20:49 +02:00
zurdi
72b63322e4 fixed resources migration 2024-06-18 11:46:36 +02:00
zurdi
84e442c0c5 cover path platform migrated to id 2024-06-18 01:24:19 +02:00
zurdi
851f354e9e fixed from trunk check 2024-06-17 19:11:12 +02:00
zurdi
ed67cfff12 fixed unit tests 2024-06-17 18:05:48 +02:00
Zurdi
242dbb9da5 cover path migration to id 2024-06-17 13:45:35 +02:00
Zurdi
b5383d3cb2 fixed getting 1080p cover from manual search 2024-06-16 18:35:37 +02:00
Georges-Antoine Assi
652047ad66 update test cassettes 2024-06-15 12:18:01 -04:00
Georges-Antoine Assi
8ab3aba614 fix trunk issues 2024-06-15 11:43:33 -04:00
Zurdi
cf38949a65 fetch 1080p cover and huge screenshots 2024-06-15 16:58:13 +02:00
Zurdi
408f9d2e1d improved config file unit test handling 2024-06-13 23:07:57 +02:00
Georges-Antoine Assi
1840390c8a finish mypy fixes 2024-05-21 21:28:17 -04:00
Georges-Antoine Assi
b2085f87a8 bunch of fixes for trunk 2024-05-21 17:10:11 -04:00
Georges-Antoine Assi
a7cf0d389a run trunk format on all files 2024-05-21 10:18:13 -04:00
Georges-Antoine Assi
880345f20c ruff-only pre-commit 2024-05-18 12:05:57 -04:00
Georges-Antoine Assi
3e42f3ab56 also ignore firmware files on scan that match 2024-05-16 23:12:27 -04:00
Georges-Antoine Assi
59575416e8 changes from self-review 2024-05-05 17:08:23 -04:00
Georges-Antoine Assi
2749aaee4a rename back to old names 2024-05-05 16:59:30 -04:00
Georges-Antoine Assi
dc33054ba1 more name refactoring 2024-05-05 16:45:58 -04:00
Georges-Antoine Assi
9ad28f3294 more refactoring again 2024-05-05 16:32:40 -04:00