YanoPettanko/cleanmedia
1
0
Fork 0
mirror of https://gitlab.com/sspaeth/cleanmedia.git synced 2025-04-03 18:30:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
61 commits 2 branches 3 tags 79 KiB
Commit graph

61 commits

This branch
This branch
All branches
Author SHA1 Message Date
Sebastian Spaeth
7a4d497353 Optimize Sanity Check SQL 
Instead of
SELECT COUNT(media_id) from mediaapi_thumbnail WHERE media_id NOT IN (SELECT media_id FROM mediaapi_media_repository);
we do
SELECT COUNT(media_id) from mediaapi_thumbnail WHERE NOT EXISTS (SELECT media_id FROM mediaapi_media_repository);

Per Till's (@s7evink:matrix.org) suggestion. All credit to him
 2024-03-12 18:02:44 +01:00
Sebastian Spaeth
0a03429abf Maintainership notice 2024-02-01 06:23:36 +00:00
Sebastian Spaeth
4ddec53928 Merge branch 'deluser' into 'main' 
Add deletion of local user media

See merge request sspaeth/cleanmedia!8
 2023-12-13 12:45:06 +00:00
Sebastian Spaeth
d6fd2f97e3 Add deletion of local user media 
Unfortunately, the database only stores the orginating server, and not the specific user id
for remote media, so we can only delete media from our users with the new
-u option. For more, we need to change the database, or dig much deeper. Also,
this will not work for E2E encrypted chats, I think.
 2023-12-13 13:42:18 +01:00
Sebastian Spaeth
b3d857e245 Delete stray debug leftover 2023-12-11 15:03:15 +01:00
Sebastian Spaeth
320af39f35 only log when we actually find a corresponding media 2023-12-11 14:40:04 +01:00
Sebastian Spaeth
3b8751c6de hotfix: Print all arguments in logging string 2023-12-11 14:37:39 +01:00
Sebastian Spaeth
f3ef851c74 Merge branch 'singlemedia' into 'main' 
add deletion of single media

See merge request sspaeth/cleanmedia!7
 2023-12-11 13:34:29 +00:00
Sebastian Spaeth
5dd80a77b0 add deletion of single media 2023-12-11 14:16:07 +01:00
Sebastian Spaeth
6dc86c9ac1 Merge branch 'refactor' into 'main' 
refactor main function

See merge request sspaeth/cleanmedia!6
 2023-12-11 12:32:55 +00:00
Sebastian Spaeth
f297f20394 refactor main function 2023-12-11 12:32:55 +00:00
Sebastian Spaeth
3e12ef1b1a More robustness against flawed config files 
1) Don't bail out with an unset CONN_STR if we don't have
   ANY connection_string set in our config file.
2) No need to guard against missing section [media_api], we
   verify its existance a few lines above.
 2023-12-04 08:57:03 +01:00
Sebastian Spaeth
cf1eb36aa6 Merge branch 'neetzsche/fix-media-api-config' into 'main' 
Fix getting connection_string from media_api

See merge request sspaeth/cleanmedia!4
 2023-12-04 07:38:37 +00:00
NEETzsche
9f36cfc8b3 Fix getting connection_string from media_api 
We were looking for [media_api][connection_string] and not for [media_api][database][connection_string]
Credits to https://gitlab.com/NEETzsche
 2023-12-04 07:38:37 +00:00
Sebastian Spaeth
a35d3ad5fb Merge branch 'neetzsche/filter-blank-avatar-urls' into 'main' 
Filter blank avatar urls

See merge request sspaeth/cleanmedia!5
 2023-12-04 07:32:32 +00:00
NEETzsche
0ceefedceb Filter blank avatar urls 
credit to https://gitlab.com/NEETzsche
 2023-12-04 07:32:32 +00:00
Sebastian Spaeth
2f2eee20d6 Add --quiet mode 
Add a mode that decreases the log level to WARNING. In case the
conflicting options -d and -q are given simultaneously, -d will win.
 2023-11-15 10:13:49 +01:00
Sebastian Spaeth
7b62b49df4 README: Update readme 
Now that we can also purge local media (optional!), we need to adapt the README.
 2023-09-18 14:01:38 +02:00
Sebastian Spaeth
c1ee679f9e Merge branch 'save_avatars' into 'main' 
Also allow to delete local media

Closes #1

See merge request sspaeth/cleanmedia!1
 2023-09-18 11:53:15 +00:00
Sebastian Spaeth
b26435d4cc cleanmedia: break up long line 2023-09-18 13:52:13 +02:00
Sebastian Spaeth
055d9b4202 cleanmedia: Allow to also delete local media files 
Using the --local option we can also delete local media.
We check which media files are used for avatar images
and never purge those though.

Local media means here, media that have been upload by
users from our home server.

Fixes: #1
 2023-09-18 13:50:59 +02:00
Sebastian Spaeth
1cb7dad3c2 Merge branch 'mypy' into 'main' 
CI: Add mypy check

See merge request sspaeth/cleanmedia!3
 2023-09-18 11:45:15 +00:00
Sebastian Spaeth
b01eb97a47 CI: flake8 the file 2023-09-18 13:44:09 +02:00
Sebastian Spaeth
9eea97f1e1 CI: Make mypy happy 
cleanmedia:133: error: Value of type "tuple[Any, ...] | None" is not indexable  [index]
So we do sanity check that row is not None now before doing something with row[0].
 2023-09-18 13:39:13 +02:00
Sebastian Spaeth
7d6dc981e3 CI: Fix cleanmedia:107: error: Name "psycopg2.extensions.connection" is not defined [name-defined] 2023-09-18 13:35:41 +02:00
Sebastian Spaeth
3e063e92d4 CI: Add mypy check 2023-09-18 13:30:47 +02:00
Sebastian Spaeth
edae37dc06 CI: Don't keep artifacts around 
The script is available from the repo, no need for an artifact.
 2023-09-18 13:13:46 +02:00
Sebastian Spaeth
7c60a9037c flake8: This is it. Green! 2023-09-18 13:11:12 +02:00
Sebastian Spaeth
329981fe49 flake8: Is this the end? 2023-09-18 13:10:08 +02:00
Sebastian Spaeth
65dff450c3 CI: Make flake8 less grumpy 2023-09-18 13:06:12 +02:00
Sebastian Spaeth
0fcf538a93 flake8: make pylinter a little more happy 2023-09-18 12:59:27 +02:00
Sebastian Spaeth
4cd9bc6136 Merge branch 'CI' into 'main' 
Add CI

See merge request sspaeth/cleanmedia!2
 2023-09-18 10:56:02 +00:00
Sebastian Spaeth
d222eba15a CI: add one 2023-09-18 12:55:00 +02:00
Sebastian Spaeth
ad184116fc cleanmedia: fix whitespace 
too much!
 2023-09-18 10:18:21 +02:00
Sebastian Spaeth
13b7ffcc5e Add requirements 2023-05-17 12:17:16 +00:00
Sebastian Spaeth
4984a5d1a4 Update README 
One todo point down, so we need to update the README accordingly.
 2023-04-20 09:55:54 +02:00
Sebastian Spaeth
55652662ca Fix deleted file calculation 
We stupidly just output the total number of files, and not the number
of files that we attempted to delete, which leads to flawed statistics
output.
 2023-04-20 09:53:41 +02:00
Sebastian Spaeth
e4a190749f Merge branch 'master' of gitlab.com:sspaeth/cleanmedia 2023-04-20 09:46:28 +02:00
Sebastian Spaeth
1f3bc1e0c9 Add sanity check for thumbnails 
Warn in case we have thumbnails in our db that do not refer to an
existing media entry in our db.
 2023-04-20 09:45:20 +02:00
Sebastian Spaeth
405c207f8c Clean up code comments a little 2023-04-19 18:29:50 +02:00
Sebastian Spaeth
1df1a8e74c use correct class for typing hints 2023-04-19 18:25:02 +02:00
Sebastian Spaeth
94a643d636 mypy: Make the typing checks pass without error 2023-04-19 18:19:50 +02:00
Sebastian Spaeth
52484e7351 remove todo list from source code 2023-04-19 17:52:37 +02:00
Sebastian Spaeth
4fad9fcb0d allow postgresql:// connection string 
Due to popular demand ;-), also allow postgresl://
 2023-02-02 07:42:16 +01:00
Sebastian Spaeth
6f99a37944 Add scary warranty disclaimer 2023-01-12 14:59:54 +01:00
Sebastian Spaeth
7b6ecf25ac reintroduce correct failure handling 2023-01-12 14:53:32 +01:00
Sebastian Spaeth
3914546c92 simplify file counting 2023-01-12 14:49:39 +01:00
Sebastian Spaeth
d23e16cbee Count deleted files 
and report a summary statistic.
 2023-01-12 14:47:20 +01:00
Sebastian Spaeth
4887745611 expand Readme somewhat 2022-11-30 10:33:24 +01:00
Sebastian Spaeth
430809c49b Update README 
Make it even betta!
 2022-11-28 15:37:11 +01:00