Commit Graph

19 Commits

Author SHA1 Message Date
66bef9279f Merge branch 'scraper' into scraper-dev 2025-10-09 20:07:14 +02:00
1dc17d7670 Don't create py34.Post for downloading images
py34.Post does extension scan, that is not required as it is saved
inside the block data.
2025-10-09 20:05:36 +02:00
5cedbb2526 Merge branch 'scraper' into scraper-dev 2025-10-09 19:18:37 +02:00
a4e3ced8ab Download videos too, prioritize video over image 2025-10-09 19:18:26 +02:00
bf9237750b Merge branch 'scraper' into scraper-dev 2025-10-03 00:09:36 +02:00
ee4ba5d5f2 Create file after the image was retrieved
`post.get_image_data()` can throw, creating file and later calling the function can create empty "image" files, confusing the program.
2025-10-02 23:40:33 +02:00
a3dca99a5d Merge branch 'scraper' into scraper-dev 2025-09-23 12:44:52 +02:00
217595fcdd Download and cache images 2025-09-23 12:44:36 +02:00
bb7d010694 Merge branch 'scraper' into scraper-dev 2025-09-23 02:27:13 +02:00
be6740a24c Added api route for getting raw block 2025-09-23 02:24:57 +02:00
ee236445c2 Made single threaded, added back delay 2025-08-28 21:38:32 +02:00
bdf77b2920 Merge remote-tracking branch 'refs/remotes/origin/scraper' into scraper 2025-08-15 18:48:23 +02:00
5f4358a3d1 Probe with HEAD requests, not GET 2025-08-15 18:47:58 +02:00
8e3a7b105a Fixed invalid extension check 2025-08-10 15:50:40 +02:00
d0dec584a8 Removed debug print 2025-08-10 15:49:03 +02:00
c1b8be46aa Initial scraper commit 2025-08-09 17:15:20 +02:00
c64c4225ff Implemented everything, removed scraper 2025-08-01 04:52:27 +02:00
42d31a68c4 Fixed imports, tags 2025-07-30 11:02:52 +02:00
46f7d3a27d Initial commit 2025-07-26 22:52:02 +02:00