D 2024-02-15 16:11:19.013304+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-02-14-12-51-02')]
I 2024-02-15 16:11:19.056267+00:00 Source: @TheRMan@vt.social (Mastodon) @therman@vt.social, https://brid.gy/mastodon/@therman@vt.social
I 2024-02-15 16:11:19.056821+00:00 Last poll: https://brid.gy/log?start_time=1707915062&key=agdicmlkLWd5ciALEghNYXN0b2RvbiISQHRoZXJtYW5AdnQuc29jaWFsDA
I 2024-02-15 16:11:19.086880+00:00 Updating @TheRMan@vt.social (Mastodon) /mastodon/@therman@vt.social : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 2, 15, 16, 11, 19, 56916, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-02-15 16:11:19.223634+00:00 no refresh_token
I 2024-02-15 16:11:19.254965+00:00 requests.get https://vt.social/api/v1/accounts/109366050795398980/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-02-15 16:11:21.546293+00:00 Redirected to https://vt.social/api/v1/accounts/109366050795398980/statuses?limit=30
I 2024-02-15 16:11:21.549177+00:00 Received 200
I 2024-02-15 16:11:21.558401+00:00 requests.get https://vt.social/api/v1/statuses/111931892585749194/reblogged_by {'headers': {'Authorization': '...'}}
I 2024-02-15 16:11:22.399086+00:00 Received 200
I 2024-02-15 16:11:22.414309+00:00 requests.get https://vt.social/api/v1/statuses/111931794735024113/reblogged_by {'headers': {'Authorization': '...'}}
I 2024-02-15 16:11:23.023450+00:00 Received 200
I 2024-02-15 16:11:23.033290+00:00 requests.get https://vt.social/api/v1/statuses/111931105622198997/reblogged_by {'headers': {'Authorization': '...'}}
I 2024-02-15 16:11:23.783245+00:00 Received 200
I 2024-02-15 16:11:23.936737+00:00 requests.get https://vt.social/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-02-15 16:11:28.711194+00:00 Redirected to https://vt.social/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-02-15 16:11:28.719329+00:00 Received 200
I 2024-02-15 16:11:28.762464+00:00 Found 49 public activities: dict_keys(['tag:vt.social,2013:111931892585749194', 'tag:vt.social,2013:111931794735024113', 'tag:vt.social,2013:111931105622198997', 'tag:vt.social,2013:111929489577409592', 'tag:vt.social,2013:111929485367217610', 'tag:vt.social,2013:111925224776249005', 'tag:vt.social,2013:111912591904562057', 'tag:vt.social,2013:111912012867526895', 'tag:vt.social,2013:111908054005540157', 'tag:vt.social,2013:111907268423267381', 'tag:vt.social,2013:111906835758742095', 'tag:vt.social,2013:111899702541366845', 'tag:vt.social,2013:111897489249914522', 'tag:vt.social,2013:111896568425232642', 'tag:vt.social,2013:111891368328767346', 'tag:vt.social,2013:111890662518778611', 'tag:vt.social,2013:111889853468492785', 'tag:vt.social,2013:111887359206966154', 'tag:vt.social,2013:111885165218104264', 'tag:vt.social,2013:111884186535138689', 'tag:vt.social,2013:111870527959647059', 'tag:vt.social,2013:111868819442945415', 'tag:vt.social,2013:111867199926331578', 'tag:vt.social,2013:111866948498201787', 'tag:vt.social,2013:111866773820884305', 'tag:vt.social,2013:111863681910146353', 'tag:vt.social,2013:111862901585062297', 'tag:vt.social,2013:111861537275944163', 'tag:vt.social,2013:111858253283010425', 'tag:vt.social,2013:111857255742649490', 'tag:vt.social,2013:111814325923492219', 'tag:vt.social,2013:111737461799973987', 'tag:vt.social,2013:111708142444406944', 'tag:vt.social,2013:111699214539523767', 'tag:vt.social,2013:111679802762744219', 'tag:vt.social,2013:111662804857751110', 'tag:vt.social,2013:111640234537422185', 'tag:vt.social,2013:111544724916804624', 'tag:vt.social,2013:111529103275061955', 'tag:vt.social,2013:111529098590469295', 'tag:vt.social,2013:111439398866564367', 'tag:vt.social,2013:111394267241907078', 'tag:vt.social,2013:111381986231845778', 'tag:vt.social,2013:111377704821730803', 'tag:vt.social,2013:111376391133579415', 'tag:vt.social,2013:111372890201593514', 'tag:vt.social,2013:111308202484104338', 'tag:vt.social,2013:111277295031419373', 'tag:vt.social,2013:111277217699277656'])
I 2024-02-15 16:11:28.763505+00:00 Found 1 private activities: dict_keys(['tag:vt.social,2013:111477070823019571'])
D 2024-02-15 16:11:28.770259+00:00 discovering original posts for: https://vt.social/@TheRMan/111931892585749194
I 2024-02-15 16:11:28.770818+00:00 requests.head https://twitch.tv/videos/2062858180 {'headers': {}}
I 2024-02-15 16:11:29.423340+00:00 Redirected to https://www.twitch.tv/videos/2062858180
I 2024-02-15 16:11:29.423753+00:00 Received 200
D 2024-02-15 16:11:29.424026+00:00 Resolved https://twitch.tv/videos/2062858180 to https://www.twitch.tv/videos/2062858180
I 2024-02-15 16:11:29.424361+00:00 requests.head https://twitch.tv/videos/2062863235 {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:29.883333+00:00 Redirected to https://www.twitch.tv/videos/2062863235
I 2024-02-15 16:11:29.883673+00:00 Received 200
D 2024-02-15 16:11:29.883770+00:00 Resolved https://twitch.tv/videos/2062863235 to https://www.twitch.tv/videos/2062863235
I 2024-02-15 16:11:29.883953+00:00 requests.head https://twitch.tv/videos/2062867005 {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:30.340325+00:00 Redirected to https://www.twitch.tv/videos/2062867005
I 2024-02-15 16:11:30.340658+00:00 Received 200
D 2024-02-15 16:11:30.341109+00:00 Resolved https://twitch.tv/videos/2062867005 to https://www.twitch.tv/videos/2062867005
I 2024-02-15 16:11:30.342084+00:00 requests.head https://twitch.tv/videos/2062869454 {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:30.696490+00:00 Redirected to https://www.twitch.tv/videos/2062869454
I 2024-02-15 16:11:30.696833+00:00 Received 200
D 2024-02-15 16:11:30.696947+00:00 Resolved https://twitch.tv/videos/2062869454 to https://www.twitch.tv/videos/2062869454
I 2024-02-15 16:11:30.697245+00:00 Original post discovery found original posts set(), mentions {'https://www.twitch.tv/videos/2062863235', 'https://www.twitch.tv/videos/2062869454', 'https://www.twitch.tv/videos/2062867005', 'https://www.twitch.tv/videos/2062858180'}
I 2024-02-15 16:11:30.697786+00:00 requests.head https://vt.social/@TheRMan/111931892585749194 {'headers': None}
I 2024-02-15 16:11:31.442534+00:00 Received 200
I 2024-02-15 16:11:31.443468+00:00 starting posse post discovery with syndicated https://vt.social/@TheRMan/111931892585749194
I 2024-02-15 16:11:31.503306+00:00 requests.head https://www.tiktok.com/@therman98 {'headers': {}}
I 2024-02-15 16:11:31.709694+00:00 Received 403
W 2024-02-15 16:11:31.710077+00:00 Couldn't resolve URL https://www.tiktok.com/@therman98: https://www.tiktok.com/@therman98
I 2024-02-15 16:11:31.710514+00:00 requests.head https://www.fiverr.com/s/p6LNxR {'headers': {}}
I 2024-02-15 16:11:32.414639+00:00 Redirected to https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
I 2024-02-15 16:11:32.443478+00:00 Received 403
W 2024-02-15 16:11:32.443757+00:00 Couldn't resolve URL https://www.fiverr.com/s/p6LNxR: https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
D 2024-02-15 16:11:32.445044+00:00 fetching author url https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
I 2024-02-15 16:11:32.448569+00:00 requests.get https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf {'gateway': False, 'headers': {}}
I 2024-02-15 16:11:32.789853+00:00 Redirected to https://block.fiverr.com/?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
I 2024-02-15 16:11:32.792833+00:00 Received 403
I 2024-02-15 16:11:32.793272+00:00 Could not fetch author url https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2229, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://block.fiverr.com/?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=e226a839-cc1c-11ee-babf-1bd0c5a2dbaf
I 2024-02-15 16:11:32.798046+00:00 requests.head https://trello.com/b/gmVCR4qD/editing-commissions {'headers': {}}
I 2024-02-15 16:11:33.160825+00:00 Received 200
D 2024-02-15 16:11:33.162291+00:00 fetching author url https://trello.com/b/gmVCR4qD/editing-commissions
I 2024-02-15 16:11:33.162467+00:00 requests.get https://trello.com/b/gmVCR4qD/editing-commissions {'gateway': False, 'headers': {}}
I 2024-02-15 16:11:33.558751+00:00 Received 200
D 2024-02-15 16:11:33.565530+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-15 16:11:33.566354+00:00 requests.head https://twittodon.com/share.php?t=therman98&amp%3Bm=therman%40vt.social {'headers': {}}
I 2024-02-15 16:11:34.084954+00:00 Redirected to https://twittodon.com
I 2024-02-15 16:11:34.085321+00:00 Received 200
D 2024-02-15 16:11:34.085447+00:00 Resolved https://twittodon.com/share.php?t=therman98&amp%3Bm=therman%40vt.social to https://twittodon.com
D 2024-02-15 16:11:34.088980+00:00 fetching author url https://twittodon.com
I 2024-02-15 16:11:34.089195+00:00 requests.get https://twittodon.com {'gateway': False, 'headers': {}}
I 2024-02-15 16:11:34.580680+00:00 Redirected to https://twittodon.com/
I 2024-02-15 16:11:34.581329+00:00 Received 200
D 2024-02-15 16:11:34.587028+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-15 16:11:34.587568+00:00 requests.head https://therman.eu/ {'headers': {}}
I 2024-02-15 16:11:35.176405+00:00 Received 200
D 2024-02-15 16:11:35.177509+00:00 fetching author url https://therman.eu/
I 2024-02-15 16:11:35.177659+00:00 requests.get https://therman.eu/ {'gateway': False, 'headers': {}}
I 2024-02-15 16:11:35.404877+00:00 Received 200
D 2024-02-15 16:11:35.558061+00:00 No h-feed found, fallback to top-level h-entrys.
D 2024-02-15 16:11:35.558791+00:00 posse post discovery found no relationship for https://vt.social/@TheRMan/111931892585749194
D 2024-02-15 16:11:35.878852+00:00 discovering original posts for: https://vt.social/@TheRMan/111931794735024113
I 2024-02-15 16:11:35.879201+00:00 Original post discovery found original posts set(), mentions set()
I 2024-02-15 16:11:35.879532+00:00 requests.head https://vt.social/@TheRMan/111931794735024113 {'headers': None}
I 2024-02-15 16:11:36.550496+00:00 Received 200
I 2024-02-15 16:11:36.550837+00:00 starting posse post discovery with syndicated https://vt.social/@TheRMan/111931794735024113
D 2024-02-15 16:11:36.657861+00:00 skipping https://www.tiktok.com/@therman98, already fetched this round
D 2024-02-15 16:11:36.658106+00:00 skipping https://www.fiverr.com/s/p6LNxR, already fetched this round
D 2024-02-15 16:11:36.658195+00:00 skipping https://trello.com/b/gmVCR4qD/editing-commissions, already fetched this round
D 2024-02-15 16:11:36.658293+00:00 skipping https://twittodon.com/share.php?t=therman98&amp%3bm=therman%40vt.social, already fetched this round
D 2024-02-15 16:11:36.658379+00:00 skipping https://therman.eu/, already fetched this round
D 2024-02-15 16:11:36.658457+00:00 posse post discovery found no relationship for https://vt.social/@TheRMan/111931794735024113
D 2024-02-15 16:11:36.940569+00:00 discovering original posts for: https://vt.social/@TheRMan/111931105622198997
W 2024-02-15 16:11:36.941401+00:00 Found 11 original post candidates, only resolving redirects for the first 10
I 2024-02-15 16:11:36.941566+00:00 requests.head https://twitch.tv/therman {'headers': {}}
I 2024-02-15 16:11:37.701886+00:00 Redirected to https://www.twitch.tv/therman
I 2024-02-15 16:11:37.702575+00:00 Received 200
D 2024-02-15 16:11:37.702695+00:00 Resolved https://twitch.tv/therman to https://www.twitch.tv/therman
I 2024-02-15 16:11:37.703122+00:00 requests.head https://vt.social/tags/ENVtuber {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:38.549778+00:00 Received 200
I 2024-02-15 16:11:38.550252+00:00 requests.head https://vt.social/tags/Twitch {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:39.383919+00:00 Received 200
I 2024-02-15 16:11:39.384348+00:00 requests.head https://vt.social/tags/VTuber {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:40.343773+00:00 Received 200
I 2024-02-15 16:11:40.344191+00:00 requests.head https://vt.social/tags/VTubers {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:41.254405+00:00 Received 200
I 2024-02-15 16:11:41.254757+00:00 requests.head https://vt.social/tags/TheRMan {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:42.448958+00:00 Received 200
I 2024-02-15 16:11:42.449310+00:00 requests.head https://vt.social/tags/Ronika {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:43.561108+00:00 Received 200
I 2024-02-15 16:11:43.561456+00:00 requests.head https://vt.social/tags/TheRLive {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:44.535429+00:00 Received 200
I 2024-02-15 16:11:44.535828+00:00 requests.head https://vt.social/tags/MembersRF {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:45.326150+00:00 Received 200
I 2024-02-15 16:11:45.326548+00:00 requests.head https://vt.social/tags/VirtualLifeDE {'headers': {'User-Agent': '...'}}
I 2024-02-15 16:11:46.347195+00:00 Received 200
I 2024-02-15 16:11:46.348284+00:00 Original post discovery found original posts set(), mentions {'https://vt.social/tags/MembersRF', 'https://vt.social/tags/VTubers', 'https://vt.social/tags/VirtualLifeDE', 'https://vt.social/tags/TheRMan', 'https://vt.social/tags/VTuber', 'https://vt.social/tags/Twitch', 'https://vt.social/tags/Ronika', 'https://vt.social/tags/TheBearsDen', 'https://www.twitch.tv/therman', 'https://vt.social/tags/ENVtuber', 'https://vt.social/tags/TheRLive'}
I 2024-02-15 16:11:46.348927+00:00 11 mentions, pruning down to 10
I 2024-02-15 16:11:46.352500+00:00 requests.head https://vt.social/tags/TheBearsDen {'headers': {}}
I 2024-02-15 16:11:47.600463+00:00 Received 200
I 2024-02-15 16:11:47.609645+00:00 requests.head https://vt.social/@TheRMan/111931105622198997 {'headers': None}
I 2024-02-15 16:11:48.663567+00:00 Received 200
I 2024-02-15 16:11:48.663948+00:00 starting posse post discovery with syndicated https://vt.social/@TheRMan/111931105622198997
D 2024-02-15 16:11:48.738487+00:00 skipping https://www.tiktok.com/@therman98, already fetched this round
D 2024-02-15 16:11:48.739238+00:00 skipping https://www.fiverr.com/s/p6LNxR, already fetched this round
D 2024-02-15 16:11:48.739441+00:00 skipping https://trello.com/b/gmVCR4qD/editing-commissions, already fetched this round
D 2024-02-15 16:11:48.739572+00:00 skipping https://twittodon.com/share.php?t=therman98&amp%3bm=therman%40vt.social, already fetched this round
D 2024-02-15 16:11:48.739680+00:00 skipping https://therman.eu/, already fetched this round
D 2024-02-15 16:11:48.739799+00:00 posse post discovery found no relationship for https://vt.social/@TheRMan/111931105622198997
I 2024-02-15 16:11:49.216036+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 1970-01-01 00:00:00+00:00
I 2024-02-15 16:11:49.314679+00:00 Updating @TheRMan@vt.social (Mastodon) /mastodon/@therman@vt.social : {'last_public_post': datetime.datetime(2024, 2, 14, 21, 12, 18, 333000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 2, 15, 16, 11, 19, 56916, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-02-15 16:11:49.440324+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/41926198960776734101 with ETA 1708095773: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ciALEghNYXN0b2RvbiISQHRoZXJtYW5AdnQuc29jaWFsDA&last_polled=2024-02-15-16-11-19', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1708095773
}