D 2024-02-04 08:41:52.011352+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-02-03-07-12-31')]
I 2024-02-04 08:41:52.035799+00:00 Source: @TheRMan@vt.social (Mastodon) @therman@vt.social, https://brid.gy/mastodon/@therman@vt.social
I 2024-02-04 08:41:52.036305+00:00 Last poll: https://brid.gy/log?start_time=1706944351&key=agdicmlkLWd5ciALEghNYXN0b2RvbiISQHRoZXJtYW5AdnQuc29jaWFsDA
I 2024-02-04 08:41:52.059294+00:00 Updating @TheRMan@vt.social (Mastodon) /mastodon/@therman@vt.social : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 2, 4, 8, 41, 52, 36442, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-02-04 08:41:52.169705+00:00 no refresh_token
I 2024-02-04 08:41:52.194301+00:00 requests.get https://vt.social/api/v1/accounts/109366050795398980/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-02-04 08:41:54.614186+00:00 Redirected to https://vt.social/api/v1/accounts/109366050795398980/statuses?limit=30
I 2024-02-04 08:41:54.618634+00:00 Received 200
I 2024-02-04 08:41:54.627463+00:00 requests.get https://vt.social/api/v1/statuses/111867199926331578/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-04 08:41:55.279656+00:00 Received 200
I 2024-02-04 08:41:55.388384+00:00 requests.get https://vt.social/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-02-04 08:41:59.563858+00:00 Redirected to https://vt.social/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-02-04 08:41:59.568075+00:00 Received 200
I 2024-02-04 08:41:59.601999+00:00 Found 50 public activities: dict_keys(['tag:vt.social,2013:111870527959647059', 'tag:vt.social,2013:111868819442945415', 'tag:vt.social,2013:111867199926331578', 'tag:vt.social,2013:111866948498201787', 'tag:vt.social,2013:111866773820884305', 'tag:vt.social,2013:111863681910146353', 'tag:vt.social,2013:111862901585062297', 'tag:vt.social,2013:111861537275944163', 'tag:vt.social,2013:111858253283010425', 'tag:vt.social,2013:111857255742649490', 'tag:vt.social,2013:111855874979225821', 'tag:vt.social,2013:111855636903948633', 'tag:vt.social,2013:111855407921983452', 'tag:vt.social,2013:111850217496807320', 'tag:vt.social,2013:111828932518657952', 'tag:vt.social,2013:111827563441634320', 'tag:vt.social,2013:111820684779527960', 'tag:vt.social,2013:111810581293803798', 'tag:vt.social,2013:111809055307148454', 'tag:vt.social,2013:111799284305742291', 'tag:vt.social,2013:111798359396682480', 'tag:vt.social,2013:111796014348741423', 'tag:vt.social,2013:111796013927748045', 'tag:vt.social,2013:111795534529336622', 'tag:vt.social,2013:111795504533159973', 'tag:vt.social,2013:111793080308630989', 'tag:vt.social,2013:111789342445305413', 'tag:vt.social,2013:111788376866463332', 'tag:vt.social,2013:111787053153696312', 'tag:vt.social,2013:111770944252909425', 'tag:vt.social,2013:111814325923492219', 'tag:vt.social,2013:111737461799973987', 'tag:vt.social,2013:111708142444406944', 'tag:vt.social,2013:111699214539523767', 'tag:vt.social,2013:111679802762744219', 'tag:vt.social,2013:111662804857751110', 'tag:vt.social,2013:111640234537422185', 'tag:vt.social,2013:111544724916804624', 'tag:vt.social,2013:111529103275061955', 'tag:vt.social,2013:111529098590469295', 'tag:vt.social,2013:111439398866564367', 'tag:vt.social,2013:111394267241907078', 'tag:vt.social,2013:111381986231845778', 'tag:vt.social,2013:111377704821730803', 'tag:vt.social,2013:111376391133579415', 'tag:vt.social,2013:111372890201593514', 'tag:vt.social,2013:111308202484104338', 'tag:vt.social,2013:111277295031419373', 'tag:vt.social,2013:111277217699277656', 'tag:vt.social,2013:111238805498988072'])
I 2024-02-04 08:41:59.602250+00:00 Found 1 private activities: dict_keys(['tag:vt.social,2013:111477070823019571'])
D 2024-02-04 08:41:59.676051+00:00 discovering original posts for: https://vt.social/@TheRMan/111867199926331578
W 2024-02-04 08:41:59.679300+00:00 Found 12 original post candidates, only resolving redirects for the first 10
I 2024-02-04 08:41:59.680282+00:00 requests.head https://twitch.tv/therman {'headers': {}}
I 2024-02-04 08:42:00.163429+00:00 Redirected to https://www.twitch.tv/therman
I 2024-02-04 08:42:00.163841+00:00 Received 200
D 2024-02-04 08:42:00.163951+00:00 Resolved https://twitch.tv/therman to https://www.twitch.tv/therman
I 2024-02-04 08:42:00.164162+00:00 requests.head https://vt.social/tags/ENVtuber {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:00.738466+00:00 Received 200
I 2024-02-04 08:42:00.738774+00:00 requests.head https://vt.social/tags/GERVTuber {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:01.342369+00:00 Received 200
I 2024-02-04 08:42:01.342750+00:00 requests.head https://vt.social/tags/Twitch {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:02.012313+00:00 Received 200
I 2024-02-04 08:42:02.012630+00:00 requests.head https://vt.social/tags/VTuber {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:02.648925+00:00 Received 200
I 2024-02-04 08:42:02.649233+00:00 requests.head https://vt.social/tags/VTubers {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:03.254877+00:00 Received 200
I 2024-02-04 08:42:03.255193+00:00 requests.head https://vt.social/tags/TheRMan {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:03.842122+00:00 Received 200
I 2024-02-04 08:42:03.842488+00:00 requests.head https://vt.social/tags/TheRLive {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:04.451445+00:00 Received 200
I 2024-02-04 08:42:04.451809+00:00 requests.head https://vt.social/tags/MembersRF {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:05.046554+00:00 Received 200
I 2024-02-04 08:42:05.046953+00:00 requests.head https://vt.social/tags/VirtualLifeDE {'headers': {'User-Agent': '...'}}
I 2024-02-04 08:42:05.651229+00:00 Received 200
I 2024-02-04 08:42:05.652563+00:00 Original post discovery found original posts set(), mentions {'https://www.twitch.tv/therman', 'https://vt.social/tags/VTubers', 'https://vt.social/tags/Twitch', 'https://vt.social/tags/MembersRF', 'https://vt.social/tags/TheBearsDen', 'https://vt.social/tags/TheRMan', 'https://vt.social/tags/VLDE', 'https://vt.social/tags/VirtualLifeDE', 'https://vt.social/tags/VTuber', 'https://vt.social/tags/GERVTuber', 'https://vt.social/tags/TheRLive', 'https://vt.social/tags/ENVtuber'}
I 2024-02-04 08:42:05.652956+00:00 12 mentions, pruning down to 10
I 2024-02-04 08:42:05.655762+00:00 requests.head https://vt.social/tags/TheBearsDen {'headers': {}}
I 2024-02-04 08:42:06.267231+00:00 Received 200
I 2024-02-04 08:42:06.271421+00:00 requests.head https://vt.social/tags/VLDE {'headers': {}}
I 2024-02-04 08:42:06.845254+00:00 Received 200
I 2024-02-04 08:42:06.848026+00:00 requests.head https://vt.social/@TheRMan/111867199926331578 {'headers': None}
I 2024-02-04 08:42:07.464780+00:00 Received 200
I 2024-02-04 08:42:07.465115+00:00 starting posse post discovery with syndicated https://vt.social/@TheRMan/111867199926331578
I 2024-02-04 08:42:07.498836+00:00 requests.head https://www.tiktok.com/@therman98 {'headers': {}}
I 2024-02-04 08:42:07.749189+00:00 Received 403
W 2024-02-04 08:42:07.749469+00:00 Couldn't resolve URL https://www.tiktok.com/@therman98: https://www.tiktok.com/@therman98
I 2024-02-04 08:42:07.749784+00:00 requests.head https://www.fiverr.com/s/p6LNxR {'headers': {}}
I 2024-02-04 08:42:08.239584+00:00 Redirected to https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
I 2024-02-04 08:42:08.240113+00:00 Received 403
W 2024-02-04 08:42:08.240316+00:00 Couldn't resolve URL https://www.fiverr.com/s/p6LNxR: https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
D 2024-02-04 08:42:08.241686+00:00 fetching author url https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
I 2024-02-04 08:42:08.241913+00:00 requests.get https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d {'gateway': False, 'headers': {}}
I 2024-02-04 08:42:08.451174+00:00 Redirected to https://block.fiverr.com/?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
I 2024-02-04 08:42:08.454281+00:00 Received 403
I 2024-02-04 08:42:08.454512+00:00 Could not fetch author url https://block.fiverr.com?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2224, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://block.fiverr.com/?url=aHR0cHM6Ly93d3cuZml2ZXJyLmNvbS9zL3A2TE54Ug%3D%3D&uuid=47c4bd7b-c339-11ee-8ff4-83da37dec19d
I 2024-02-04 08:42:08.478474+00:00 requests.head https://trello.com/b/gmVCR4qD/editing-commissions {'headers': {}}
I 2024-02-04 08:42:08.798604+00:00 Received 200
D 2024-02-04 08:42:08.799655+00:00 fetching author url https://trello.com/b/gmVCR4qD/editing-commissions
I 2024-02-04 08:42:08.799863+00:00 requests.get https://trello.com/b/gmVCR4qD/editing-commissions {'gateway': False, 'headers': {}}
I 2024-02-04 08:42:09.178231+00:00 Received 200
D 2024-02-04 08:42:09.189121+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-04 08:42:09.191267+00:00 requests.head https://twittodon.com/share.php?t=therman98&amp%3Bm=therman%40vt.social {'headers': {}}
I 2024-02-04 08:42:09.746327+00:00 Redirected to https://twittodon.com
I 2024-02-04 08:42:09.746938+00:00 Received 200
D 2024-02-04 08:42:09.747164+00:00 Resolved https://twittodon.com/share.php?t=therman98&amp%3Bm=therman%40vt.social to https://twittodon.com
D 2024-02-04 08:42:09.748429+00:00 fetching author url https://twittodon.com
I 2024-02-04 08:42:09.748899+00:00 requests.get https://twittodon.com {'gateway': False, 'headers': {}}
I 2024-02-04 08:42:10.122182+00:00 Redirected to https://twittodon.com/
I 2024-02-04 08:42:10.123984+00:00 Received 200
D 2024-02-04 08:42:10.130902+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-04 08:42:10.131537+00:00 requests.head https://therman.eu/ {'headers': {}}
I 2024-02-04 08:42:10.434807+00:00 Received 200
D 2024-02-04 08:42:10.438821+00:00 fetching author url https://therman.eu/
I 2024-02-04 08:42:10.439101+00:00 requests.get https://therman.eu/ {'gateway': False, 'headers': {}}
I 2024-02-04 08:42:10.688789+00:00 Received 200
D 2024-02-04 08:42:10.781057+00:00 No h-feed found, fallback to top-level h-entrys.
D 2024-02-04 08:42:10.782089+00:00 posse post discovery found no relationship for https://vt.social/@TheRMan/111867199926331578
I 2024-02-04 08:42:11.153536+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 1970-01-01 00:00:00+00:00
I 2024-02-04 08:42:11.262676+00:00 Updating @TheRMan@vt.social (Mastodon) /mastodon/@therman@vt.social : {'last_public_post': datetime.datetime(2024, 2, 4, 1, 6, 28, 447000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 2, 4, 8, 41, 52, 36442, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-02-04 08:42:11.399438+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/59649823754801067541 with ETA 1707119151: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ciALEghNYXN0b2RvbiISQHRoZXJtYW5AdnQuc29jaWFsDA&last_polled=2024-02-04-08-41-52', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1707119151
}