D 2024-01-30 01:07:03.008133+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-01-28-22-09-39')]
I 2024-01-30 01:07:03.040231+00:00 Source: @adam@ruby.social (Mastodon) @adam@ruby.social, https://brid.gy/mastodon/@adam@ruby.social
I 2024-01-30 01:07:03.040647+00:00 Last poll: https://brid.gy/log?start_time=1706479779&key=agdicmlkLWd5ch8LEghNYXN0b2RvbiIRQGFkYW1AcnVieS5zb2NpYWwM
I 2024-01-30 01:07:03.063474+00:00 Updating @adam@ruby.social (Mastodon) /mastodon/@adam@ruby.social : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 1, 30, 1, 7, 3, 40762, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-01-30 01:07:03.298862+00:00 no refresh_token
I 2024-01-30 01:07:03.322724+00:00 requests.get https://ruby.social/api/v1/accounts/109344395062540706/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-01-30 01:07:04.595770+00:00 Redirected to https://ruby.social/api/v1/accounts/109344395062540706/statuses?limit=30
I 2024-01-30 01:07:04.700401+00:00 Received 200
I 2024-01-30 01:07:04.711440+00:00 requests.get https://ruby.social/api/v1/statuses/111837120732125879/favourited_by {'headers': {'Authorization': '...'}}
I 2024-01-30 01:07:05.320682+00:00 Received 200
I 2024-01-30 01:07:05.368538+00:00 requests.get https://ruby.social/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-01-30 01:07:06.732960+00:00 Redirected to https://ruby.social/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-01-30 01:07:06.841094+00:00 Received 200
I 2024-01-30 01:07:06.881127+00:00 Found 64 public activities: dict_keys(['tag:ruby.social,2013:111837120732125879', 'tag:ruby.social,2013:111823718183620417', 'tag:ruby.social,2013:111817880748108876', 'tag:ruby.social,2013:111811829950270505', 'tag:ruby.social,2013:111811740326987427', 'tag:ruby.social,2013:111807426806763166', 'tag:ruby.social,2013:111802306097812558', 'tag:ruby.social,2013:111802223043470383', 'tag:ruby.social,2013:111801323612317006', 'tag:ruby.social,2013:111795450025109208', 'tag:ruby.social,2013:111789633523767961', 'tag:ruby.social,2013:111789551623985992', 'tag:ruby.social,2013:111785235922347076', 'tag:ruby.social,2013:111784882114197768', 'tag:ruby.social,2013:111784865913155009', 'tag:ruby.social,2013:111784824919673130', 'tag:ruby.social,2013:111784817292331486', 'tag:ruby.social,2013:111778618721433330', 'tag:ruby.social,2013:111772978464593912', 'tag:ruby.social,2013:111763985277895077', 'tag:ruby.social,2013:111755759582886293', 'tag:ruby.social,2013:111722773984339418', 'tag:ruby.social,2013:111711818400908275', 'tag:ruby.social,2013:111699077575920900', 'tag:ruby.social,2013:111699072619118157', 'tag:ruby.social,2013:111698530382095231', 'tag:ruby.social,2013:111694502775156495', 'tag:ruby.social,2013:111693649069329625', 'tag:ruby.social,2013:111693560265522141', 'tag:ruby.social,2013:111806335671494864', 'tag:ruby.social,2013:111794683518711738', 'tag:ruby.social,2013:111785203038598711', 'tag:ruby.social,2013:111723337589763493', 'tag:ruby.social,2013:111723061010563072', 'tag:ruby.social,2013:111697985283545990', 'tag:ruby.social,2013:111693672287112329', 'tag:ruby.social,2013:111683788014031868', 'tag:ruby.social,2013:111677576409983897', 'tag:ruby.social,2013:111676034851565018', 'tag:ruby.social,2013:111675853760225787', 'tag:ruby.social,2013:111672302703448329', 'tag:ruby.social,2013:111672294076665741', 'tag:ruby.social,2013:111665145196167157', 'tag:ruby.social,2013:111665134960112081', 'tag:ruby.social,2013:111664940222581325', 'tag:ruby.social,2013:111659934214045474', 'tag:ruby.social,2013:111659596417627579', 'tag:ruby.social,2013:111656070499995138', 'tag:ruby.social,2013:111654763736414469', 'tag:ruby.social,2013:111653311495453907', 'tag:ruby.social,2013:111653135003089535', 'tag:ruby.social,2013:111652811080026597', 'tag:ruby.social,2013:111652489454464207', 'tag:ruby.social,2013:111651140945981459', 'tag:ruby.social,2013:111650880752397706', 'tag:ruby.social,2013:111650860374638968', 'tag:ruby.social,2013:111650559424445376', 'tag:ruby.social,2013:111650483262124416', 'tag:ruby.social,2013:111648568966512802', 'tag:ruby.social,2013:111641674071846854', 'tag:ruby.social,2013:111641095160844191', 'tag:ruby.social,2013:111638802355438472', 'tag:ruby.social,2013:111638795287402547', 'tag:ruby.social,2013:111638765231461099'])
I 2024-01-30 01:07:06.881343+00:00 Found 2 private activities: dict_keys(['tag:ruby.social,2013:111784318168385112', 'tag:ruby.social,2013:111783512666724751'])
D 2024-01-30 01:07:06.883825+00:00 discovering original posts for: https://ruby.social/@adam/111837120732125879
I 2024-01-30 01:07:06.884191+00:00 requests.head https://www.youtube.com/watch?v=Lrle0x_DHBM {'headers': {}}
I 2024-01-30 01:07:09.049504+00:00 Received 200
I 2024-01-30 01:07:09.049840+00:00 Original post discovery found original posts set(), mentions {'https://www.youtube.com/watch?v=Lrle0x_DHBM'}
I 2024-01-30 01:07:09.050256+00:00 requests.head https://ruby.social/@adam/111837120732125879 {'headers': None}
I 2024-01-30 01:07:09.722971+00:00 Received 200
I 2024-01-30 01:07:09.723362+00:00 starting posse post discovery with syndicated https://ruby.social/@adam/111837120732125879
I 2024-01-30 01:07:09.759334+00:00 requests.head https://adamfortuna.com/ {'headers': {}}
I 2024-01-30 01:07:10.265753+00:00 Received 200
D 2024-01-30 01:07:10.266941+00:00 fetching author url https://adamfortuna.com/
I 2024-01-30 01:07:10.267095+00:00 requests.get https://adamfortuna.com/ {'gateway': False, 'headers': {}}
I 2024-01-30 01:07:11.682619+00:00 Received 200
D 2024-01-30 01:07:11.864196+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-01-30 01:07:11.864970+00:00 requests.head https://minafi.com/ {'headers': {}}
I 2024-01-30 01:07:12.537206+00:00 Received 200
D 2024-01-30 01:07:12.538437+00:00 fetching author url https://minafi.com/
I 2024-01-30 01:07:12.538594+00:00 requests.get https://minafi.com/ {'gateway': False, 'headers': {}}
I 2024-01-30 01:07:14.393930+00:00 Received 422
I 2024-01-30 01:07:14.394831+00:00 Could not fetch author url https://minafi.com/
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2224, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 422 Client Error: OK for url: https://minafi.com/
I 2024-01-30 01:07:14.461977+00:00 requests.head https://hardcover.app/ {'headers': {}}
I 2024-01-30 01:07:14.914550+00:00 Received 200
D 2024-01-30 01:07:14.915892+00:00 fetching author url https://hardcover.app/
I 2024-01-30 01:07:14.916090+00:00 requests.get https://hardcover.app/ {'gateway': False, 'headers': {}}
I 2024-01-30 01:07:18.478354+00:00 Received 200
D 2024-01-30 01:07:18.964144+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-01-30 01:07:18.965059+00:00 requests.head https://lineofthought.com/ {'headers': {}}
I 2024-01-30 01:07:20.096323+00:00 Received 200
D 2024-01-30 01:07:20.097445+00:00 fetching author url https://lineofthought.com/
I 2024-01-30 01:07:20.097592+00:00 requests.get https://lineofthought.com/ {'gateway': False, 'headers': {}}
I 2024-01-30 01:07:20.884621+00:00 Received 200
D 2024-01-30 01:07:20.888783+00:00 No h-feed found, fallback to top-level h-entrys.
D 2024-01-30 01:07:20.961173+00:00 posse post discovery found no relationship for https://ruby.social/@adam/111837120732125879
I 2024-01-30 01:07:21.319284+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 2023-02-02 07:59:29.416884+00:00
I 2024-01-30 01:07:21.354397+00:00 Updating @adam@ruby.social (Mastodon) /mastodon/@adam@ruby.social : {'last_public_post': datetime.datetime(2024, 1, 29, 3, 30, 34, 609000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 1, 30, 1, 7, 3, 40762, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-01-30 01:07:21.474206+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/4676974933936673642 with ETA 1706650137: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ch8LEghNYXN0b2RvbiIRQGFkYW1AcnVieS5zb2NpYWwM&last_polled=2024-01-30-01-07-03', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1706650137
}