D 2024-01-21 19:04:26.506183+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-01-20-15-56-22')]
I 2024-01-21 19:04:26.527463+00:00 Source: @adam@ruby.social (Mastodon) @adam@ruby.social, https://brid.gy/mastodon/@adam@ruby.social
I 2024-01-21 19:04:26.603387+00:00 Last poll: https://brid.gy/log?start_time=1705766182&key=agdicmlkLWd5ch8LEghNYXN0b2RvbiIRQGFkYW1AcnVieS5zb2NpYWwM
I 2024-01-21 19:04:26.709538+00:00 Updating @adam@ruby.social (Mastodon) /mastodon/@adam@ruby.social : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 1, 21, 19, 4, 26, 603940, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-01-21 19:04:26.926808+00:00 no refresh_token
I 2024-01-21 19:04:27.002372+00:00 requests.get https://ruby.social/api/v1/accounts/109344395062540706/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-01-21 19:04:28.842723+00:00 Redirected to https://ruby.social/api/v1/accounts/109344395062540706/statuses?limit=30
I 2024-01-21 19:04:28.949190+00:00 Received 200
I 2024-01-21 19:04:28.961414+00:00 requests.get https://ruby.social/api/v1/statuses/111789633523767961/favourited_by {'headers': {'Authorization': '...'}}
I 2024-01-21 19:04:29.901307+00:00 Received 200
I 2024-01-21 19:04:30.013806+00:00 requests.get https://ruby.social/api/v1/statuses/111784824919673130/favourited_by {'headers': {'Authorization': '...'}}
I 2024-01-21 19:04:31.162618+00:00 Received 200
I 2024-01-21 19:04:31.414649+00:00 requests.get https://ruby.social/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-01-21 19:04:33.622898+00:00 Redirected to https://ruby.social/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-01-21 19:04:33.837327+00:00 Received 200
I 2024-01-21 19:04:34.122863+00:00 Found 63 public activities: dict_keys(['tag:ruby.social,2013:111795450025109208', 'tag:ruby.social,2013:111789633523767961', 'tag:ruby.social,2013:111789551623985992', 'tag:ruby.social,2013:111785235922347076', 'tag:ruby.social,2013:111784882114197768', 'tag:ruby.social,2013:111784865913155009', 'tag:ruby.social,2013:111784824919673130', 'tag:ruby.social,2013:111784817292331486', 'tag:ruby.social,2013:111778618721433330', 'tag:ruby.social,2013:111772978464593912', 'tag:ruby.social,2013:111763985277895077', 'tag:ruby.social,2013:111755759582886293', 'tag:ruby.social,2013:111722773984339418', 'tag:ruby.social,2013:111711818400908275', 'tag:ruby.social,2013:111699077575920900', 'tag:ruby.social,2013:111699072619118157', 'tag:ruby.social,2013:111698530382095231', 'tag:ruby.social,2013:111694502775156495', 'tag:ruby.social,2013:111693649069329625', 'tag:ruby.social,2013:111693560265522141', 'tag:ruby.social,2013:111683768312477451', 'tag:ruby.social,2013:111683280333837136', 'tag:ruby.social,2013:111678459784507881', 'tag:ruby.social,2013:111676283524844220', 'tag:ruby.social,2013:111676257073664322', 'tag:ruby.social,2013:111676220239007002', 'tag:ruby.social,2013:111676219548538064', 'tag:ruby.social,2013:111676105727755457', 'tag:ruby.social,2013:111676034268113676', 'tag:ruby.social,2013:111794683518711738', 'tag:ruby.social,2013:111785203038598711', 'tag:ruby.social,2013:111723337589763493', 'tag:ruby.social,2013:111723061010563072', 'tag:ruby.social,2013:111697985283545990', 'tag:ruby.social,2013:111693672287112329', 'tag:ruby.social,2013:111683788014031868', 'tag:ruby.social,2013:111677576409983897', 'tag:ruby.social,2013:111676034851565018', 'tag:ruby.social,2013:111675853760225787', 'tag:ruby.social,2013:111672302703448329', 'tag:ruby.social,2013:111672294076665741', 'tag:ruby.social,2013:111665145196167157', 'tag:ruby.social,2013:111665134960112081', 'tag:ruby.social,2013:111664940222581325', 'tag:ruby.social,2013:111659934214045474', 'tag:ruby.social,2013:111659596417627579', 'tag:ruby.social,2013:111656070499995138', 'tag:ruby.social,2013:111654763736414469', 'tag:ruby.social,2013:111653311495453907', 'tag:ruby.social,2013:111653135003089535', 'tag:ruby.social,2013:111652811080026597', 'tag:ruby.social,2013:111652489454464207', 'tag:ruby.social,2013:111651140945981459', 'tag:ruby.social,2013:111650880752397706', 'tag:ruby.social,2013:111650860374638968', 'tag:ruby.social,2013:111650559424445376', 'tag:ruby.social,2013:111650483262124416', 'tag:ruby.social,2013:111648568966512802', 'tag:ruby.social,2013:111641674071846854', 'tag:ruby.social,2013:111641095160844191', 'tag:ruby.social,2013:111638802355438472', 'tag:ruby.social,2013:111638795287402547', 'tag:ruby.social,2013:111638765231461099'])
I 2024-01-21 19:04:34.123125+00:00 Found 2 private activities: dict_keys(['tag:ruby.social,2013:111784318168385112', 'tag:ruby.social,2013:111783512666724751'])
D 2024-01-21 19:04:34.127625+00:00 discovering original posts for: https://ruby.social/@adam/111789633523767961
I 2024-01-21 19:04:34.128165+00:00 requests.head https://realsocial.life/@imtheq {'headers': {}}
I 2024-01-21 19:04:35.201823+00:00 Received 200
I 2024-01-21 19:04:35.202206+00:00 Original post discovery found original posts set(), mentions {'https://realsocial.life/@imtheq'}
I 2024-01-21 19:04:35.202564+00:00 requests.head https://ruby.social/@adam/111789633523767961 {'headers': None}
I 2024-01-21 19:04:36.083165+00:00 Received 200
I 2024-01-21 19:04:36.083446+00:00 starting posse post discovery with syndicated https://ruby.social/@adam/111789633523767961
I 2024-01-21 19:04:36.114394+00:00 requests.head https://adamfortuna.com/ {'headers': {}}
I 2024-01-21 19:04:37.023212+00:00 Received 200
D 2024-01-21 19:04:37.024244+00:00 fetching author url https://adamfortuna.com/
I 2024-01-21 19:04:37.024452+00:00 requests.get https://adamfortuna.com/ {'gateway': False, 'headers': {}}
I 2024-01-21 19:04:37.423940+00:00 Received 200
D 2024-01-21 19:04:37.815817+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-01-21 19:04:37.817116+00:00 requests.head https://minafi.com/ {'headers': {}}
I 2024-01-21 19:04:39.270633+00:00 Received 200
D 2024-01-21 19:04:39.272048+00:00 fetching author url https://minafi.com/
I 2024-01-21 19:04:39.272348+00:00 requests.get https://minafi.com/ {'gateway': False, 'headers': {}}
I 2024-01-21 19:04:39.922140+00:00 Received 422
I 2024-01-21 19:04:39.922462+00:00 Could not fetch author url https://minafi.com/
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2224, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 422 Client Error: OK for url: https://minafi.com/
I 2024-01-21 19:04:40.002254+00:00 requests.head https://hardcover.app/ {'headers': {}}
I 2024-01-21 19:04:40.901911+00:00 Received 200
D 2024-01-21 19:04:40.902928+00:00 fetching author url https://hardcover.app/
I 2024-01-21 19:04:40.903090+00:00 requests.get https://hardcover.app/ {'gateway': False, 'headers': {}}
I 2024-01-21 19:04:42.909313+00:00 Received 200
D 2024-01-21 19:04:43.109258+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-01-21 19:04:43.110339+00:00 requests.head https://lineofthought.com/ {'headers': {}}
I 2024-01-21 19:04:45.501777+00:00 Received 200
D 2024-01-21 19:04:45.502840+00:00 fetching author url https://lineofthought.com/
I 2024-01-21 19:04:45.503041+00:00 requests.get https://lineofthought.com/ {'gateway': False, 'headers': {}}
I 2024-01-21 19:04:46.005809+00:00 Received 200
D 2024-01-21 19:04:46.010325+00:00 No h-feed found, fallback to top-level h-entrys.
D 2024-01-21 19:04:46.010698+00:00 posse post discovery found no relationship for https://ruby.social/@adam/111789633523767961
D 2024-01-21 19:04:46.219524+00:00 discovering original posts for: https://ruby.social/@adam/111784824919673130
I 2024-01-21 19:04:46.219826+00:00 Original post discovery found original posts set(), mentions set()
I 2024-01-21 19:04:46.220114+00:00 requests.head https://ruby.social/@adam/111784824919673130 {'headers': None}
I 2024-01-21 19:04:47.601331+00:00 Received 200
I 2024-01-21 19:04:47.601633+00:00 starting posse post discovery with syndicated https://ruby.social/@adam/111784824919673130
D 2024-01-21 19:04:47.701837+00:00 skipping https://adamfortuna.com/, already fetched this round
D 2024-01-21 19:04:47.702090+00:00 skipping https://minafi.com/, already fetched this round
D 2024-01-21 19:04:47.702187+00:00 skipping https://hardcover.app/, already fetched this round
D 2024-01-21 19:04:47.702268+00:00 skipping https://lineofthought.com/, already fetched this round
D 2024-01-21 19:04:47.702350+00:00 posse post discovery found no relationship for https://ruby.social/@adam/111784824919673130
I 2024-01-21 19:04:48.351285+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 2023-02-02 07:59:29.416884+00:00
I 2024-01-21 19:04:48.419961+00:00 Updating @adam@ruby.social (Mastodon) /mastodon/@adam@ruby.social : {'last_public_post': datetime.datetime(2024, 1, 21, 18, 53, 10, 82000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 1, 21, 19, 4, 26, 603940, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-01-21 19:04:48.592273+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/2134287054225201303 with ETA 1705953880: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ch8LEghNYXN0b2RvbiIRQGFkYW1AcnVieS5zb2NpYWwM&last_polled=2024-01-21-19-04-26', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1705953880
}