D 2024-02-16 04:02:37.007610+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-02-15-03-05-28')]
I 2024-02-16 04:02:37.028060+00:00 Source: @brad@bne.social (Mastodon) @brad@bne.social, https://brid.gy/mastodon/@brad@bne.social
I 2024-02-16 04:02:37.028492+00:00 Last poll: https://brid.gy/log?start_time=1707966328&key=agdicmlkLWd5ch4LEghNYXN0b2RvbiIQQGJyYWRAYm5lLnNvY2lhbAw
I 2024-02-16 04:02:37.044578+00:00 Updating @brad@bne.social (Mastodon) /mastodon/@brad@bne.social : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 2, 16, 4, 2, 37, 28589, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-02-16 04:02:37.153154+00:00 no refresh_token
I 2024-02-16 04:02:37.228904+00:00 requests.get https://bne.social/api/v1/accounts/109277231880074032/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:38.704106+00:00 Redirected to https://bne.social/api/v1/accounts/109277231880074032/statuses?limit=30
I 2024-02-16 04:02:38.706942+00:00 Received 200
I 2024-02-16 04:02:38.712140+00:00 requests.get https://bne.social/api/v1/statuses/111938241506202289/context {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:39.596747+00:00 Received 200
I 2024-02-16 04:02:39.600026+00:00 requests.get https://bne.social/api/v1/statuses/111938241506202289/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:40.377862+00:00 Received 200
I 2024-02-16 04:02:40.381436+00:00 requests.get https://bne.social/api/v1/statuses/111938241506202289/reblogged_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:41.233033+00:00 Received 200
I 2024-02-16 04:02:41.239451+00:00 requests.get https://bne.social/api/v1/statuses/111937514106350027/context {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:42.061103+00:00 Received 200
I 2024-02-16 04:02:42.066517+00:00 requests.get https://bne.social/api/v1/statuses/111937514106350027/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:42.847580+00:00 Received 200
I 2024-02-16 04:02:42.851692+00:00 requests.get https://bne.social/api/v1/statuses/111937514106350027/reblogged_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:43.672315+00:00 Received 200
I 2024-02-16 04:02:43.680438+00:00 requests.get https://bne.social/api/v1/statuses/111933744207049922/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:44.458516+00:00 Received 200
I 2024-02-16 04:02:44.462001+00:00 requests.get https://bne.social/api/v1/statuses/111932634169444126/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:45.272009+00:00 Received 200
I 2024-02-16 04:02:45.289855+00:00 requests.get https://bne.social/api/v1/statuses/111922129698678598/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:46.070357+00:00 Received 200
I 2024-02-16 04:02:46.142096+00:00 requests.get https://bne.social/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-02-16 04:02:48.310600+00:00 Redirected to https://bne.social/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-02-16 04:02:48.314383+00:00 Received 200
I 2024-02-16 04:02:48.363696+00:00 Found 39 public activities: dict_keys(['tag:bne.social,2013:111938241506202289', 'tag:bne.social,2013:111937514106350027', 'tag:bne.social,2013:111933744207049922', 'tag:bne.social,2013:111932634169444126', 'tag:bne.social,2013:111929550548226010', 'tag:bne.social,2013:111929067880398119', 'tag:bne.social,2013:111928533564153382', 'tag:bne.social,2013:111922129698678598', 'tag:bne.social,2013:111921692098338194', 'tag:bne.social,2013:111916608100345790', 'tag:bne.social,2013:111915704123104388', 'tag:bne.social,2013:111915457558646886', 'tag:bne.social,2013:111911351630395381', 'tag:bne.social,2013:111896104648729573', 'tag:bne.social,2013:111894617386752945', 'tag:bne.social,2013:111894519253419200', 'tag:bne.social,2013:111890104952129224', 'tag:bne.social,2013:111883509223605510', 'tag:bne.social,2013:111882021447000105', 'tag:bne.social,2013:111938387759142438', 'tag:bne.social,2013:111937842115555191', 'tag:bne.social,2013:111933055453245847', 'tag:bne.social,2013:111931726768107917', 'tag:bne.social,2013:111928553161508538', 'tag:bne.social,2013:111924275685732134', 'tag:bne.social,2013:111923323893082561', 'tag:bne.social,2013:111915785121762233', 'tag:bne.social,2013:111860986851970064', 'tag:bne.social,2013:111855029708258451', 'tag:bne.social,2013:111850867865699306', 'tag:bne.social,2013:111850832471971644', 'tag:bne.social,2013:111850253829459172', 'tag:bne.social,2013:111844845175109379', 'tag:bne.social,2013:111844773279312273', 'tag:bne.social,2013:111844526543625516', 'tag:bne.social,2013:111844499518212553', 'tag:bne.social,2013:111844484228030642', 'tag:bne.social,2013:111842162389801379', 'tag:bne.social,2013:111837097630121633'])
I 2024-02-16 04:02:48.363983+00:00 Found 20 private activities: dict_keys(['tag:bne.social,2013:111928200450497725', 'tag:bne.social,2013:111922579550337742', 'tag:bne.social,2013:111922098598851130', 'tag:bne.social,2013:111895220098999207', 'tag:bne.social,2013:111895216576208819', 'tag:bne.social,2013:111894615033302342', 'tag:bne.social,2013:111894594234076025', 'tag:bne.social,2013:111890109614818740', 'tag:bne.social,2013:111887899979719365', 'tag:bne.social,2013:111883604008119296', 'tag:bne.social,2013:111883311752560448', 'tag:bne.social,2013:111894650712161184', 'tag:bne.social,2013:111890112022569708', 'tag:bne.social,2013:111883674392178185', 'tag:bne.social,2013:111883329085874612', 'tag:bne.social,2013:111861399488066641', 'tag:bne.social,2013:111856374025657090', 'tag:bne.social,2013:111856277419440814', 'tag:bne.social,2013:111855236014996371', 'tag:bne.social,2013:111836338482982921'])
I 2024-02-16 04:02:48.377052+00:00 Skipping blocked/opt out user: {
"objectType": "person",
"id": "tag:mstdn.social,2013:mike_k",
"numeric_id": "109696967354448630",
"username": "mike_k",
"displayName": "Mike \ud83c\udff4\u200d\u2620\ufe0f",
"url": "https://mstdn.social/@mike_k",
"urls": [
{
"value": "https://mstdn.social/@mike_k"
},
{
"value": "https://mikekreuzer.com"
},
{
"value": "https://codeberg.org/kreuzer"
}
],
"image": {
"url": "https://assets.bne.social/cache/accounts/avatars/109/696/967/354/448/630/original/5b0902ab1bc430ec.png"
},
"published": "2023-01-16T00:00:00.000Z",
"description": "<p>Once and future code ... VK1OMG on the air.</p><p>Satisfaction guaranteed or your money cheerfully refunded.</p><p><a href=\"https://mstdn.social/tags/gamedev\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>gamedev</span></a> <a href=\"https://mstdn.social/tags/indiedev\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>indiedev</span></a> <a href=\"https://mstdn.social/tags/wargame\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>wargame</span></a> <a href=\"https://mstdn.social/tags/hamradio\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>hamradio</span></a> <a href=\"https://mstdn.social/tags/nobot\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>nobot</span></a></p>"
}
D 2024-02-16 04:02:48.443818+00:00 discovering original posts for: https://bne.social/@brad/111937514106350027
I 2024-02-16 04:02:48.444257+00:00 requests.head https://bne.social/tags/BNEWeather {'headers': {}}
I 2024-02-16 04:02:49.250695+00:00 Received 200
I 2024-02-16 04:02:49.251165+00:00 requests.head https://bne.social/tags/WeatherTootin {'headers': {'User-Agent': '...'}}
I 2024-02-16 04:02:50.092219+00:00 Received 200
I 2024-02-16 04:02:50.092628+00:00 Original post discovery found original posts set(), mentions {'https://bne.social/tags/BNEWeather', 'https://bne.social/tags/WeatherTootin'}
I 2024-02-16 04:02:50.094454+00:00 requests.head https://bne.social/@brad/111937514106350027 {'headers': None}
I 2024-02-16 04:02:50.958095+00:00 Received 200
I 2024-02-16 04:02:50.958511+00:00 starting posse post discovery with syndicated https://bne.social/@brad/111937514106350027
I 2024-02-16 04:02:51.001215+00:00 requests.head https://bradparker.com/ {'headers': {}}
I 2024-02-16 04:02:51.996370+00:00 Received 200
D 2024-02-16 04:02:51.997655+00:00 fetching author url https://bradparker.com/
I 2024-02-16 04:02:51.997917+00:00 requests.get https://bradparker.com/ {'gateway': False, 'headers': {}}
I 2024-02-16 04:02:52.968347+00:00 Received 200
D 2024-02-16 04:02:52.990330+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-16 04:02:52.991051+00:00 requests.head https://toot.cafe/@brad {'headers': {}}
I 2024-02-16 04:02:53.422916+00:00 Received 200
D 2024-02-16 04:02:53.424065+00:00 fetching author url https://toot.cafe/@brad
I 2024-02-16 04:02:53.424212+00:00 requests.get https://toot.cafe/@brad {'gateway': False, 'headers': {}}
I 2024-02-16 04:02:53.810574+00:00 Received 200
D 2024-02-16 04:02:53.817487+00:00 No h-feed found, fallback to top-level h-entrys.
I 2024-02-16 04:02:53.818147+00:00 requests.head https://pronoun.is/he {'headers': {}}
I 2024-02-16 04:02:54.147406+00:00 Received 404
W 2024-02-16 04:02:54.147690+00:00 Couldn't resolve URL https://pronoun.is/he: https://pronoun.is/he
D 2024-02-16 04:02:54.148685+00:00 fetching author url https://pronoun.is/he
I 2024-02-16 04:02:54.148822+00:00 requests.get https://pronoun.is/he {'gateway': False, 'headers': {}}
I 2024-02-16 04:02:54.461105+00:00 Received 404
I 2024-02-16 04:02:54.461430+00:00 Could not fetch author url https://pronoun.is/he
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2229, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pronoun.is/he
D 2024-02-16 04:02:54.463772+00:00 posse post discovery found no relationship for https://bne.social/@brad/111937514106350027
D 2024-02-16 04:02:54.604133+00:00 discovering original posts for: https://bne.social/@brad/111938241506202289
I 2024-02-16 04:02:54.604540+00:00 Original post discovery found original posts set(), mentions set()
I 2024-02-16 04:02:54.604845+00:00 requests.head https://bne.social/@brad/111938241506202289 {'headers': None}
I 2024-02-16 04:02:55.362023+00:00 Received 200
I 2024-02-16 04:02:55.362463+00:00 starting posse post discovery with syndicated https://bne.social/@brad/111938241506202289
D 2024-02-16 04:02:55.404057+00:00 skipping https://bradparker.com/, already fetched this round
D 2024-02-16 04:02:55.404304+00:00 skipping https://toot.cafe/@brad, already fetched this round
D 2024-02-16 04:02:55.404401+00:00 skipping https://pronoun.is/he, already fetched this round
D 2024-02-16 04:02:55.404486+00:00 posse post discovery found no relationship for https://bne.social/@brad/111938241506202289
D 2024-02-16 04:02:57.179386+00:00 discovering original posts for: https://bne.social/@brad/111933744207049922
I 2024-02-16 04:02:57.179721+00:00 Original post discovery found original posts set(), mentions set()
I 2024-02-16 04:02:57.180022+00:00 requests.head https://bne.social/@brad/111933744207049922 {'headers': None}
I 2024-02-16 04:02:57.986569+00:00 Received 200
I 2024-02-16 04:02:57.986915+00:00 starting posse post discovery with syndicated https://bne.social/@brad/111933744207049922
D 2024-02-16 04:02:58.024131+00:00 skipping https://bradparker.com/, already fetched this round
D 2024-02-16 04:02:58.024427+00:00 skipping https://toot.cafe/@brad, already fetched this round
D 2024-02-16 04:02:58.024528+00:00 skipping https://pronoun.is/he, already fetched this round
D 2024-02-16 04:02:58.024614+00:00 posse post discovery found no relationship for https://bne.social/@brad/111933744207049922
D 2024-02-16 04:02:58.393725+00:00 discovering original posts for: https://bne.social/@brad/111932634169444126
I 2024-02-16 04:02:58.394154+00:00 Original post discovery found original posts set(), mentions set()
I 2024-02-16 04:02:58.394468+00:00 requests.head https://bne.social/@brad/111932634169444126 {'headers': None}
I 2024-02-16 04:02:59.165586+00:00 Received 200
I 2024-02-16 04:02:59.165924+00:00 starting posse post discovery with syndicated https://bne.social/@brad/111932634169444126
D 2024-02-16 04:02:59.319075+00:00 discovering original posts for: https://bne.social/@brad/111922129698678598
I 2024-02-16 04:02:59.319547+00:00 requests.head https://bne.social/tags/Florespondence {'headers': {}}
I 2024-02-16 04:03:00.104997+00:00 Received 200
I 2024-02-16 04:03:00.106705+00:00 Original post discovery found original posts set(), mentions {'https://bne.social/tags/Florespondence'}
I 2024-02-16 04:03:00.108005+00:00 requests.head https://bne.social/@brad/111922129698678598 {'headers': None}
I 2024-02-16 04:03:00.956668+00:00 Received 200
I 2024-02-16 04:03:00.957082+00:00 starting posse post discovery with syndicated https://bne.social/@brad/111922129698678598
I 2024-02-16 04:03:01.395722+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 2022-11-21 06:22:43.218422+00:00
I 2024-02-16 04:03:01.444615+00:00 Updating @brad@bne.social (Mastodon) /mastodon/@brad@bne.social : {'last_public_post': datetime.datetime(2024, 2, 16, 0, 44, 6, 814000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 2, 16, 4, 2, 37, 28589, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-02-16 04:03:01.681060+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/68147045183620487511 with ETA 1708153355: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ch4LEghNYXN0b2RvbiIQQGJyYWRAYm5lLnNvY2lhbAw&last_polled=2024-02-16-04-02-37', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1708153355
}