D 2024-02-16 15:08:12.008470+00:00 Params: [('source_key', 'agdicmlk...'), ('last_polled', '2024-02-16-09-39-54')]
I 2024-02-16 15:08:12.041869+00:00 Source: @cubicgarden@mas.to (Mastodon) @cubicgarden@mas.to, https://brid.gy/mastodon/@cubicgarden@mas.to
I 2024-02-16 15:08:12.043169+00:00 Last poll: https://brid.gy/log?start_time=1708076394&key=agdicmlkLWd5ciELEghNYXN0b2RvbiITQGN1YmljZ2FyZGVuQG1hcy50bww
I 2024-02-16 15:08:12.070547+00:00 Updating @cubicgarden@mas.to (Mastodon) /mastodon/@cubicgarden@mas.to : {'poll_status': 'polling', 'last_poll_attempt': datetime.datetime(2024, 2, 16, 15, 8, 12, 43844, tzinfo=datetime.timezone.utc), 'rate_limited': False}
I 2024-02-16 15:08:12.262479+00:00 no refresh_token
I 2024-02-16 15:08:12.367039+00:00 requests.get https://mas.to/api/v1/accounts/148312/statuses {'params': {'limit': 30}, 'headers': {'Authorization': '...'}}
I 2024-02-16 15:08:13.449278+00:00 Redirected to https://mas.to/api/v1/accounts/148312/statuses?limit=30
I 2024-02-16 15:08:13.451670+00:00 Received 200
I 2024-02-16 15:08:13.542604+00:00 requests.get https://mas.to/api/v1/statuses/111941706703864012/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 15:08:14.024630+00:00 Received 200
I 2024-02-16 15:08:14.028071+00:00 requests.get https://mas.to/api/v1/statuses/111940889860073827/favourited_by {'headers': {'Authorization': '...'}}
I 2024-02-16 15:08:14.695601+00:00 Received 200
I 2024-02-16 15:08:14.761115+00:00 requests.get https://mas.to/api/v1/notifications {'params': {'exclude_types[]': ['follow', 'favourite', 'reblog']}, 'headers': {'Authorization': '...'}}
I 2024-02-16 15:08:16.376716+00:00 Redirected to https://mas.to/api/v1/notifications?exclude_types%5B%5D=follow&exclude_types%5B%5D=favourite&exclude_types%5B%5D=reblog
I 2024-02-16 15:08:16.380416+00:00 Received 200
I 2024-02-16 15:08:16.462712+00:00 Found 62 public activities: dict_keys(['tag:mas.to,2013:111941706703864012', 'tag:mas.to,2013:111940889860073827', 'tag:mas.to,2013:111940856761394398', 'tag:mas.to,2013:111932887379288846', 'tag:mas.to,2013:111932883375044037', 'tag:mas.to,2013:111930971015556644', 'tag:mas.to,2013:111930400009964795', 'tag:mas.to,2013:111930298462411509', 'tag:mas.to,2013:111929770947879011', 'tag:mas.to,2013:111929376716918499', 'tag:mas.to,2013:111925135523665538', 'tag:mas.to,2013:111925123558190129', 'tag:mas.to,2013:111925093483268806', 'tag:mas.to,2013:111920354829692732', 'tag:mas.to,2013:111920338233002015', 'tag:mas.to,2013:111920300571956538', 'tag:mas.to,2013:111897590336204136', 'tag:mas.to,2013:111896589263914924', 'tag:mas.to,2013:111896544497202478', 'tag:mas.to,2013:111895596835223080', 'tag:mas.to,2013:111895589916547913', 'tag:mas.to,2013:111890696208405075', 'tag:mas.to,2013:111886145390510217', 'tag:mas.to,2013:111884238975926004', 'tag:mas.to,2013:111881577346621421', 'tag:mas.to,2013:111874480606614588', 'tag:mas.to,2013:111874404342394257', 'tag:mas.to,2013:111863281529808890', 'tag:mas.to,2013:111861809266023170', 'tag:mas.to,2013:111855290016602436', 'tag:mas.to,2013:111933028122526068', 'tag:mas.to,2013:111933016625500264', 'tag:mas.to,2013:111930472385852113', 'tag:mas.to,2013:111930470904110363', 'tag:mas.to,2013:111930268696781312', 'tag:mas.to,2013:111930251608447734', 'tag:mas.to,2013:111929888446718097', 'tag:mas.to,2013:111927332163622462', 'tag:mas.to,2013:111927036482700441', 'tag:mas.to,2013:111925317811457143', 'tag:mas.to,2013:111898147264146780', 'tag:mas.to,2013:111896982320624869', 'tag:mas.to,2013:111896372290886610', 'tag:mas.to,2013:111891889742123094', 'tag:mas.to,2013:111884906251163074', 'tag:mas.to,2013:111876463917588040', 'tag:mas.to,2013:111876344200020232', 'tag:mas.to,2013:111876278810068695', 'tag:mas.to,2013:111876171038625759', 'tag:mas.to,2013:111874746179304192', 'tag:mas.to,2013:111863394774283123', 'tag:mas.to,2013:111810987281883936', 'tag:mas.to,2013:111783708663487337', 'tag:mas.to,2013:111775353046332788', 'tag:mas.to,2013:111767820449300296', 'tag:mas.to,2013:111765767770892685', 'tag:mas.to,2013:111762364620984911', 'tag:mas.to,2013:111762294027322565', 'tag:mas.to,2013:111709248989881572', 'tag:mas.to,2013:111692335975193031', 'tag:mas.to,2013:111691344591566391', 'tag:mas.to,2013:111686787133132232'])
I 2024-02-16 15:08:16.462955+00:00 Found 8 private activities: dict_keys(['tag:mas.to,2013:111876314154503375', 'tag:mas.to,2013:111773873519721951', 'tag:mas.to,2013:111773834592439423', 'tag:mas.to,2013:111772660817225422', 'tag:mas.to,2013:111771710206709087', 'tag:mas.to,2013:111768973317043948', 'tag:mas.to,2013:111766317418038935', 'tag:mas.to,2013:111766285903864241'])
D 2024-02-16 15:08:16.467032+00:00 discovering original posts for: https://mas.to/@cubicgarden/111941706703864012
I 2024-02-16 15:08:16.537060+00:00 requests.head https://twitter.com/skylinermcr/status/1758468233072407026 {'headers': {}}
I 2024-02-16 15:08:16.778892+00:00 Received 403
W 2024-02-16 15:08:16.779152+00:00 Couldn't resolve URL https://twitter.com/skylinermcr/status/1758468233072407026: https://twitter.com/skylinermcr/status/1758468233072407026
I 2024-02-16 15:08:16.779366+00:00 requests.head https://mas.to/tags/Manchester {'headers': {'User-Agent': '...'}}
I 2024-02-16 15:08:17.235754+00:00 Received 200
I 2024-02-16 15:08:17.236083+00:00 requests.head https://someone.elses.computer/@billt {'headers': {'User-Agent': '...'}}
I 2024-02-16 15:08:17.973340+00:00 Received 200
I 2024-02-16 15:08:17.973834+00:00 requests.head https://skylinermcr.substack.com/about {'headers': {'User-Agent': '...'}}
I 2024-02-16 15:08:18.358670+00:00 Received 200
I 2024-02-16 15:08:18.359100+00:00 Original post discovery found original posts set(), mentions {'https://skylinermcr.substack.com/about', 'https://someone.elses.computer/@billt', 'https://mas.to/tags/Manchester', 'https://twitter.com/skylinermcr/status/1758468233072407026'}
I 2024-02-16 15:08:18.361567+00:00 requests.head https://mas.to/@cubicgarden/111941706703864012 {'headers': None}
I 2024-02-16 15:08:18.916018+00:00 Received 200
I 2024-02-16 15:08:18.916338+00:00 starting posse post discovery with syndicated https://mas.to/@cubicgarden/111941706703864012
I 2024-02-16 15:08:18.956697+00:00 requests.head https://cubicgarden.info/mixes {'headers': {}}
I 2024-02-16 15:08:20.278547+00:00 Redirected to https://mixes.cubicgarden.info/mixes
I 2024-02-16 15:08:20.278859+00:00 Received 502
W 2024-02-16 15:08:20.278978+00:00 Couldn't resolve URL https://cubicgarden.info/mixes: https://mixes.cubicgarden.info/mixes
D 2024-02-16 15:08:20.280032+00:00 fetching author url https://mixes.cubicgarden.info/mixes
I 2024-02-16 15:08:20.280277+00:00 requests.get https://mixes.cubicgarden.info/mixes {'gateway': False, 'headers': {}}
I 2024-02-16 15:08:20.805741+00:00 Received 502
I 2024-02-16 15:08:20.806068+00:00 Could not fetch author url https://mixes.cubicgarden.info/mixes
Traceback (most recent call last):
File "/workspace/original_post_discovery.py", line 300, in _process_author
author_mf2 = util.fetch_mf2(author_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/util.py", line 273, in fetch_mf2
return util.fetch_mf2(url, get_fn=requests_get, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/oauth_dropins/webutil/util.py", line 2229, in fetch_mf2
resp.raise_for_status()
File "/layers/google.python.pip/pip/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://mixes.cubicgarden.info/mixes
I 2024-02-16 15:08:20.810637+00:00 requests.head https://cubicgarden.com/ {'headers': {}}
I 2024-02-16 15:08:21.148837+00:00 Received 200
D 2024-02-16 15:08:21.150313+00:00 fetching author url https://cubicgarden.com/
I 2024-02-16 15:08:21.150482+00:00 requests.get https://cubicgarden.com/ {'gateway': False, 'headers': {}}
I 2024-02-16 15:08:21.486282+00:00 Received 200
D 2024-02-16 15:08:22.753615+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.754947+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.755291+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.756077+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.757760+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.757937+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758199+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758350+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758463+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758559+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758648+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758737+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758825+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.758909+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.759049+00:00 ignoring h-entry with no u-url!
D 2024-02-16 15:08:22.759159+00:00 posse post discovery found no relationship for https://mas.to/@cubicgarden/111941706703864012
D 2024-02-16 15:08:23.135980+00:00 discovering original posts for: https://mas.to/@cubicgarden/111940889860073827
I 2024-02-16 15:08:23.137336+00:00 requests.head https://hdi-network.org/intro_to_hdi/ {'headers': {}}
I 2024-02-16 15:08:23.781311+00:00 Received 200
I 2024-02-16 15:08:23.781687+00:00 requests.head https://mastodon.online/@parismarx {'headers': {'User-Agent': '...'}}
I 2024-02-16 15:08:24.227822+00:00 Received 200
I 2024-02-16 15:08:24.228346+00:00 Original post discovery found original posts set(), mentions {'https://hdi-network.org/intro_to_hdi/', 'https://mastodon.online/@parismarx'}
I 2024-02-16 15:08:24.229595+00:00 requests.head https://mas.to/@cubicgarden/111940889860073827 {'headers': None}
I 2024-02-16 15:08:24.818052+00:00 Received 200
I 2024-02-16 15:08:24.818355+00:00 starting posse post discovery with syndicated https://mas.to/@cubicgarden/111940889860073827
D 2024-02-16 15:08:24.860988+00:00 skipping https://cubicgarden.info/mixes, already fetched this round
D 2024-02-16 15:08:24.861238+00:00 skipping https://cubicgarden.com/, already fetched this round
D 2024-02-16 15:08:24.861336+00:00 posse post discovery found no relationship for https://mas.to/@cubicgarden/111940889860073827
I 2024-02-16 15:08:25.257122+00:00 skipping refetch h-feed. last-syndication-url None, last-refetch 1970-01-01 00:00:00+00:00
I 2024-02-16 15:08:25.271270+00:00 Updating @cubicgarden@mas.to (Mastodon) /mastodon/@cubicgarden@mas.to : {'last_public_post': datetime.datetime(2024, 2, 16, 14, 48, 9, 893000), 'recent_private_posts': 0, 'last_polled': datetime.datetime(2024, 2, 16, 15, 8, 12, 43844, tzinfo=datetime.timezone.utc), 'poll_status': 'ok'}
I 2024-02-16 15:08:25.380766+00:00 Added poll task projects/brid-gy/locations/us-central1/queues/poll/tasks/1264333969971734154 with ETA 1708114752: {'app_engine_http_request': {'http_method': 'POST', 'relative_uri': '/_ah/queue/poll', 'body': b'source_key=agdicmlkLWd5ciELEghNYXN0b2RvbiITQGN1YmljZ2FyZGVuQG1hcy50bww&last_polled=2024-02-16-15-08-12', 'headers': {'Content-Type': 'application/x-www-form-urlencoded'}}, 'schedule_time': seconds: 1708114752
}