This is not a programming question, the code is used to facilitate the demonstration of an issue in facebook app which can be reproduced manually without affecting the outcome.
Comment loading slows down exponentially with the number of comments loaded in browser. The code below uses selenium to login and gets this post which has 10k+ comments. Then the view more comments button is continuously clicked to load more comments as shown below:
from datetime import timedelta
from time import perf_counter
from selenium.common.exceptions import (
ElementClickInterceptedException,
ElementNotInteractableException,
StaleElementReferenceException,
)
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
def login(driver, email, password):
driver.find_element(By.ID, 'email').send_keys(email)
driver.find_element(By.ID, 'pass').send_keys(password)
driver.find_element(By.XPATH, '//*[contains(@type, "submit")]').click()
WebDriverWait(driver, 1000).until(
expected_conditions.presence_of_element_located(
(By.XPATH, '//*[contains(@action, "/logout")]')
)
)
if __name__ == '__main__':
options = ChromeOptions()
options.add_argument('disable-notifications')
_driver = Chrome(options=options)
_driver.get('https://facebook.com')
login(_driver, 'email', 'password') # replace with credentials
_driver.get(
'https://www.facebook.com/SarcasmSociety/photos/a.10150919189073866/10155498125313866'
)
reload_btn = WebDriverWait(_driver, 1000).until(
expected_conditions.presence_of_element_located(
(By.XPATH, '//*[contains(text(), "View more comments")]')
)
)
prev_count = 0
t = perf_counter()
while True:
_driver.execute_script('arguments[0].scrollIntoView();', reload_btn)
try:
reload_btn.click()
except (
ElementClickInterceptedException,
StaleElementReferenceException,
ElementNotInteractableException,
):
pass
total_loaded = (
WebDriverWait(_driver, 1000)
.until(
expected_conditions.presence_of_element_located(
(By.XPATH, '//div[4]/div[2]/div[2]/span')
)
)
.text.split('of')[0]
)
if prev_count != total_loaded:
print(
f'comments loaded: {total_loaded} load time {timedelta(seconds=perf_counter() - t)}'
)
prev_count = total_loaded
t = perf_counter()
As the number of comments loaded increases, the loading time keeps increasing and the browser eventually hangs and crashes. I tried with chrome, safari, firefox and I get the same result.
comments loaded: 6 load time 0:00:00.956970
comments loaded: 56 load time 0:00:05.018559
comments loaded: 106 load time 0:00:05.097524
comments loaded: 156 load time 0:00:12.881246
comments loaded: 206 load time 0:00:14.915316
comments loaded: 256 load time 0:00:07.909231
comments loaded: 306 load time 0:00:14.662676
comments loaded: 356 load time 0:00:17.050463
comments loaded: 406 load time 0:00:14.737875
comments loaded: 456 load time 0:00:10.441015
comments loaded: 506 load time 0:00:22.218294
comments loaded: 556 load time 0:00:20.039933
comments loaded: 606 load time 0:00:14.072164
comments loaded: 656 load time 0:00:33.570592
comments loaded: 706 load time 0:00:29.157080
comments loaded: 756 load time 0:00:27.104434
comments loaded: 806 load time 0:00:28.285868
comments loaded: 856 load time 0:00:29.938833
comments loaded: 906 load time 0:00:31.281724
comments loaded: 956 load time 0:00:36.025857
comments loaded: 1,006 load time 0:00:34.422590
comments loaded: 1,056 load time 0:00:32.973732
comments loaded: 1,156 load time 0:01:16.370328
comments loaded: 1,206 load time 0:00:44.005518
comments loaded: 1,256 load time 0:00:47.166408
comments loaded: 1,306 load time 0:00:44.392992
comments loaded: 1,356 load time 0:01:01.694369
comments loaded: 1,406 load time 0:00:28.100825
comments loaded: 1,456 load time 0:00:52.131026
comments loaded: 1,506 load time 0:00:57.448965
comments loaded: 1,556 load time 0:00:53.571871
comments loaded: 1,606 load time 0:00:57.142948
comments loaded: 1,656 load time 0:00:58.251598
comments loaded: 1,706 load time 0:01:33.050878
comments loaded: 1,756 load time 0:00:33.725328
comments loaded: 1,856 load time 0:02:20.364518
comments loaded: 1,906 load time 0:01:08.370199
comments loaded: 1,956 load time 0:01:22.470486
comments loaded: 2,006 load time 0:01:35.449901
comments loaded: 2,056 load time 0:02:20.828311
comments loaded: 2,106 load time 0:00:49.856041
comments loaded: 2,156 load time 0:01:38.646713
comments loaded: 2,206 load time 0:01:38.485394
comments loaded: 2,256 load time 0:01:46.626105
comments loaded: 2,306 load time 0:02:00.718524
Traceback (most recent call last):
File "/Users/user/Library/Application Support/JetBrains/PyCharm2022.3/scratches/scratch_2.py", line 48, in <module>
reload_btn.click()
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 93, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 410, in _execute
return self._parent.execute(command, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 444, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 249, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: chrome=108.0.5359.124)
Stacktrace:
0 chromedriver 0x0000000100d16f38 chromedriver + 4910904
1 chromedriver 0x0000000100c96a03 chromedriver + 4385283
2 chromedriver 0x00000001008db5bd chromedriver + 472509
3 chromedriver 0x00000001008c4064 chromedriver + 376932
4 chromedriver 0x00000001008c27f2 chromedriver + 370674
5 chromedriver 0x00000001008c2d18 chromedriver + 371992
6 chromedriver 0x00000001008d144c chromedriver + 431180
7 chromedriver 0x00000001008d2752 chromedriver + 436050
8 chromedriver 0x00000001008e50ce chromedriver + 512206
9 chromedriver 0x0000000100961425 chromedriver + 1020965
10 chromedriver 0x0000000100945ee3 chromedriver + 909027
11 chromedriver 0x000000010091030c chromedriver + 688908
12 chromedriver 0x000000010091188e chromedriver + 694414
13 chromedriver 0x0000000100ce41de chromedriver + 4702686
14 chromedriver 0x0000000100ce8b19 chromedriver + 4721433
15 chromedriver 0x0000000100cf028e chromedriver + 4752014
16 chromedriver 0x0000000100ce991a chromedriver + 4725018
17 chromedriver 0x0000000100cbdb02 chromedriver + 4545282
18 chromedriver 0x0000000100d08888 chromedriver + 4851848
19 chromedriver 0x0000000100d08a05 chromedriver + 4852229
20 chromedriver 0x0000000100d1ee5f chromedriver + 4943455
21 libsystem_pthread.dylib 0x00007ff8163df259 _pthread_start + 125
22 libsystem_pthread.dylib 0x00007ff8163dac7b thread_start + 15
python-BaseException
At first I thought the browser is being crowded by comments and that's why it's crashing and maybe deletion of the loaded comments from the page might fix the issue, so I tried:
for (let element of $x("//div[4]/ul/li")) {
element.parentNode.removeChild(element)
}
Nothing improves and the browser eventually crashes again. I thought perhaps the reload requests are being throttled by the server specially after getting the same exact response time multiple times. But if this is the case, the page won't freeze and stop responding, just the delay would be expected in this case without a browser crash. So, what's causing the crash? How to avoid this outcome and eventually load all comments in a reasonable time?
