0

This is not a programming question, the code is used to facilitate the demonstration of an issue in facebook app which can be reproduced manually without affecting the outcome.

Comment loading slows down exponentially with the number of comments loaded in browser. The code below uses selenium to login and gets this post which has 10k+ comments. Then the view more comments button is continuously clicked to load more comments as shown below:

from datetime import timedelta
from time import perf_counter

from selenium.common.exceptions import (
    ElementClickInterceptedException,
    ElementNotInteractableException,
    StaleElementReferenceException,
)
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait


def login(driver, email, password):
    driver.find_element(By.ID, 'email').send_keys(email)
    driver.find_element(By.ID, 'pass').send_keys(password)
    driver.find_element(By.XPATH, '//*[contains(@type, "submit")]').click()
    WebDriverWait(driver, 1000).until(
        expected_conditions.presence_of_element_located(
            (By.XPATH, '//*[contains(@action, "/logout")]')
        )
    )


if __name__ == '__main__':
    options = ChromeOptions()
    options.add_argument('disable-notifications')
    _driver = Chrome(options=options)
    _driver.get('https://facebook.com')
    login(_driver, 'email', 'password')  # replace with credentials
    _driver.get(
        'https://www.facebook.com/SarcasmSociety/photos/a.10150919189073866/10155498125313866'
    )
    reload_btn = WebDriverWait(_driver, 1000).until(
        expected_conditions.presence_of_element_located(
            (By.XPATH, '//*[contains(text(), "View more comments")]')
        )
    )
    prev_count = 0
    t = perf_counter()
    while True:
        _driver.execute_script('arguments[0].scrollIntoView();', reload_btn)
        try:
            reload_btn.click()
        except (
            ElementClickInterceptedException,
            StaleElementReferenceException,
            ElementNotInteractableException,
        ):
            pass
        total_loaded = (
            WebDriverWait(_driver, 1000)
            .until(
                expected_conditions.presence_of_element_located(
                    (By.XPATH, '//div[4]/div[2]/div[2]/span')
                )
            )
            .text.split('of')[0]
        )
        if prev_count != total_loaded:
            print(
                f'comments loaded: {total_loaded} load time {timedelta(seconds=perf_counter() - t)}'
            )
            prev_count = total_loaded
            t = perf_counter()

As the number of comments loaded increases, the loading time keeps increasing and the browser eventually hangs and crashes. I tried with chrome, safari, firefox and I get the same result.

comments loaded: 6  load time 0:00:00.956970
comments loaded: 56  load time 0:00:05.018559
comments loaded: 106  load time 0:00:05.097524
comments loaded: 156  load time 0:00:12.881246
comments loaded: 206  load time 0:00:14.915316
comments loaded: 256  load time 0:00:07.909231
comments loaded: 306  load time 0:00:14.662676
comments loaded: 356  load time 0:00:17.050463
comments loaded: 406  load time 0:00:14.737875
comments loaded: 456  load time 0:00:10.441015
comments loaded: 506  load time 0:00:22.218294
comments loaded: 556  load time 0:00:20.039933
comments loaded: 606  load time 0:00:14.072164
comments loaded: 656  load time 0:00:33.570592
comments loaded: 706  load time 0:00:29.157080
comments loaded: 756  load time 0:00:27.104434
comments loaded: 806  load time 0:00:28.285868
comments loaded: 856  load time 0:00:29.938833
comments loaded: 906  load time 0:00:31.281724
comments loaded: 956  load time 0:00:36.025857
comments loaded: 1,006  load time 0:00:34.422590
comments loaded: 1,056  load time 0:00:32.973732
comments loaded: 1,156  load time 0:01:16.370328
comments loaded: 1,206  load time 0:00:44.005518
comments loaded: 1,256  load time 0:00:47.166408
comments loaded: 1,306  load time 0:00:44.392992
comments loaded: 1,356  load time 0:01:01.694369
comments loaded: 1,406  load time 0:00:28.100825
comments loaded: 1,456  load time 0:00:52.131026
comments loaded: 1,506  load time 0:00:57.448965
comments loaded: 1,556  load time 0:00:53.571871
comments loaded: 1,606  load time 0:00:57.142948
comments loaded: 1,656  load time 0:00:58.251598
comments loaded: 1,706  load time 0:01:33.050878
comments loaded: 1,756  load time 0:00:33.725328
comments loaded: 1,856  load time 0:02:20.364518
comments loaded: 1,906  load time 0:01:08.370199
comments loaded: 1,956  load time 0:01:22.470486
comments loaded: 2,006  load time 0:01:35.449901
comments loaded: 2,056  load time 0:02:20.828311
comments loaded: 2,106  load time 0:00:49.856041
comments loaded: 2,156  load time 0:01:38.646713
comments loaded: 2,206  load time 0:01:38.485394
comments loaded: 2,256  load time 0:01:46.626105
comments loaded: 2,306  load time 0:02:00.718524
Traceback (most recent call last):
  File "/Users/user/Library/Application Support/JetBrains/PyCharm2022.3/scratches/scratch_2.py", line 48, in <module>
    reload_btn.click()
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 93, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 410, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 444, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 249, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
  (Session info: chrome=108.0.5359.124)
Stacktrace:
0   chromedriver                        0x0000000100d16f38 chromedriver + 4910904
1   chromedriver                        0x0000000100c96a03 chromedriver + 4385283
2   chromedriver                        0x00000001008db5bd chromedriver + 472509
3   chromedriver                        0x00000001008c4064 chromedriver + 376932
4   chromedriver                        0x00000001008c27f2 chromedriver + 370674
5   chromedriver                        0x00000001008c2d18 chromedriver + 371992
6   chromedriver                        0x00000001008d144c chromedriver + 431180
7   chromedriver                        0x00000001008d2752 chromedriver + 436050
8   chromedriver                        0x00000001008e50ce chromedriver + 512206
9   chromedriver                        0x0000000100961425 chromedriver + 1020965
10  chromedriver                        0x0000000100945ee3 chromedriver + 909027
11  chromedriver                        0x000000010091030c chromedriver + 688908
12  chromedriver                        0x000000010091188e chromedriver + 694414
13  chromedriver                        0x0000000100ce41de chromedriver + 4702686
14  chromedriver                        0x0000000100ce8b19 chromedriver + 4721433
15  chromedriver                        0x0000000100cf028e chromedriver + 4752014
16  chromedriver                        0x0000000100ce991a chromedriver + 4725018
17  chromedriver                        0x0000000100cbdb02 chromedriver + 4545282
18  chromedriver                        0x0000000100d08888 chromedriver + 4851848
19  chromedriver                        0x0000000100d08a05 chromedriver + 4852229
20  chromedriver                        0x0000000100d1ee5f chromedriver + 4943455
21  libsystem_pthread.dylib             0x00007ff8163df259 _pthread_start + 125
22  libsystem_pthread.dylib             0x00007ff8163dac7b thread_start + 15
python-BaseException

At first I thought the browser is being crowded by comments and that's why it's crashing and maybe deletion of the loaded comments from the page might fix the issue, so I tried:

for (let element of $x("//div[4]/ul/li")) {
    element.parentNode.removeChild(element)
}

Nothing improves and the browser eventually crashes again. I thought perhaps the reload requests are being throttled by the server specially after getting the same exact response time multiple times. But if this is the case, the page won't freeze and stop responding, just the delay would be expected in this case without a browser crash. So, what's causing the crash? How to avoid this outcome and eventually load all comments in a reasonable time?

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.