What can i do when i have operation TimeOut ?

[karimWorldSpace]

Hi community,

I'm working on a personal project where I need to retrieve the title and HTML content of a webpage (a simple task).

Sometimes, the URL I visit has protections like cookies, but the HTML content is already fully loaded, so I don’t actually care about the cookie. All the information I need is in the HTML.

Here’s my problem:

1. When I try to evaluate the title of the page, I often get a timeout error.
2. To handle this, I retry the process after the first failure, but I can’t do more than that ?.
3. What’s confusing is that if I manually check the title in the browser’s console, it works perfectly. However, when I try to retrieve the title programmatically in PHP, it doesn’t work.
4. Does anyone know why this might happen or how I can fix it?

Thanks for your help!

Here is my code

` $urls = $urlScrapedByKeyWordRepository->findBy(['isUsedForGeneration' => false]);
shuffle($urls);
$urls = array_slice($urls, 0, 2);

    if ($urls) {
        /** @var UrlScrapedByKeyword[] $urls */
        foreach ($urls as $key => $url) {
            $urlScrapped = ltrim($url->getUrl(), './');
           // $urlScrapped =  $urlScrapped;

            $browser = $this->createBrowser();
            $page = $browser->createPage();
            $html = false;

            try {
                $page->navigate($urlScrapped, ['strict'])->waitForNavigation(Page::INTERACTIVE_TIME, 6000);
                $page->evaluate("console.log('document.title')");

               // -> here where my code crash so i catch the error below
                $pageTitle = $page->evaluate('document.title')->waitForResponse()->getReturnValue();

                if ($pageTitle == 'Before you continue')
                {
                    $this->AcceptGoogleCookies($page);
                    $pageTitle = $page->evaluate('document.title')->waitForResponse()->getReturnValue();
                } 

                echo($pageTitle.' from normal way');
                $pageContent = $page->getHtml(2500);
                sleep(1);
                if ($pageContent) echo('content OK');

                if ($pageTitle == 'Before you continue') $this->AcceptGoogleCookies($page);
            } catch (OperationTimedOut $e) {
                // Here in the console of the navigator, i can see this operation work correctly
               $page->evaluate("console.log(document.title)");

                // !!----catch the error and retry to evaluate title but again crash ----!!
                $pageTitle = $page->evaluate('document.title')->getReturnValue(); 

                if ($pageTitle == 'Before your continue') $this->AcceptGoogleCookies($page);

                echo $pageTitle.' from error';
                $pageContent = $page->getHtml(2500);
                sleep(1);
                if ($pageContent) echo('content OK from error');
            } catch (NavigationExpired $e) {
                echo "Erreur de NavigationExpired lors de l'évaluation du titre : $pageTitle</br>";
            }
        }`

[divinity76]

There is a bug in chrome-php/chrome (or maybe chrome-php/wrench)
That can cause anything to timeout, occasionally.

Even stupidly trivial stuff like
Page->evaluate("document.querySelectorAll("video")?.[0]?.src?.trim()")->getReturnValue()

Can timeout. That's an instant operation, it should take no time at all, but sometimes the return value never arrive.

I have been unable to find the source. Last time I went looking for it, I found chrome-php/wrench#24 but it didn't help.

I use this workaround:

private function evaluate(string $script): mixed
    {
        // there are random timeouts in the page evaluation.. probably an issue in the chrome-php/chrome library...
        try {
            return $this->page->evaluate($script)->getReturnValue();
        } catch (\HeadlessChromium\Exception\OperationTimedOut $e) {
            try {
                return $this->page->evaluate($script)->getReturnValue();
            } catch (\HeadlessChromium\Exception\OperationTimedOut $e) {
                return $this->page->evaluate($script)->getReturnValue();
            }
        }
    }

I would like to hear from other people who experience it 🤔

• it seems to happen most often on Windows, but it also happens on Linux.
• happens more often on laptops running on battery, less often on computers plugged in to the socket.

[prime-5506]

Thanks for sharing the issue and the code, . You're facing OperationTimedOut exceptions when trying to evaluate document.title with a headless browser (likely based on Chromium), while it works manually in the browser. This typically means there's a mismatch between what your script thinks the page state is vs. what the browser is actually doing.

Here’s a breakdown of why this might happen and how to fix or work around it:

---

✅ Understanding the Problem

1. Page not ready when evaluate() is called: Even after waitForNavigation, dynamic pages (especially behind JavaScript or cookie walls like Google’s) may delay setting document.title.

2. Protection mechanisms (like Google's):

Pages like Before you continue are often JavaScript-heavy consent pages.

These interfere with your browser automation because the actual content is hidden behind a consent interaction.

3. Incorrect use of waitForResponse():

You're calling evaluate()->waitForResponse()->getReturnValue() — but evaluate() often returns immediately unless you're awaiting a JS promise.

---

🔧 Suggestions for Fixing

1. Replace waitForNavigation(Page::INTERACTIVE_TIME, 6000)

This might be firing too early. Try:

$page->navigate($urlScrapped)->waitForLoad(Page::LOAD);

This waits until the page load event has fully fired. If the page is JS-heavy, INTERACTIVE_TIME might be too early.

---

2. Wait for document.title dynamically

Instead of immediately calling evaluate('document.title'), use a loop to poll the title until it changes from 'Before you continue' or is not empty.

$maxTries = 5;
$waitedTitle = null;
for ($i = 0; $i < $maxTries; $i++) {
try {
$waitedTitle = $page->evaluate('document.title')->getReturnValue();
if (!empty($waitedTitle) && $waitedTitle !== 'Before you continue') {
break;
}
} catch (OperationTimedOut $e) {
// You could optionally retry here
}
sleep(1); // wait between retries
}

---

3. Use waitForFunction() to block until document.title is correct

Some headless libraries support this:

$page->waitForFunction('document.title && document.title !== "Before you continue"', 5000);

If your library supports something like waitForFunction(), it’s more reliable than catching timeouts.

---

4. Fallback evaluation in catch block

In your catch (OperationTimedOut) block, you retry with:

$pageTitle = $page->evaluate('document.title')->getReturnValue();

But this may also fail. Wrap this retry in its own try {} block to catch nested errors.

---

5. Log full HTML for debugging

To inspect what the browser sees when it fails:

file_put_contents("debug.html", $page->getHtml());

Then open debug.html in your own browser — this helps diagnose if you're seeing a consent page, captcha, or blank.

---

🧪 Optional Debug Tip

Add this to log page console output if supported:

$browser->on('console', function ($message) {
echo "Console: " . $message->text() . "\n";
});

---

✅ Summary of Fixes

Use waitForLoad() instead of waitForNavigation(INTERACTIVE_TIME)

Poll document.title with retries/sleep

Wrap retry logic inside its own try block

Log page HTML to diagnose consent/captcha walls

If possible, implement waitForFunction('document.title !== "Before you continue"')

Your welcome..🛡😊

[hopcv58]

Is there any update on this? I faced this problem for a long time, and started to think about returning to puppeteer for consistency

[divinity76]

I suggest looking very closely at the networking code. I think messages from the browser to php get lost in transmission, rarely, causing these timeouts.

[divinity76]

Going by the reproduce code in #560
Where a usleep(1) triggers it, there's probably a race condition involved.

(Btw #560 is incorrectly closed by the timeout bot for inactivity. It's not a solved issue.)

[roben]

I also have these issues running Linux. I try to crawl a web page with an infinite loading content list. My injected javacript does it's thing on the web page and php-chrome waits in a loop for the result variable to be populated. After about 2 minutes (randomly more or less), the timeout occurs. It is not recoverable by trying again, I have to reconnect to the browser.

Strangely enough, I initially forgot a sleep in this loop so it checked reeaally often and quickly, so I thought that to be the issue. But even after adding a sleep(1), it still times out after roughly the same duration. So it also seems to depend on the duration of the connection and not the check frequency (alone).

[roben]

I fixed it for now by filtering / chunking the result set to produce smaller responses. Not very satisfying, though...

[stale]

This issue has been automatically marked as stale because there has been no recent activity. It will be closed after 30 days if no further activity occurs. Thank you for your contributions.