Use of OS (ie my) installed Chrome browser as managed browser using CDP #1344
Unanswered
helicalchris
asked this question in
Forums - Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have danced around this and every variant I can find within crawl4ai to connect to my Chrome, not a Chromium it starts. I know playwright can do it, and its part of mainstream CDP use, but I cannot for the life of life of me work out how in crawl4ai to point at an existing Chrome instance and use that. I specifically want this because the site I am crawling is very very sensitive to bots and I need to have regular interaction with it, not just a recorded profile. Otherwise as soon as I do deep-crawling I get booted off it.
I can set CDP endpoints using crwl or in code, but the only time I got it to even faintly work is when I used crwl, which killed my Chrome instance and started its own Chromium! Otherwise it seems to say its working but I see no interaction with Chrome so I think that actually its starting a headless chromium to scrape and returning that result and totally ignoring the CDP URLs I provide (which is similar to the behaviour I see with crwl)
Is it possible? If not, could it be, as I can see in the code that connect_with_cdp is being used, and its all playwright based, so its very much feasible to I believe...
Chris
Beta Was this translation helpful? Give feedback.
All reactions