Target page, context or browser has been closed on single FastAPI request processing

[Dopamine-Max]

I am currently try to crawl multiple pages parallely for my LLM knowledge base using crawl4ai using an API endpoint from fast API. But after serving a single multi page crawl request. I get the following error "BrowserType.launch: Target page, context or browser has been closed". I could not seem to find a way to keep the page open or restart the API after crawl. Feel free to ask for any clarifications!

Here is my FastAPI code:

`
from fastapi import FastAPI
from contextlib import asynccontextmanager
from pydantic import BaseModel
from crawler import crawl_parallel
import asyncio
import uvicorn

@asynccontextmanager
async def lifespan(app: FastAPI):
# Initialize per-app state here
yield
# Cleanup after all requests

app = FastAPI(lifespan=lifespan)

class SearchRequest(BaseModel):
query: str

@app.post("/web-search")
async def web_search(request: SearchRequest):
try:
return await asyncio.wait_for(
crawl_parallel(request.query),
timeout=20
)
except asyncio.TimeoutError:
return {"results": []}

if name == "main":
uvicorn.run(app, host="0.0.0.0", port=8000)
`

[egehakan]

Have the same exact error in FastAPI.

[aravindkarnam]

@Dopamine-Max @egehakan There's a bug reported for this. Already patched in bug fix branch. This will be merged soon!

[Dopamine-Max]

Thanks! just checked out the bug report your solution makes sense for the issue I am facing. Hope to see it merged soon.