Scrapy Selenium #13
Description
In my scrapy app, I use scrapy -selenium with geckodriver. When I try use command 'scrapy-deploy', I get an error:
file "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}
What can I do to deploy app with selenium?
All track for error is here:
Server response (200):
{"node_name": "cbb42d40-da02-4d65-bb04-ff436c5aacc1", "status": "error", "message": "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/project.py:94: ScrapyDeprecationWarning: Use of environment variables prefixed with SCRAPY_ to override settings is deprecated. The following environment variables are currently defined: EGG_VERSION\n ScrapyDeprecationWarning\nTraceback (most recent call last):\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n "main", mod_spec)\n File "/app/.heroku/python/lib/python3.6/runpy.py", line 85, in _run_code\n exec(code, run_globals)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 40, in \n main()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapyd/runner.py", line 37, in main\n execute()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/cmdline.py", line 142, in execute\n cmd.crawler_process = CrawlerProcess(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 280, in init\n super(CrawlerProcess, self).init(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 152, in init\n self.spider_loader = self._get_spider_loader(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/crawler.py", line 146, in _get_spider_loader\n return loader_cls.from_settings(settings.frozencopy())\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 60, in from_settings\n return cls(settings)\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 24, in init\n self._load_all_spiders()\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/spiderloader.py", line 46, in _load_all_spiders\n for module in walk_modules(name):\n File "/app/.heroku/python/lib/python3.6/site-packages/scrapy/utils/misc.py", line 77, in walk_modules\n submod = import_module(fullpath)\n File "/app/.heroku/python/lib/python3.6/importlib/init.py", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File "", line 994, in _gcd_import\n File "", line 971, in _find_and_load\n File "", line 955, in _find_and_load_unlocked\n File "", line 656, in _load_unlocked\n File "", line 626, in _load_backward_compatible\n File "/tmp/scrapy_job_it-1590437526-tezy3xue.egg/scrapy_job_it/spiders/bdcrawler.py", line 3, in \nModuleNotFoundError: No module named 'scrapy_selenium'\n"}