fetching DemocracyClub/ONS-Geography-Releases-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 3)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-sl2b8v70 Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-sl2b8v70 Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.1) running job DemocracyClub/ONS-Geography-Releases-Scraper ... Traceback (most recent call last): File "scraper.py", line 128, in scrape(lgd_url, "lgd") File "scraper.py", line 84, in scrape res.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=geoportal1-ons.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=LGD+Boundaries&sort=-updatedAt fetching DemocracyClub/Ubuntu-AMI-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 6)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-8acogxqd Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-8acogxqd Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: json5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.16) Collecting json5 Using cached json5-0.9.16-py2.py3-none-any.whl (19 kB) Using cached json5-0.9.15-py2.py3-none-any.whl (19 kB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.1) running job DemocracyClub/Ubuntu-AMI-Scraper ... {'date': '20260218', 'ami_id': 'ami-090f0cf86291c77fa', 'version': '22.04 LTS', 'zone': 'eu-west-2', 'cpu_arch': 'amd64', 'instance_type': 'hvm:ebs-ssd'} {'status': '404', 'message': 'Not Found', 'documentation_url': 'https://docs.github.com/rest/git/refs#create-a-reference'} Traceback (most recent call last): File "scraper.py", line 127, in scrape() File "scraper.py", line 119, in scrape open_pull_request(repo, record) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/Ubuntu-AMI-Scraper/github.py", line 18, in open_pull_request g.create_branch(newbranch) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 111, in create_branch return self._request("POST", url, payload) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 39, in _request r.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/DemocracyClub/who_deploy/git/refs fetching DemocracyClub/womblr ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 4)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-agoiy03a Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-agoiy03a Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.9.0.post0) Collecting python-dateutil Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached python_dateutil-2.9.0-py2.py3-none-any.whl (230 kB) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: uk-election-timetables==2.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.3.0) Collecting uk-election-timetables==2.3.0 Using cached uk_election_timetables-2.3.0-py3-none-any.whl Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.1) running job DemocracyClub/womblr ... Nothing to do today, but here's the results from the last run.. ===== :satellite_antenna: *By-Election update!* :ballot_box_with_ballot: 19/02/2026: . known candidates: 5 :lock: 05/03/2026: . known candidates: 5 :lock: 05/03/2026: . known candidates: 5 :lock: 05/03/2026: . known candidates: 5 :lock: 05/03/2026: . known candidates: 6 :lock: 05/03/2026: . known candidates: 4 :lock: 12/03/2026: . known candidates: 5 :lock: 12/03/2026: . known candidates: 6 :lock: 12/03/2026: . known candidates: 7 :lock: 12/03/2026: . known candidates: 4 :lock: 17/03/2026: . known candidates: 0 :womble: required (SoPN should be published) fetching DemocracyClub/boundary-bot ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 8)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-f5_4v8si Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-f5_4v8si Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: cssselect==0.9.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.1) Collecting cssselect==0.9.1 Using cached cssselect-0.9.1.tar.gz (32 kB) Requirement already satisfied: rapidfuzz==0.2.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.2.2) Collecting rapidfuzz==0.2.2 Using cached rapidfuzz-0.2.2-cp35-cp35m-manylinux2010_x86_64.whl (1.3 MB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 6)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: scrapy==1.8.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 7)) (1.8.1) Collecting scrapy==1.8.1 Using cached Scrapy-1.8.1-py2.py3-none-any.whl (239 kB) Requirement already satisfied: pybind11>=2.4 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from rapidfuzz==0.2.2->-r requirements.txt (line 3)) (2.4.3) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.4.2) Requirement already satisfied: zope.interface>=4.1.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (5.0.1) Requirement already satisfied: cryptography>=2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.8) Requirement already satisfied: pyOpenSSL>=16.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: PyDispatcher>=2.0.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.5) Requirement already satisfied: service-identity>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: Twisted>=17.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (20.3.0) Requirement already satisfied: w3lib>=1.17.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.21.0) Requirement already satisfied: protego>=0.1.15 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (0.1.16) Requirement already satisfied: parsel>=1.5.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.2) Requirement already satisfied: queuelib>=1.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2.9) Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (1.14.0) Requirement already satisfied: pycparser in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: pyasn1-modules in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.2.8) Requirement already satisfied: pyasn1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.4.8) Requirement already satisfied: attrs>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.3.0) Requirement already satisfied: PyHamcrest!=1.10.0,>=1.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.2) Requirement already satisfied: Automat>=0.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: constantly>=15.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (15.1.0) Requirement already satisfied: hyperlink>=17.1.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.0.0) Requirement already satisfied: incremental>=16.10.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (17.5.0) Requirement already satisfied: setuptools in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from zope.interface>=4.1.3->scrapy==1.8.1->-r requirements.txt (line 7)) (46.1.1) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.1) running job DemocracyClub/boundary-bot ... Traceback (most recent call last): File "scraper.py", line 17, in scraper.scrape() File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 261, in scrape self.parse_index(self.scrape_index()) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 79, in parse_index % (str(found_headings), str(expected_headings)) boundary_bot.scraper.ScraperException: Unexpected headings: Found [], expected ['Current Reviews', 'Recent Reviews'] fetching DemocracyClub/GDSRegisters ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2019.11.28) running job DemocracyClub/GDSRegisters ... Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 66, in for name in get_all_register_names(): File "scraper.py", line 35, in get_all_register_names req = requests.get(make_url("register")) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) fetching wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-2rktdhdg Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-2rktdhdg Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... found 97 stations 97 stations in database found 99 districts 99 districts in database fetching wdiv-scrapers/DC-PollingStations-Barnet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-j52p1pj_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-j52p1pj_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Barnet ... Traceback (most recent call last): File "scraper.py", line 21, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 33, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 23, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Bristol ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-f8l39kkx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-f8l39kkx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Bristol ... https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_community_and_safety/MapServer/1/query?outFields=*&where=1%3D1&f=geojson found 168 stations 168 stations in database https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_boundaries/MapServer/3/query?outFields=*&where=1%3D1&f=geojson found 240 districts 240 districts in database fetching wdiv-scrapers/DC-PollingStations-Broxtowe ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-bx94q1r_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-bx94q1r_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Broxtowe ... Traceback (most recent call last): File "scraper.py", line 13, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Calderdale ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-osk8aigq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-osk8aigq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Calderdale ... Traceback (most recent call last): File "scraper.py", line 47, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Camden ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-w2dj04_l Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-w2dj04_l Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Camden ... Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Canterbury ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ubtrwcmq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ubtrwcmq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Canterbury ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1288, in connect server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 752, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 117, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3m3zv8ux Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3m3zv8ux Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Doncaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_j3_zv66 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_j3_zv66 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Doncaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Dover ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-w6rcy8u3 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-w6rcy8u3 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dover ... found 69 stations 69 stations in database found 69 districts 69 districts in database fetching wdiv-scrapers/DC-PollingStations-Dundee ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-r7lx7564 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-r7lx7564 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dundee ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized fetching wdiv-scrapers/DC-PollingStations-Edinburgh ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-5864a6kv Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-5864a6kv Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Edinburgh ... found 158 stations 158 stations in database found 158 districts 158 districts in database fetching wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-__x_9zmi Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-__x_9zmi Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... found 35 stations 35 stations in database found 25 districts 25 districts in database fetching wdiv-scrapers/DC-PollingStations-Fife ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0kovyhkg Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0kovyhkg Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Fife ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Guildford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-11imurw3 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-11imurw3 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Guildford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 40, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 58, in scrape data_str = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 35, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Hackney ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-gqr3v0bk Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-gqr3v0bk Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Hackney ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-wf8pr4aq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-wf8pr4aq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Kingston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-uoi80uso Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-uoi80uso Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Kingston ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Lambeth ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-gohrlt5v Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-gohrlt5v Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lambeth ... Traceback (most recent call last): File "scraper.py", line 12, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Lancaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-sv04_afq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-sv04_afq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lancaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Moray ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-h8ry7hy7 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-h8ry7hy7 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Moray ... found 60 stations 60 stations in database found 60 districts 60 districts in database fetching wdiv-scrapers/DC-PollingStations-Nottingham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ym780ivo Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ym780ivo Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Nottingham ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Pembrokeshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-y2y5m_if Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-y2y5m_if Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Pembrokeshire ... found 106 stations 106 stations in database found 135 districts 135 districts in database fetching wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-p13fcqnz Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-p13fcqnz Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Preston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-v8kmqzgm Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-v8kmqzgm Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Preston ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Salford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-9hyft4_4 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-9hyft4_4 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Salford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 62, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/ckan_scraper.py", line 33, in scrape data_str, data = self.get_data() File "scraper.py", line 24, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "scraper.py", line 16, in get_data_from_url with urllib.request.urlopen(url, timeout=300, context=context) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Sevenoaks ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-v85woz08 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-v85woz08 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Sevenoaks ... Traceback (most recent call last): File "scraper.py", line 8, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Southampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-jafh4dtx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-jafh4dtx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Southampton ... fetching wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-1b2vna9z Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-1b2vna9z Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Traceback (most recent call last): File "scraper.py", line 12, in search_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-StAlbans ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-14n6fs0l Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-14n6fs0l Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-StAlbans ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -5] No address associated with hostname During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Thanet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-mghpuaca Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-mghpuaca Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Thanet ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3xx4_a9r Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3xx4_a9r Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1264, in do_open r = h.getresponse() File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response fetching wdiv-scrapers/DC-PollingStations-TunbridgeWells ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-1r_lurax Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-1r_lurax Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TunbridgeWells ... https://opendata.arcgis.com/datasets/ea6894c98dfb4cf99775d1fb949cf0de_6.geojson found 53 stations 53 stations in database https://opendata.arcgis.com/datasets/24b6d662cd1145eba4743d27e91c0436_5.geojson found 51 districts 51 districts in database fetching wdiv-scrapers/DC-PollingStations-Uttlesford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_0eq1oza Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_0eq1oza Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Uttlesford ... found 79 districts 79 districts in database fetching wdiv-scrapers/DC-PollingStations-Wolverhampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-mz52s7ot Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-mz52s7ot Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Wolverhampton ... found 104 stations 104 stations in database found 125 districts 125 districts in database fetching wdiv-scrapers/DC-PollingStations-York ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-1ktj_xuy Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-1ktj_xuy Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-York ... found 86 stations 86 stations in database found 92 districts Traceback (most recent call last): File "scraper.py", line 55, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 56, in scrape record['pk'] = feature['properties'][self.key] KeyError: 'OBJECTID_1' Successfully ran DemocracyClub/womblr Successfully ran wdiv-scrapers/DC-PollingStations-ArgyllAndBute Successfully ran wdiv-scrapers/DC-PollingStations-Bristol Successfully ran wdiv-scrapers/DC-PollingStations-Dover Successfully ran wdiv-scrapers/DC-PollingStations-Edinburgh Successfully ran wdiv-scrapers/DC-PollingStations-EpsomAndEwell Successfully ran wdiv-scrapers/DC-PollingStations-Moray Successfully ran wdiv-scrapers/DC-PollingStations-Pembrokeshire Successfully ran wdiv-scrapers/DC-PollingStations-Southampton Successfully ran wdiv-scrapers/DC-PollingStations-TunbridgeWells Successfully ran wdiv-scrapers/DC-PollingStations-Uttlesford Successfully ran wdiv-scrapers/DC-PollingStations-Wolverhampton Failed to run DemocracyClub/ONS-Geography-Releases-Scraper Failed to run DemocracyClub/Ubuntu-AMI-Scraper Failed to run DemocracyClub/boundary-bot Failed to run DemocracyClub/GDSRegisters Failed to run wdiv-scrapers/DC-PollingStations-Barnet Failed to run wdiv-scrapers/DC-PollingStations-Broxtowe Failed to run wdiv-scrapers/DC-PollingStations-Calderdale Failed to run wdiv-scrapers/DC-PollingStations-Camden Failed to run wdiv-scrapers/DC-PollingStations-Canterbury Failed to run wdiv-scrapers/DC-PollingStations-Carmarthenshire Failed to run wdiv-scrapers/DC-PollingStations-Doncaster Failed to run wdiv-scrapers/DC-PollingStations-Dundee Failed to run wdiv-scrapers/DC-PollingStations-Fife Failed to run wdiv-scrapers/DC-PollingStations-Guildford Failed to run wdiv-scrapers/DC-PollingStations-Hackney Failed to run wdiv-scrapers/DC-PollingStations-HammersmithAndFulham Failed to run wdiv-scrapers/DC-PollingStations-Kingston Failed to run wdiv-scrapers/DC-PollingStations-Lambeth Failed to run wdiv-scrapers/DC-PollingStations-Lancaster Failed to run wdiv-scrapers/DC-PollingStations-Nottingham Failed to run wdiv-scrapers/DC-PollingStations-PerthAndKinross Failed to run wdiv-scrapers/DC-PollingStations-Preston Failed to run wdiv-scrapers/DC-PollingStations-Salford Failed to run wdiv-scrapers/DC-PollingStations-Sevenoaks Failed to run wdiv-scrapers/DC-PollingStations-SouthAyrshire Failed to run wdiv-scrapers/DC-PollingStations-StAlbans Failed to run wdiv-scrapers/DC-PollingStations-Thanet Failed to run wdiv-scrapers/DC-PollingStations-TonbridgeMalling Failed to run wdiv-scrapers/DC-PollingStations-York