fetching DemocracyClub/ONS-Geography-Releases-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 3)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-ioktoet6 Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-ioktoet6 Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.1) running job DemocracyClub/ONS-Geography-Releases-Scraper ... Traceback (most recent call last): File "scraper.py", line 128, in scrape(lgd_url, "lgd") File "scraper.py", line 84, in scrape res.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=geoportal1-ons.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=LGD+Boundaries&sort=-updatedAt fetching DemocracyClub/Ubuntu-AMI-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 6)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-4jv9t7q8 Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-4jv9t7q8 Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: json5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.16) Collecting json5 Using cached json5-0.9.16-py2.py3-none-any.whl (19 kB) Using cached json5-0.9.15-py2.py3-none-any.whl (19 kB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.1) running job DemocracyClub/Ubuntu-AMI-Scraper ... {'zone': 'eu-west-2', 'instance_type': 'hvm:ebs-ssd', 'version': '22.04 LTS', 'date': '20260410', 'ami_id': 'ami-052c9005e24cd7236', 'cpu_arch': 'amd64'} {'documentation_url': 'https://docs.github.com/rest/git/refs#create-a-reference', 'message': 'Not Found', 'status': '404'} Traceback (most recent call last): File "scraper.py", line 127, in scrape() File "scraper.py", line 119, in scrape open_pull_request(repo, record) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/Ubuntu-AMI-Scraper/github.py", line 18, in open_pull_request g.create_branch(newbranch) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 111, in create_branch return self._request("POST", url, payload) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 39, in _request r.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/DemocracyClub/who_deploy/git/refs fetching DemocracyClub/womblr ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 4)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-yrsfswr9 Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-yrsfswr9 Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.9.0.post0) Collecting python-dateutil Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached python_dateutil-2.9.0-py2.py3-none-any.whl (230 kB) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: uk-election-timetables==2.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.3.0) Collecting uk-election-timetables==2.3.0 Using cached uk_election_timetables-2.3.0-py3-none-any.whl Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.1) running job DemocracyClub/womblr ... Nothing to do today, but here's the results from the last run.. ===== :satellite_antenna: *By-Elections coming up this month* :alarm_clock: 23/04/2026: . known candidates: 6 :lock: 30/04/2026: . known candidates: 4 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 9 :lock: 07/05/2026: . known candidates: 6 :lock: 07/05/2026: . known candidates: 7 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 5 :lock: 07/05/2026: . known candidates: 6 :lock: 21/05/2026: . known candidates: 0 :womble: required (SoPN due 27/04/2026) 21/05/2026: . known candidates: 1 (SoPN due 27/04/2026) 21/05/2026: . known candidates: 1 (SoPN due 27/04/2026) fetching DemocracyClub/boundary-bot ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 8)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-uw1nnlki Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-uw1nnlki Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: cssselect==0.9.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.1) Collecting cssselect==0.9.1 Using cached cssselect-0.9.1.tar.gz (32 kB) Requirement already satisfied: rapidfuzz==0.2.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.2.2) Collecting rapidfuzz==0.2.2 Using cached rapidfuzz-0.2.2-cp35-cp35m-manylinux2010_x86_64.whl (1.3 MB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 6)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: scrapy==1.8.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 7)) (1.8.1) Collecting scrapy==1.8.1 Using cached Scrapy-1.8.1-py2.py3-none-any.whl (239 kB) Requirement already satisfied: pybind11>=2.4 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from rapidfuzz==0.2.2->-r requirements.txt (line 3)) (2.4.3) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.3.15) Requirement already satisfied: zope.interface>=4.1.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (5.0.1) Requirement already satisfied: Twisted>=17.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (20.3.0) Requirement already satisfied: w3lib>=1.17.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.21.0) Requirement already satisfied: cryptography>=2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.8) Requirement already satisfied: protego>=0.1.15 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (0.1.16) Requirement already satisfied: pyOpenSSL>=16.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: parsel>=1.5.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.2) Requirement already satisfied: service-identity>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: queuelib>=1.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.0) Requirement already satisfied: PyDispatcher>=2.0.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.5) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (3.0.4) Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (1.14.0) Requirement already satisfied: pycparser in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: pyasn1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.4.8) Requirement already satisfied: pyasn1-modules in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.2.8) Requirement already satisfied: attrs>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.3.0) Requirement already satisfied: incremental>=16.10.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (17.5.0) Requirement already satisfied: Automat>=0.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: constantly>=15.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (15.1.0) Requirement already satisfied: hyperlink>=17.1.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.0.0) Requirement already satisfied: PyHamcrest!=1.10.0,>=1.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.2) Requirement already satisfied: setuptools in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from zope.interface>=4.1.3->scrapy==1.8.1->-r requirements.txt (line 7)) (46.1.1) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.1) running job DemocracyClub/boundary-bot ... Traceback (most recent call last): File "scraper.py", line 17, in scraper.scrape() File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 261, in scrape self.parse_index(self.scrape_index()) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 79, in parse_index % (str(found_headings), str(expected_headings)) boundary_bot.scraper.ScraperException: Unexpected headings: Found [], expected ['Current Reviews', 'Recent Reviews'] fetching DemocracyClub/GDSRegisters ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2019.11.28) running job DemocracyClub/GDSRegisters ... Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 66, in for name in get_all_register_names(): File "scraper.py", line 35, in get_all_register_names req = requests.get(make_url("register")) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) fetching wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-gw8vhy6j Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-gw8vhy6j Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... found 97 stations 97 stations in database found 99 districts 99 districts in database fetching wdiv-scrapers/DC-PollingStations-Barnet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-pfyu9dob Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-pfyu9dob Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Barnet ... Traceback (most recent call last): File "scraper.py", line 21, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 33, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 23, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Bristol ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-vd2uzzo7 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-vd2uzzo7 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Bristol ... https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_community_and_safety/MapServer/1/query?outFields=*&where=1%3D1&f=geojson found 168 stations 168 stations in database https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_boundaries/MapServer/3/query?outFields=*&where=1%3D1&f=geojson found 240 districts 240 districts in database fetching wdiv-scrapers/DC-PollingStations-Broxtowe ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-eg22fea1 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-eg22fea1 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Broxtowe ... Traceback (most recent call last): File "scraper.py", line 13, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Calderdale ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-s6l5t6mq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-s6l5t6mq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Calderdale ... Traceback (most recent call last): File "scraper.py", line 47, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Camden ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-toz89t0i Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-toz89t0i Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Camden ... Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Canterbury ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-zvm35iz6 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-zvm35iz6 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Canterbury ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1288, in connect server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 752, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 117, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-a86tjrfm Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-a86tjrfm Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Doncaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-9cjsza_j Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-9cjsza_j Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Doncaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Dover ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-4i1lxssa Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-4i1lxssa Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dover ... found 69 stations 69 stations in database found 69 districts 69 districts in database fetching wdiv-scrapers/DC-PollingStations-Dundee ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-vlkvm1wj Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-vlkvm1wj Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dundee ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized fetching wdiv-scrapers/DC-PollingStations-Edinburgh ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-__nvktjz Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-__nvktjz Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Edinburgh ... found 158 stations 158 stations in database found 158 districts 158 districts in database fetching wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-nivgeg38 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-nivgeg38 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... found 37 stations 37 stations in database found 26 districts 26 districts in database fetching wdiv-scrapers/DC-PollingStations-Fife ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-swz8p_dx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-swz8p_dx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Fife ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Guildford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3eat9od9 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3eat9od9 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Guildford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 40, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 58, in scrape data_str = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 35, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Hackney ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_8vh0mf4 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_8vh0mf4 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Hackney ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-001z2sro Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-001z2sro Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... found 71 stations 71 stations in database found 71 districts 71 districts in database fetching wdiv-scrapers/DC-PollingStations-Kingston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-8dervj67 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-8dervj67 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Kingston ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Lambeth ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-42em6_of Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-42em6_of Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lambeth ... Traceback (most recent call last): File "scraper.py", line 12, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Lancaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-7bwdpj23 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-7bwdpj23 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lancaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 504: Gateway Time-out fetching wdiv-scrapers/DC-PollingStations-Moray ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-zsqbl3dv Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-zsqbl3dv Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Moray ... found 60 stations 60 stations in database found 60 districts 60 districts in database fetching wdiv-scrapers/DC-PollingStations-Nottingham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-icaaiju8 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-icaaiju8 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Nottingham ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Pembrokeshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-tyyo_mjd Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-tyyo_mjd Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Pembrokeshire ... found 106 stations 106 stations in database found 135 districts 135 districts in database fetching wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-g4uddxk8 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-g4uddxk8 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Preston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-d98zkx6_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-d98zkx6_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Preston ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Salford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-wzac8hf_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-wzac8hf_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Salford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 62, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/ckan_scraper.py", line 33, in scrape data_str, data = self.get_data() File "scraper.py", line 24, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "scraper.py", line 16, in get_data_from_url with urllib.request.urlopen(url, timeout=300, context=context) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Sevenoaks ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-odthsb72 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-odthsb72 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Sevenoaks ... Traceback (most recent call last): File "scraper.py", line 8, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Southampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-jwk2ltnf Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-jwk2ltnf Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Southampton ... fetching wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-1xtdhfud Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-1xtdhfud Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Traceback (most recent call last): File "scraper.py", line 12, in search_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-StAlbans ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3010cjsh Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3010cjsh Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-StAlbans ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -5] No address associated with hostname During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Thanet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-b969vfdo Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-b969vfdo Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Thanet ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-5czrjqsw Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-5czrjqsw Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1264, in do_open r = h.getresponse() File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response fetching wdiv-scrapers/DC-PollingStations-TunbridgeWells ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kolte4ym Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kolte4ym Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TunbridgeWells ... https://opendata.arcgis.com/datasets/ea6894c98dfb4cf99775d1fb949cf0de_6.geojson found 53 stations 53 stations in database https://opendata.arcgis.com/datasets/24b6d662cd1145eba4743d27e91c0436_5.geojson found 51 districts 51 districts in database fetching wdiv-scrapers/DC-PollingStations-Uttlesford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-renz2ffp Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-renz2ffp Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Uttlesford ... found 79 districts 79 districts in database fetching wdiv-scrapers/DC-PollingStations-Wolverhampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-dpovf5wz Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-dpovf5wz Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Wolverhampton ... found 104 stations 104 stations in database found 125 districts 125 districts in database fetching wdiv-scrapers/DC-PollingStations-York ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0oh60uft Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0oh60uft Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-York ... found 86 stations 86 stations in database found 92 districts Traceback (most recent call last): File "scraper.py", line 55, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 56, in scrape record['pk'] = feature['properties'][self.key] KeyError: 'OBJECTID_1' Successfully ran DemocracyClub/womblr Successfully ran wdiv-scrapers/DC-PollingStations-ArgyllAndBute Successfully ran wdiv-scrapers/DC-PollingStations-Bristol Successfully ran wdiv-scrapers/DC-PollingStations-Dover Successfully ran wdiv-scrapers/DC-PollingStations-Edinburgh Successfully ran wdiv-scrapers/DC-PollingStations-EpsomAndEwell Successfully ran wdiv-scrapers/DC-PollingStations-HammersmithAndFulham Successfully ran wdiv-scrapers/DC-PollingStations-Moray Successfully ran wdiv-scrapers/DC-PollingStations-Pembrokeshire Successfully ran wdiv-scrapers/DC-PollingStations-Southampton Successfully ran wdiv-scrapers/DC-PollingStations-TunbridgeWells Successfully ran wdiv-scrapers/DC-PollingStations-Uttlesford Successfully ran wdiv-scrapers/DC-PollingStations-Wolverhampton Failed to run DemocracyClub/ONS-Geography-Releases-Scraper Failed to run DemocracyClub/Ubuntu-AMI-Scraper Failed to run DemocracyClub/boundary-bot Failed to run DemocracyClub/GDSRegisters Failed to run wdiv-scrapers/DC-PollingStations-Barnet Failed to run wdiv-scrapers/DC-PollingStations-Broxtowe Failed to run wdiv-scrapers/DC-PollingStations-Calderdale Failed to run wdiv-scrapers/DC-PollingStations-Camden Failed to run wdiv-scrapers/DC-PollingStations-Canterbury Failed to run wdiv-scrapers/DC-PollingStations-Carmarthenshire Failed to run wdiv-scrapers/DC-PollingStations-Doncaster Failed to run wdiv-scrapers/DC-PollingStations-Dundee Failed to run wdiv-scrapers/DC-PollingStations-Fife Failed to run wdiv-scrapers/DC-PollingStations-Guildford Failed to run wdiv-scrapers/DC-PollingStations-Hackney Failed to run wdiv-scrapers/DC-PollingStations-Kingston Failed to run wdiv-scrapers/DC-PollingStations-Lambeth Failed to run wdiv-scrapers/DC-PollingStations-Lancaster Failed to run wdiv-scrapers/DC-PollingStations-Nottingham Failed to run wdiv-scrapers/DC-PollingStations-PerthAndKinross Failed to run wdiv-scrapers/DC-PollingStations-Preston Failed to run wdiv-scrapers/DC-PollingStations-Salford Failed to run wdiv-scrapers/DC-PollingStations-Sevenoaks Failed to run wdiv-scrapers/DC-PollingStations-SouthAyrshire Failed to run wdiv-scrapers/DC-PollingStations-StAlbans Failed to run wdiv-scrapers/DC-PollingStations-Thanet Failed to run wdiv-scrapers/DC-PollingStations-TonbridgeMalling Failed to run wdiv-scrapers/DC-PollingStations-York