fetching DemocracyClub/ONS-Geography-Releases-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 3)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-svx6l46i Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-svx6l46i Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.1) running job DemocracyClub/ONS-Geography-Releases-Scraper ... Traceback (most recent call last): File "scraper.py", line 128, in scrape(lgd_url, "lgd") File "scraper.py", line 84, in scrape res.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=geoportal1-ons.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=LGD+Boundaries&sort=-updatedAt fetching DemocracyClub/Ubuntu-AMI-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 6)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-tgqshqbp Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-tgqshqbp Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: json5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.16) Collecting json5 Using cached json5-0.9.16-py2.py3-none-any.whl (19 kB) Using cached json5-0.9.15-py2.py3-none-any.whl (19 kB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.1) running job DemocracyClub/Ubuntu-AMI-Scraper ... {'zone': 'eu-west-2', 'date': '20260410', 'ami_id': 'ami-052c9005e24cd7236', 'cpu_arch': 'amd64', 'version': '22.04 LTS', 'instance_type': 'hvm:ebs-ssd'} {'message': 'Not Found', 'documentation_url': 'https://docs.github.com/rest/git/refs#create-a-reference', 'status': '404'} Traceback (most recent call last): File "scraper.py", line 127, in scrape() File "scraper.py", line 119, in scrape open_pull_request(repo, record) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/Ubuntu-AMI-Scraper/github.py", line 18, in open_pull_request g.create_branch(newbranch) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 111, in create_branch return self._request("POST", url, payload) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 39, in _request r.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/DemocracyClub/who_deploy/git/refs fetching DemocracyClub/womblr ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 4)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-n22p778z Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-n22p778z Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.9.0.post0) Collecting python-dateutil Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached python_dateutil-2.9.0-py2.py3-none-any.whl (230 kB) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: uk-election-timetables==2.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.3.0) Collecting uk-election-timetables==2.3.0 Using cached uk_election_timetables-2.3.0-py3-none-any.whl Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.1) running job DemocracyClub/womblr ... Nothing to do today, but here's the results from the last run.. ===== :alarm_clock: *By-Elections coming up this month* :alarm_clock: 09/04/2026: . known candidates: 6 :lock: 16/04/2026: . known candidates: 6 :lock: 16/04/2026: . known candidates: 6 :lock: 22/04/2026: . known candidates: 6 :lock: 23/04/2026: . known candidates: 6 :lock: 30/04/2026: . known candidates: 4 :lock: 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 1 (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 3 (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 1 (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 1 (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 0 :womble: required (SoPN should be published) 07/05/2026: . known candidates: 4 (SoPN should be published) 07/05/2026: . known candidates: 1 (SoPN should be published) fetching DemocracyClub/boundary-bot ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 8)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-ps8if4kt Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-ps8if4kt Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: cssselect==0.9.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.1) Collecting cssselect==0.9.1 Using cached cssselect-0.9.1.tar.gz (32 kB) Requirement already satisfied: rapidfuzz==0.2.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.2.2) Collecting rapidfuzz==0.2.2 Using cached rapidfuzz-0.2.2-cp35-cp35m-manylinux2010_x86_64.whl (1.3 MB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 6)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: scrapy==1.8.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 7)) (1.8.1) Collecting scrapy==1.8.1 Using cached Scrapy-1.8.1-py2.py3-none-any.whl (239 kB) Requirement already satisfied: pybind11>=2.4 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from rapidfuzz==0.2.2->-r requirements.txt (line 3)) (2.4.3) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.3.15) Requirement already satisfied: protego>=0.1.15 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (0.1.16) Requirement already satisfied: Twisted>=17.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (20.3.0) Requirement already satisfied: w3lib>=1.17.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.21.0) Requirement already satisfied: pyOpenSSL>=16.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: PyDispatcher>=2.0.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.5) Requirement already satisfied: zope.interface>=4.1.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (5.0.1) Requirement already satisfied: parsel>=1.5.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.2) Requirement already satisfied: queuelib>=1.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.0) Requirement already satisfied: service-identity>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: cryptography>=2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2019.11.28) Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (1.14.0) Requirement already satisfied: pycparser in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: attrs>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.3.0) Requirement already satisfied: pyasn1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.4.8) Requirement already satisfied: pyasn1-modules in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.2.8) Requirement already satisfied: PyHamcrest!=1.10.0,>=1.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.2) Requirement already satisfied: constantly>=15.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (15.1.0) Requirement already satisfied: Automat>=0.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: incremental>=16.10.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (17.5.0) Requirement already satisfied: hyperlink>=17.1.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.0.0) Requirement already satisfied: setuptools in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from zope.interface>=4.1.3->scrapy==1.8.1->-r requirements.txt (line 7)) (46.1.1) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.1) running job DemocracyClub/boundary-bot ... Traceback (most recent call last): File "scraper.py", line 17, in scraper.scrape() File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 261, in scrape self.parse_index(self.scrape_index()) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 79, in parse_index % (str(found_headings), str(expected_headings)) boundary_bot.scraper.ScraperException: Unexpected headings: Found [], expected ['Current Reviews', 'Recent Reviews'] fetching DemocracyClub/GDSRegisters ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2.9) running job DemocracyClub/GDSRegisters ... Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 66, in for name in get_all_register_names(): File "scraper.py", line 35, in get_all_register_names req = requests.get(make_url("register")) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) fetching wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-qqp8_b0s Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-qqp8_b0s Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... found 97 stations 97 stations in database found 99 districts 99 districts in database fetching wdiv-scrapers/DC-PollingStations-Barnet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-7gx_xq77 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-7gx_xq77 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Barnet ... Traceback (most recent call last): File "scraper.py", line 21, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 33, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 23, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Bristol ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-tazjzx0w Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-tazjzx0w Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Bristol ... https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_community_and_safety/MapServer/1/query?outFields=*&where=1%3D1&f=geojson found 168 stations 168 stations in database https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_boundaries/MapServer/3/query?outFields=*&where=1%3D1&f=geojson found 240 districts 240 districts in database fetching wdiv-scrapers/DC-PollingStations-Broxtowe ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-hhvxk_75 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-hhvxk_75 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Broxtowe ... Traceback (most recent call last): File "scraper.py", line 13, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Calderdale ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-jsgcii_o Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-jsgcii_o Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Calderdale ... Traceback (most recent call last): File "scraper.py", line 47, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Camden ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-emvmni98 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-emvmni98 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Camden ... Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Canterbury ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-4cq46qen Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-4cq46qen Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Canterbury ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1288, in connect server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 752, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 117, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-p2bd6ngy Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-p2bd6ngy Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Doncaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ny00tyoc Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ny00tyoc Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Doncaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Dover ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-6ebdsacu Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-6ebdsacu Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dover ... found 69 stations 69 stations in database found 69 districts 69 districts in database fetching wdiv-scrapers/DC-PollingStations-Dundee ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ikyjm2gq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ikyjm2gq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dundee ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized fetching wdiv-scrapers/DC-PollingStations-Edinburgh ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-pg24pgzt Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-pg24pgzt Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Edinburgh ... found 158 stations 158 stations in database found 158 districts 158 districts in database fetching wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_uipv71j Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_uipv71j Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... found 37 stations 37 stations in database found 26 districts 26 districts in database fetching wdiv-scrapers/DC-PollingStations-Fife ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-6e1253qq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-6e1253qq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Fife ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Guildford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kba7p0f8 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kba7p0f8 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Guildford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 40, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 58, in scrape data_str = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 35, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Hackney ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-s9riufj3 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-s9riufj3 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Hackney ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-84tp8zsn Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-84tp8zsn Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... found 71 stations 71 stations in database found 71 districts 71 districts in database fetching wdiv-scrapers/DC-PollingStations-Kingston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-5d1_o8x2 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-5d1_o8x2 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Kingston ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Lambeth ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-hrnuysaa Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-hrnuysaa Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lambeth ... Traceback (most recent call last): File "scraper.py", line 12, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Lancaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-lzq7jf7l Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-lzq7jf7l Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lancaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Moray ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-tokgfo_p Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-tokgfo_p Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Moray ... found 60 stations 60 stations in database found 60 districts 60 districts in database fetching wdiv-scrapers/DC-PollingStations-Nottingham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_s5x5_lb Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_s5x5_lb Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Nottingham ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Pembrokeshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-qqj7iam1 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-qqj7iam1 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Pembrokeshire ... found 106 stations 106 stations in database found 135 districts 135 districts in database fetching wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-1cduczqu Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-1cduczqu Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Preston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-v6zr53yl Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-v6zr53yl Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Preston ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Salford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-qwxgfmdo Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-qwxgfmdo Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Salford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 62, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/ckan_scraper.py", line 33, in scrape data_str, data = self.get_data() File "scraper.py", line 24, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "scraper.py", line 16, in get_data_from_url with urllib.request.urlopen(url, timeout=300, context=context) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Sevenoaks ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-cr4r7glz Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-cr4r7glz Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Sevenoaks ... Traceback (most recent call last): File "scraper.py", line 8, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Southampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-n2qbt50h Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-n2qbt50h Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Southampton ... fetching wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_g9sa0zr Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_g9sa0zr Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Traceback (most recent call last): File "scraper.py", line 12, in search_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-StAlbans ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-q4j3j3r9 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-q4j3j3r9 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-StAlbans ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -5] No address associated with hostname During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Thanet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-c5jj0m7k Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-c5jj0m7k Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Thanet ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-4xcuocyr Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-4xcuocyr Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1264, in do_open r = h.getresponse() File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response fetching wdiv-scrapers/DC-PollingStations-TunbridgeWells ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-o_kbyvbr Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-o_kbyvbr Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TunbridgeWells ... https://opendata.arcgis.com/datasets/ea6894c98dfb4cf99775d1fb949cf0de_6.geojson found 53 stations 53 stations in database https://opendata.arcgis.com/datasets/24b6d662cd1145eba4743d27e91c0436_5.geojson found 51 districts 51 districts in database fetching wdiv-scrapers/DC-PollingStations-Uttlesford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-aowpw0k6 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-aowpw0k6 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Uttlesford ... found 79 districts 79 districts in database fetching wdiv-scrapers/DC-PollingStations-Wolverhampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-nlg5pr_o Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-nlg5pr_o Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Wolverhampton ... found 104 stations 104 stations in database found 125 districts 125 districts in database fetching wdiv-scrapers/DC-PollingStations-York ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0qd0vu9b Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0qd0vu9b Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-York ... found 86 stations 86 stations in database found 92 districts Traceback (most recent call last): File "scraper.py", line 55, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 56, in scrape record['pk'] = feature['properties'][self.key] KeyError: 'OBJECTID_1' Successfully ran DemocracyClub/womblr Successfully ran wdiv-scrapers/DC-PollingStations-ArgyllAndBute Successfully ran wdiv-scrapers/DC-PollingStations-Bristol Successfully ran wdiv-scrapers/DC-PollingStations-Dover Successfully ran wdiv-scrapers/DC-PollingStations-Edinburgh Successfully ran wdiv-scrapers/DC-PollingStations-EpsomAndEwell Successfully ran wdiv-scrapers/DC-PollingStations-HammersmithAndFulham Successfully ran wdiv-scrapers/DC-PollingStations-Moray Successfully ran wdiv-scrapers/DC-PollingStations-Pembrokeshire Successfully ran wdiv-scrapers/DC-PollingStations-Southampton Successfully ran wdiv-scrapers/DC-PollingStations-TunbridgeWells Successfully ran wdiv-scrapers/DC-PollingStations-Uttlesford Successfully ran wdiv-scrapers/DC-PollingStations-Wolverhampton Failed to run DemocracyClub/ONS-Geography-Releases-Scraper Failed to run DemocracyClub/Ubuntu-AMI-Scraper Failed to run DemocracyClub/boundary-bot Failed to run DemocracyClub/GDSRegisters Failed to run wdiv-scrapers/DC-PollingStations-Barnet Failed to run wdiv-scrapers/DC-PollingStations-Broxtowe Failed to run wdiv-scrapers/DC-PollingStations-Calderdale Failed to run wdiv-scrapers/DC-PollingStations-Camden Failed to run wdiv-scrapers/DC-PollingStations-Canterbury Failed to run wdiv-scrapers/DC-PollingStations-Carmarthenshire Failed to run wdiv-scrapers/DC-PollingStations-Doncaster Failed to run wdiv-scrapers/DC-PollingStations-Dundee Failed to run wdiv-scrapers/DC-PollingStations-Fife Failed to run wdiv-scrapers/DC-PollingStations-Guildford Failed to run wdiv-scrapers/DC-PollingStations-Hackney Failed to run wdiv-scrapers/DC-PollingStations-Kingston Failed to run wdiv-scrapers/DC-PollingStations-Lambeth Failed to run wdiv-scrapers/DC-PollingStations-Lancaster Failed to run wdiv-scrapers/DC-PollingStations-Nottingham Failed to run wdiv-scrapers/DC-PollingStations-PerthAndKinross Failed to run wdiv-scrapers/DC-PollingStations-Preston Failed to run wdiv-scrapers/DC-PollingStations-Salford Failed to run wdiv-scrapers/DC-PollingStations-Sevenoaks Failed to run wdiv-scrapers/DC-PollingStations-SouthAyrshire Failed to run wdiv-scrapers/DC-PollingStations-StAlbans Failed to run wdiv-scrapers/DC-PollingStations-Thanet Failed to run wdiv-scrapers/DC-PollingStations-TonbridgeMalling Failed to run wdiv-scrapers/DC-PollingStations-York