fetching DemocracyClub/ONS-Geography-Releases-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 3)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-5g6t3kfj Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-5g6t3kfj Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.1) running job DemocracyClub/ONS-Geography-Releases-Scraper ... Traceback (most recent call last): File "scraper.py", line 128, in scrape(lgd_url, "lgd") File "scraper.py", line 84, in scrape res.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=geoportal1-ons.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=LGD+Boundaries&sort=-updatedAt fetching DemocracyClub/Ubuntu-AMI-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 6)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-nz9ttxxp Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-nz9ttxxp Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: json5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.16) Collecting json5 Using cached json5-0.9.16-py2.py3-none-any.whl (19 kB) Using cached json5-0.9.15-py2.py3-none-any.whl (19 kB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.1) running job DemocracyClub/Ubuntu-AMI-Scraper ... {'version': '22.04 LTS', 'zone': 'eu-west-2', 'instance_type': 'hvm:ebs-ssd', 'ami_id': 'ami-090f0cf86291c77fa', 'date': '20260218', 'cpu_arch': 'amd64'} {'status': '404', 'documentation_url': 'https://docs.github.com/rest/git/refs#create-a-reference', 'message': 'Not Found'} Traceback (most recent call last): File "scraper.py", line 127, in scrape() File "scraper.py", line 119, in scrape open_pull_request(repo, record) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/Ubuntu-AMI-Scraper/github.py", line 18, in open_pull_request g.create_branch(newbranch) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 111, in create_branch return self._request("POST", url, payload) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 39, in _request r.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/DemocracyClub/ee_deploy/git/refs fetching DemocracyClub/womblr ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 4)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-g4eyf2gi Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-g4eyf2gi Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.9.0.post0) Collecting python-dateutil Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached python_dateutil-2.9.0-py2.py3-none-any.whl (230 kB) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: uk-election-timetables==2.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.3.0) Collecting uk-election-timetables==2.3.0 Using cached uk_election_timetables-2.3.0-py3-none-any.whl Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.1) running job DemocracyClub/womblr ... Nothing to do today, but here's the results from the last run.. ===== :satellite_antenna: *By-Elections happening in the next month* :ballot_box_with_ballot: 12/03/2026: . known candidates: 5 :lock: 12/03/2026: . known candidates: 6 :lock: 12/03/2026: . known candidates: 7 :lock: 12/03/2026: . known candidates: 4 :lock: 17/03/2026: . known candidates: 8 :lock: 26/03/2026: . known candidates: 5 :lock: 26/03/2026: . known candidates: 5 :lock: 26/03/2026: . known candidates: 4 :lock: 02/04/2026: . known candidates: 5 :lock: 02/04/2026: . known candidates: 5 :lock: 09/04/2026: . known candidates: 5 (SoPN should be published) fetching DemocracyClub/boundary-bot ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 8)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-wvzounej Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-wvzounej Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: cssselect==0.9.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.1) Collecting cssselect==0.9.1 Using cached cssselect-0.9.1.tar.gz (32 kB) Requirement already satisfied: rapidfuzz==0.2.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.2.2) Collecting rapidfuzz==0.2.2 Using cached rapidfuzz-0.2.2-cp35-cp35m-manylinux2010_x86_64.whl (1.3 MB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 6)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: scrapy==1.8.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 7)) (1.8.1) Collecting scrapy==1.8.1 Using cached Scrapy-1.8.1-py2.py3-none-any.whl (239 kB) Requirement already satisfied: pybind11>=2.4 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from rapidfuzz==0.2.2->-r requirements.txt (line 3)) (2.4.3) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.3.15) Requirement already satisfied: cryptography>=2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.8) Requirement already satisfied: w3lib>=1.17.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.21.0) Requirement already satisfied: zope.interface>=4.1.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (5.0.1) Requirement already satisfied: pyOpenSSL>=16.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: protego>=0.1.15 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (0.1.16) Requirement already satisfied: parsel>=1.5.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.2) Requirement already satisfied: PyDispatcher>=2.0.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.5) Requirement already satisfied: queuelib>=1.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.0) Requirement already satisfied: service-identity>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: Twisted>=17.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (20.3.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (1.25.8) Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (1.14.0) Requirement already satisfied: pycparser in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: pyasn1-modules in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.2.8) Requirement already satisfied: pyasn1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.4.8) Requirement already satisfied: attrs>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.3.0) Requirement already satisfied: incremental>=16.10.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (17.5.0) Requirement already satisfied: PyHamcrest!=1.10.0,>=1.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.2) Requirement already satisfied: constantly>=15.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (15.1.0) Requirement already satisfied: Automat>=0.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: hyperlink>=17.1.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.0.0) Requirement already satisfied: setuptools in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from zope.interface>=4.1.3->scrapy==1.8.1->-r requirements.txt (line 7)) (46.1.1) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.1) running job DemocracyClub/boundary-bot ... Traceback (most recent call last): File "scraper.py", line 17, in scraper.scrape() File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 261, in scrape self.parse_index(self.scrape_index()) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 79, in parse_index % (str(found_headings), str(expected_headings)) boundary_bot.scraper.ScraperException: Unexpected headings: Found [], expected ['Current Reviews', 'Recent Reviews'] fetching DemocracyClub/GDSRegisters ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (1.25.8) running job DemocracyClub/GDSRegisters ... Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 66, in for name in get_all_register_names(): File "scraper.py", line 35, in get_all_register_names req = requests.get(make_url("register")) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) fetching wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-o8biltbr Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-o8biltbr Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... found 97 stations 97 stations in database found 99 districts 99 districts in database fetching wdiv-scrapers/DC-PollingStations-Barnet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-t7zgkcwl Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-t7zgkcwl Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Barnet ... Traceback (most recent call last): File "scraper.py", line 21, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 33, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 23, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Bristol ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kcqifgch Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kcqifgch Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Bristol ... https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_community_and_safety/MapServer/1/query?outFields=*&where=1%3D1&f=geojson found 168 stations 168 stations in database https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_boundaries/MapServer/3/query?outFields=*&where=1%3D1&f=geojson found 240 districts 240 districts in database fetching wdiv-scrapers/DC-PollingStations-Broxtowe ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-fr34wz8m Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-fr34wz8m Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Broxtowe ... Traceback (most recent call last): File "scraper.py", line 13, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Calderdale ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-lqz2e2kn Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-lqz2e2kn Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Calderdale ... Traceback (most recent call last): File "scraper.py", line 47, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Camden ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-fmdik1lx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-fmdik1lx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Camden ... Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Canterbury ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-n6drjsb5 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-n6drjsb5 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Canterbury ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1288, in connect server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 752, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 117, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-jntmhaxv Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-jntmhaxv Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Doncaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-gsma2k42 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-gsma2k42 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Doncaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Dover ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3u27qfr4 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3u27qfr4 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dover ... found 69 stations 69 stations in database found 69 districts 69 districts in database fetching wdiv-scrapers/DC-PollingStations-Dundee ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-7idqjtk9 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-7idqjtk9 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dundee ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized fetching wdiv-scrapers/DC-PollingStations-Edinburgh ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-d5h26thk Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-d5h26thk Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Edinburgh ... found 158 stations 158 stations in database found 158 districts 158 districts in database fetching wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-tntmb79c Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-tntmb79c Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... found 37 stations 37 stations in database found 26 districts 26 districts in database fetching wdiv-scrapers/DC-PollingStations-Fife ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kaftzjgm Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kaftzjgm Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Fife ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Guildford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0ylibnxi Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0ylibnxi Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Guildford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 40, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 58, in scrape data_str = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 35, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Hackney ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-9hwslz9d Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-9hwslz9d Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Hackney ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0t1qrjuy Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0t1qrjuy Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Kingston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-a3bf7_se Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-a3bf7_se Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Kingston ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Lambeth ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-8e6zyqo5 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-8e6zyqo5 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lambeth ... Traceback (most recent call last): File "scraper.py", line 12, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Lancaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-r_tvnm7f Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-r_tvnm7f Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lancaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Moray ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-psb6xtn4 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-psb6xtn4 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Moray ... found 60 stations 60 stations in database found 60 districts 60 districts in database fetching wdiv-scrapers/DC-PollingStations-Nottingham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_zhtlda5 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_zhtlda5 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Nottingham ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Pembrokeshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-6w73bjar Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-6w73bjar Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Pembrokeshire ... found 106 stations 106 stations in database found 135 districts 135 districts in database fetching wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-stvovakq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-stvovakq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Preston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-eiobdr6w Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-eiobdr6w Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Preston ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Salford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-8sqpyn1k Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-8sqpyn1k Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Salford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 62, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/ckan_scraper.py", line 33, in scrape data_str, data = self.get_data() File "scraper.py", line 24, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "scraper.py", line 16, in get_data_from_url with urllib.request.urlopen(url, timeout=300, context=context) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Sevenoaks ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-guzpojg1 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-guzpojg1 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Sevenoaks ... Traceback (most recent call last): File "scraper.py", line 8, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Southampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-dh712az1 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-dh712az1 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Southampton ... fetching wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-7mmg4y3x Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-7mmg4y3x Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Traceback (most recent call last): File "scraper.py", line 12, in search_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-StAlbans ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-rutjby9o Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-rutjby9o Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-StAlbans ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -5] No address associated with hostname During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Thanet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-c3ew4q57 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-c3ew4q57 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Thanet ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ddletnqx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ddletnqx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1264, in do_open r = h.getresponse() File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response fetching wdiv-scrapers/DC-PollingStations-TunbridgeWells ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-szdmwjtg Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-szdmwjtg Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TunbridgeWells ... https://opendata.arcgis.com/datasets/ea6894c98dfb4cf99775d1fb949cf0de_6.geojson found 53 stations 53 stations in database https://opendata.arcgis.com/datasets/24b6d662cd1145eba4743d27e91c0436_5.geojson found 51 districts 51 districts in database fetching wdiv-scrapers/DC-PollingStations-Uttlesford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-5avz8psq Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-5avz8psq Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Uttlesford ... found 79 districts 79 districts in database fetching wdiv-scrapers/DC-PollingStations-Wolverhampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-n2jnf406 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-n2jnf406 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Wolverhampton ... found 104 stations 104 stations in database found 125 districts 125 districts in database fetching wdiv-scrapers/DC-PollingStations-York ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-l7sccl8_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-l7sccl8_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-York ... found 86 stations 86 stations in database found 92 districts Traceback (most recent call last): File "scraper.py", line 55, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 56, in scrape record['pk'] = feature['properties'][self.key] KeyError: 'OBJECTID_1' Successfully ran DemocracyClub/womblr Successfully ran wdiv-scrapers/DC-PollingStations-ArgyllAndBute Successfully ran wdiv-scrapers/DC-PollingStations-Bristol Successfully ran wdiv-scrapers/DC-PollingStations-Dover Successfully ran wdiv-scrapers/DC-PollingStations-Edinburgh Successfully ran wdiv-scrapers/DC-PollingStations-EpsomAndEwell Successfully ran wdiv-scrapers/DC-PollingStations-Moray Successfully ran wdiv-scrapers/DC-PollingStations-Pembrokeshire Successfully ran wdiv-scrapers/DC-PollingStations-Southampton Successfully ran wdiv-scrapers/DC-PollingStations-TunbridgeWells Successfully ran wdiv-scrapers/DC-PollingStations-Uttlesford Successfully ran wdiv-scrapers/DC-PollingStations-Wolverhampton Failed to run DemocracyClub/ONS-Geography-Releases-Scraper Failed to run DemocracyClub/Ubuntu-AMI-Scraper Failed to run DemocracyClub/boundary-bot Failed to run DemocracyClub/GDSRegisters Failed to run wdiv-scrapers/DC-PollingStations-Barnet Failed to run wdiv-scrapers/DC-PollingStations-Broxtowe Failed to run wdiv-scrapers/DC-PollingStations-Calderdale Failed to run wdiv-scrapers/DC-PollingStations-Camden Failed to run wdiv-scrapers/DC-PollingStations-Canterbury Failed to run wdiv-scrapers/DC-PollingStations-Carmarthenshire Failed to run wdiv-scrapers/DC-PollingStations-Doncaster Failed to run wdiv-scrapers/DC-PollingStations-Dundee Failed to run wdiv-scrapers/DC-PollingStations-Fife Failed to run wdiv-scrapers/DC-PollingStations-Guildford Failed to run wdiv-scrapers/DC-PollingStations-Hackney Failed to run wdiv-scrapers/DC-PollingStations-HammersmithAndFulham Failed to run wdiv-scrapers/DC-PollingStations-Kingston Failed to run wdiv-scrapers/DC-PollingStations-Lambeth Failed to run wdiv-scrapers/DC-PollingStations-Lancaster Failed to run wdiv-scrapers/DC-PollingStations-Nottingham Failed to run wdiv-scrapers/DC-PollingStations-PerthAndKinross Failed to run wdiv-scrapers/DC-PollingStations-Preston Failed to run wdiv-scrapers/DC-PollingStations-Salford Failed to run wdiv-scrapers/DC-PollingStations-Sevenoaks Failed to run wdiv-scrapers/DC-PollingStations-SouthAyrshire Failed to run wdiv-scrapers/DC-PollingStations-StAlbans Failed to run wdiv-scrapers/DC-PollingStations-Thanet Failed to run wdiv-scrapers/DC-PollingStations-TonbridgeMalling Failed to run wdiv-scrapers/DC-PollingStations-York