fetching DemocracyClub/ONS-Geography-Releases-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 3)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-h90ni_xi Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-h90ni_xi Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 2)) (1.1.1) running job DemocracyClub/ONS-Geography-Releases-Scraper ... Traceback (most recent call last): File "scraper.py", line 128, in scrape(lgd_url, "lgd") File "scraper.py", line 84, in scrape res.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=geoportal1-ons.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=LGD+Boundaries&sort=-updatedAt fetching DemocracyClub/Ubuntu-AMI-Scraper ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 6)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-9xk9pw1y Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-9xk9pw1y Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: json5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.16) Collecting json5 Using cached json5-0.9.16-py2.py3-none-any.whl (19 kB) Using cached json5-0.9.15-py2.py3-none-any.whl (19 kB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 4)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 5)) (1.1.1) running job DemocracyClub/Ubuntu-AMI-Scraper ... {'zone': 'eu-west-2', 'cpu_arch': 'amd64', 'date': '20260218', 'instance_type': 'hvm:ebs-ssd', 'version': '22.04 LTS', 'ami_id': 'ami-090f0cf86291c77fa'} {'status': '404', 'message': 'Not Found', 'documentation_url': 'https://docs.github.com/rest/git/refs#create-a-reference'} Traceback (most recent call last): File "scraper.py", line 127, in scrape() File "scraper.py", line 119, in scrape open_pull_request(repo, record) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/Ubuntu-AMI-Scraper/github.py", line 18, in open_pull_request g.create_branch(newbranch) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 111, in create_branch return self._request("POST", url, payload) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/commitment/commitment.py", line 39, in _request r.raise_for_status() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/DemocracyClub/ee_deploy/git/refs fetching DemocracyClub/womblr ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 4)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-qcjhku2c Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-qcjhku2c Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.9.0.post0) Collecting python-dateutil Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached python_dateutil-2.9.0-py2.py3-none-any.whl (230 kB) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: uk-election-timetables==2.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.3.0) Collecting uk-election-timetables==2.3.0 Using cached uk_election_timetables-2.3.0-py3-none-any.whl Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 3)) (1.1.1) running job DemocracyClub/womblr ... Nothing to do today, but here's the results from the last run.. ===== :mega: *By-Elections happening in the next month* :alarm_clock: 26/02/2026: . known candidates: 11 :lock: 05/03/2026: . known candidates: 5 :lock: 05/03/2026: . known candidates: 4 :lock: 12/03/2026: . known candidates: 5 :lock: 12/03/2026: . known candidates: 6 :lock: 12/03/2026: . known candidates: 7 :lock: 12/03/2026: . known candidates: 5 :lock: 17/03/2026: . known candidates: 8 :lock: fetching DemocracyClub/boundary-bot ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/polling-bot-4000/polling-bot.git (from -r requirements.txt (line 8)) Cloning https://github.com/polling-bot-4000/polling-bot.git to /tmp/pip-req-build-2mp29o_a Running command git clone -q https://github.com/polling-bot-4000/polling-bot.git /tmp/pip-req-build-2mp29o_a Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: cssselect==0.9.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (0.9.1) Collecting cssselect==0.9.1 Using cached cssselect-0.9.1.tar.gz (32 kB) Requirement already satisfied: rapidfuzz==0.2.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 3)) (0.2.2) Collecting rapidfuzz==0.2.2 Using cached rapidfuzz-0.2.2-cp35-cp35m-manylinux2010_x86_64.whl (1.3 MB) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 4)) (4.9.4) Collecting lxml<5,>=4.2 Using cached lxml-4.9.4-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Using cached lxml-4.9.3-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB) Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 5)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 6)) (0.5.1) Collecting scraperwiki==0.5.1 Using cached scraperwiki-0.5.1.tar.gz (7.7 kB) Requirement already satisfied: scrapy==1.8.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 7)) (1.8.1) Collecting scrapy==1.8.1 Using cached Scrapy-1.8.1-py2.py3-none-any.whl (239 kB) Requirement already satisfied: pybind11>=2.4 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from rapidfuzz==0.2.2->-r requirements.txt (line 3)) (2.4.3) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.4.2) Requirement already satisfied: Twisted>=17.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (20.3.0) Requirement already satisfied: queuelib>=1.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.0) Requirement already satisfied: service-identity>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: zope.interface>=4.1.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (5.0.1) Requirement already satisfied: PyDispatcher>=2.0.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.5) Requirement already satisfied: parsel>=1.5.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.5.2) Requirement already satisfied: w3lib>=1.17.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (1.21.0) Requirement already satisfied: cryptography>=2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (2.8) Requirement already satisfied: protego>=0.1.15 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (0.1.16) Requirement already satisfied: pyOpenSSL>=16.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scrapy==1.8.1->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 5)) (1.25.8) Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (1.14.0) Requirement already satisfied: pycparser in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: pyasn1-modules in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.2.8) Requirement already satisfied: pyasn1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (0.4.8) Requirement already satisfied: attrs>=16.0.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from service-identity>=16.0.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.3.0) Requirement already satisfied: Automat>=0.3.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: incremental>=16.10.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (17.5.0) Requirement already satisfied: constantly>=15.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (15.1.0) Requirement already satisfied: PyHamcrest!=1.10.0,>=1.9.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (2.0.2) Requirement already satisfied: hyperlink>=17.1.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Twisted>=17.9.0->scrapy==1.8.1->-r requirements.txt (line 7)) (19.0.0) Requirement already satisfied: setuptools in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from zope.interface>=4.1.3->scrapy==1.8.1->-r requirements.txt (line 7)) (46.1.1) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->-r requirements.txt (line 6)) (1.1.1) running job DemocracyClub/boundary-bot ... Traceback (most recent call last): File "scraper.py", line 17, in scraper.scrape() File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 261, in scrape self.parse_index(self.scrape_index()) File "/home/ubuntu/development/scraper-orchestration/DemocracyClub/boundary-bot/boundary_bot/scraper.py", line 79, in parse_index % (str(found_headings), str(expected_headings)) boundary_bot.scraper.ScraperException: Unexpected headings: Found [], expected ['Current Reviews', 'Recent Reviews'] fetching DemocracyClub/GDSRegisters ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 1)) (2.0.2) Collecting commitment<3,>=2 Using cached commitment-2.0.2-py3-none-any.whl (4.3 kB) Using cached commitment-2.0.1-cp35-none-any.whl Requirement already satisfied: requests<3,>=2.20.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from -r requirements.txt (line 2)) (2.25.1) Collecting requests<3,>=2.20.0 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests<3,>=2.20.0->-r requirements.txt (line 2)) (2019.11.28) running job DemocracyClub/GDSRegisters ... Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 66, in for name in get_all_register_names(): File "scraper.py", line 35, in get_all_register_names req = requests.get(make_url("register")) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.registers.service.gov.uk', port=443): Max retries exceeded with url: /registers/register/download-csv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',)) fetching wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-3t0p_u3x Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-3t0p_u3x Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-ArgyllAndBute ... found 97 stations 97 stations in database Traceback (most recent call last): File "/usr/lib/python3.5/http/client.py", line 571, in _readall_chunked value.append(self._safe_read(chunk_left)) File "/usr/lib/python3.5/http/client.py", line 619, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(8040 bytes read, 36341 more expected) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 96, in get_data_from_url data = response.read() File "/usr/lib/python3.5/http/client.py", line 465, in read return self._readall_chunked() File "/usr/lib/python3.5/http/client.py", line 575, in _readall_chunked raise IncompleteRead(b''.join(value)) http.client.IncompleteRead: IncompleteRead(40502 bytes read) fetching wdiv-scrapers/DC-PollingStations-Barnet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-pk51fqlv Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-pk51fqlv Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Barnet ... Traceback (most recent call last): File "scraper.py", line 21, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 33, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/datapress_scraper.py", line 23, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Bristol ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-erw8s2fe Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-erw8s2fe Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Bristol ... https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_community_and_safety/MapServer/1/query?outFields=*&where=1%3D1&f=geojson found 168 stations 168 stations in database https://maps2.bristol.gov.uk/server2/rest/services/ext/ll_boundaries/MapServer/3/query?outFields=*&where=1%3D1&f=geojson found 240 districts 240 districts in database fetching wdiv-scrapers/DC-PollingStations-Broxtowe ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-v3y2mwq2 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-v3y2mwq2 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Broxtowe ... Traceback (most recent call last): File "scraper.py", line 13, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Calderdale ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-yo02hf6e Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-yo02hf6e Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Calderdale ... Traceback (most recent call last): File "scraper.py", line 47, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Camden ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-2kbbwa93 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-2kbbwa93 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Camden ... Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Canterbury ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-dpj_jgd8 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-dpj_jgd8 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Canterbury ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1288, in connect server_hostname=server_hostname) File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/usr/lib/python3.5/ssl.py", line 752, in __init__ self.do_handshake() File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 117, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-a89ossw9 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-a89ossw9 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Carmarthenshire ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Doncaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-qu9jqnlx Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-qu9jqnlx Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Doncaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Dover ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-2to22a0l Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-2to22a0l Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dover ... found 69 stations 69 stations in database found 69 districts 69 districts in database fetching wdiv-scrapers/DC-PollingStations-Dundee ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0cdu025b Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0cdu025b Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Dundee ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized fetching wdiv-scrapers/DC-PollingStations-Edinburgh ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kz6qcfvy Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kz6qcfvy Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Edinburgh ... found 158 stations 158 stations in database found 158 districts 158 districts in database fetching wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-ci4ortln Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-ci4ortln Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-EpsomAndEwell ... found 35 stations 35 stations in database found 25 districts 25 districts in database fetching wdiv-scrapers/DC-PollingStations-Fife ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-k4lmone9 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-k4lmone9 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Fife ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Guildford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-bxrrl91_ Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-bxrrl91_ Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Guildford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 40, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 58, in scrape data_str = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/xml_scraper.py", line 35, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Hackney ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-0x4dmlww Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-0x4dmlww Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Hackney ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 9, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-z3yb_hjb Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-z3yb_hjb Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-HammersmithAndFulham ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Kingston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-zxg2cj60 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-zxg2cj60 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Kingston ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Lambeth ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-pgf1_8a6 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-pgf1_8a6 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lambeth ... Traceback (most recent call last): File "scraper.py", line 12, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request fetching wdiv-scrapers/DC-PollingStations-Lancaster ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_wlhbee3 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_wlhbee3 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Lancaster ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 82, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-Moray ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-6e65fthm Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-6e65fthm Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Moray ... found 60 stations 60 stations in database Traceback (most recent call last): File "/usr/lib/python3.5/http/client.py", line 571, in _readall_chunked value.append(self._safe_read(chunk_left)) File "/usr/lib/python3.5/http/client.py", line 619, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(17842 bytes read, 28678 more expected) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 15, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 96, in get_data_from_url data = response.read() File "/usr/lib/python3.5/http/client.py", line 465, in read return self._readall_chunked() File "/usr/lib/python3.5/http/client.py", line 575, in _readall_chunked raise IncompleteRead(b''.join(value)) http.client.IncompleteRead: IncompleteRead(30703 bytes read) fetching wdiv-scrapers/DC-PollingStations-Nottingham ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_z67r7fs Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_z67r7fs Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Nottingham ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Pembrokeshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-62bf_h35 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-62bf_h35 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Pembrokeshire ... found 106 stations 106 stations in database found 135 districts 135 districts in database fetching wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-i8geysx6 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-i8geysx6 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-PerthAndKinross ... Traceback (most recent call last): File "scraper.py", line 14, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 504, in error result = self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Preston ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-9s4ckdsf Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-9s4ckdsf Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Preston ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Salford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-7el12ie5 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-7el12ie5 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Salford ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 1280, in connect super().connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 62, in stations_url = stations_meta_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/ckan_scraper.py", line 33, in scrape data_str, data = self.get_data() File "scraper.py", line 24, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "scraper.py", line 16, in get_data_from_url with urllib.request.urlopen(url, timeout=300, context=context) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Sevenoaks ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-_asyymd5 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-_asyymd5 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Sevenoaks ... Traceback (most recent call last): File "scraper.py", line 8, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden fetching wdiv-scrapers/DC-PollingStations-Southampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-damecqcr Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-damecqcr Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Southampton ... fetching wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-u3__pefu Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-u3__pefu Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-SouthAyrshire ... Traceback (most recent call last): File "scraper.py", line 12, in search_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 27, in scrape data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/hashonly_scraper.py", line 21, in get_data return get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 472, in open response = meth(req, response) File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.5/urllib/request.py", line 510, in error return self._call_chain(*args) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found fetching wdiv-scrapers/DC-PollingStations-StAlbans ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-91pj4spu Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-91pj4spu Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-StAlbans ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 693, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.5/socket.py", line 732, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -5] No address associated with hostname During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 90, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-Thanet ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-d14t2r70 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-d14t2r70 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Thanet ... Traceback (most recent call last): File "/usr/lib/python3.5/urllib/request.py", line 1261, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.5/http/client.py", line 1134, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1179, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1130, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 946, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 889, in send self.connect() File "/usr/lib/python3.5/http/client.py", line 861, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.5/socket.py", line 711, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 702, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1289, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.5/urllib/request.py", line 1263, in do_open raise URLError(err) urllib.error.URLError: fetching wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-rn9bsqu1 Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-rn9bsqu1 Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TonbridgeMalling ... Traceback (most recent call last): File "scraper.py", line 10, in stations_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 38, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/arcgis_scraper.py", line 31, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 87, in get_data_from_url with urllib.request.urlopen(url, timeout=300) as response: File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/lib/python3.5/urllib/request.py", line 1304, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.5/urllib/request.py", line 1264, in do_open r = h.getresponse() File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response fetching wdiv-scrapers/DC-PollingStations-TunbridgeWells ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-fmplmedl Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-fmplmedl Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-TunbridgeWells ... https://opendata.arcgis.com/datasets/ea6894c98dfb4cf99775d1fb949cf0de_6.geojson found 53 stations 53 stations in database https://opendata.arcgis.com/datasets/24b6d662cd1145eba4743d27e91c0436_5.geojson Traceback (most recent call last): File "/usr/lib/python3.5/http/client.py", line 571, in _readall_chunked value.append(self._safe_read(chunk_left)) File "/usr/lib/python3.5/http/client.py", line 619, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(32177 bytes read, 19036 more expected) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 12, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 96, in get_data_from_url data = response.read() File "/usr/lib/python3.5/http/client.py", line 465, in read return self._readall_chunked() File "/usr/lib/python3.5/http/client.py", line 575, in _readall_chunked raise IncompleteRead(b''.join(value)) http.client.IncompleteRead: IncompleteRead(0 bytes read) fetching wdiv-scrapers/DC-PollingStations-Uttlesford ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-r62dqxlv Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-r62dqxlv Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Uttlesford ... found 79 districts 79 districts in database fetching wdiv-scrapers/DC-PollingStations-Wolverhampton ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-fa5qz9sa Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-fa5qz9sa Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-Wolverhampton ... found 104 stations 104 stations in database found 125 districts 125 districts in database fetching wdiv-scrapers/DC-PollingStations-York ... Already up-to-date. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality. Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r requirements.txt (line 1)) Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-kogxt6oc Running command git clone -q https://github.com/wdiv-scrapers/dc-base-scrapers.git /tmp/pip-req-build-kogxt6oc Requirement already satisfied: arcgis2geojson<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.0) Requirement already satisfied: commitment<3,>=2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.0.2) Requirement already satisfied: geojson-rewind==0.2.0 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.2.0) Requirement already satisfied: lxml<5,>=4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.9.4) Requirement already satisfied: retry==0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.9.2) Requirement already satisfied: scraperwiki==0.5.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (0.5.1) Requirement already satisfied: py<2.0.0,>=1.4.26 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.8.1) Requirement already satisfied: decorator>=3.4.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from retry==0.9.2->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (4.4.2) Requirement already satisfied: six in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.14.0) Requirement already satisfied: alembic in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.4.2) Requirement already satisfied: requests in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.25.1) Requirement already satisfied: sqlalchemy in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.3.15) Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.25.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9) Requirement already satisfied: Mako in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.2) Requirement already satisfied: python-dateutil in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (2.9.0.post0) Requirement already satisfied: python-editor>=0.3 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.0.4) Requirement already satisfied: MarkupSafe>=0.9.2 in /home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r requirements.txt (line 1)) (1.1.1) running job wdiv-scrapers/DC-PollingStations-York ... found 86 stations 86 stations in database Traceback (most recent call last): File "/usr/lib/python3.5/http/client.py", line 571, in _readall_chunked value.append(self._safe_read(chunk_left)) File "/usr/lib/python3.5/http/client.py", line 619, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(32178 bytes read, 8972 more expected) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scraper.py", line 55, in districts_scraper.scrape() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 40, in scrape data_str, data = self.get_data() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/geojson_scraper.py", line 32, in get_data data_str = get_data_from_url(self.url) File "", line 2, in get_data_from_url File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 74, in retry_decorator logger) File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/home/ubuntu/.virtualenvs/py3-morph/lib/python3.5/site-packages/dc_base_scrapers/common.py", line 96, in get_data_from_url data = response.read() File "/usr/lib/python3.5/http/client.py", line 465, in read return self._readall_chunked() File "/usr/lib/python3.5/http/client.py", line 575, in _readall_chunked raise IncompleteRead(b''.join(value)) http.client.IncompleteRead: IncompleteRead(0 bytes read) Successfully ran DemocracyClub/womblr Successfully ran wdiv-scrapers/DC-PollingStations-Bristol Successfully ran wdiv-scrapers/DC-PollingStations-Dover Successfully ran wdiv-scrapers/DC-PollingStations-Edinburgh Successfully ran wdiv-scrapers/DC-PollingStations-EpsomAndEwell Successfully ran wdiv-scrapers/DC-PollingStations-Pembrokeshire Successfully ran wdiv-scrapers/DC-PollingStations-Southampton Successfully ran wdiv-scrapers/DC-PollingStations-Uttlesford Successfully ran wdiv-scrapers/DC-PollingStations-Wolverhampton Failed to run DemocracyClub/ONS-Geography-Releases-Scraper Failed to run DemocracyClub/Ubuntu-AMI-Scraper Failed to run DemocracyClub/boundary-bot Failed to run DemocracyClub/GDSRegisters Failed to run wdiv-scrapers/DC-PollingStations-ArgyllAndBute Failed to run wdiv-scrapers/DC-PollingStations-Barnet Failed to run wdiv-scrapers/DC-PollingStations-Broxtowe Failed to run wdiv-scrapers/DC-PollingStations-Calderdale Failed to run wdiv-scrapers/DC-PollingStations-Camden Failed to run wdiv-scrapers/DC-PollingStations-Canterbury Failed to run wdiv-scrapers/DC-PollingStations-Carmarthenshire Failed to run wdiv-scrapers/DC-PollingStations-Doncaster Failed to run wdiv-scrapers/DC-PollingStations-Dundee Failed to run wdiv-scrapers/DC-PollingStations-Fife Failed to run wdiv-scrapers/DC-PollingStations-Guildford Failed to run wdiv-scrapers/DC-PollingStations-Hackney Failed to run wdiv-scrapers/DC-PollingStations-HammersmithAndFulham Failed to run wdiv-scrapers/DC-PollingStations-Kingston Failed to run wdiv-scrapers/DC-PollingStations-Lambeth Failed to run wdiv-scrapers/DC-PollingStations-Lancaster Failed to run wdiv-scrapers/DC-PollingStations-Moray Failed to run wdiv-scrapers/DC-PollingStations-Nottingham Failed to run wdiv-scrapers/DC-PollingStations-PerthAndKinross Failed to run wdiv-scrapers/DC-PollingStations-Preston Failed to run wdiv-scrapers/DC-PollingStations-Salford Failed to run wdiv-scrapers/DC-PollingStations-Sevenoaks Failed to run wdiv-scrapers/DC-PollingStations-SouthAyrshire Failed to run wdiv-scrapers/DC-PollingStations-StAlbans Failed to run wdiv-scrapers/DC-PollingStations-Thanet Failed to run wdiv-scrapers/DC-PollingStations-TonbridgeMalling Failed to run wdiv-scrapers/DC-PollingStations-TunbridgeWells Failed to run wdiv-scrapers/DC-PollingStations-York