adrianshort
/
planningalerts
mirror of https://github.com/adrianshort/planningalerts.git
1
0
Fork 0
Code Issues 0 Releases 0 Wiki Activity
Automatically exported from code.google.com/p/planningalerts
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
291 Commits
2 Branches
1.3 MiB
 
 
 
 
 
 
Tree: 09ca7050c3
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '09ca7050c3'
${ noResults }
planningalerts/python_scrapers
duncan.parkes 09ca7050c3 Add scraper for Leicestershire County Council. 16 years ago
..
Aberdeenshire.py Add parser for Kirklees. Get rid of some unnecessary imports. 16 years ago
AcolnetParser.py Fix Lewisham comments email address. 16 years ago
AcolnetParser_HTMLParser.py Add New version of the Acolnet scraper using BeautifulSoup instead of HTMLParser. 17 years ago
AmberValley.py Add scraper for Amber Valley. This was a really irritating one. There is no way to link to individual info pages, so 16 years ago
ApplicationSearchServletParser.py Daventry have replaced their nice url with an IP address... 16 years ago
AtriumePlanning.py Fix Dorset County Council scraper (the council seem to be using an IP address now rather than the domain they had 16 years ago
Barnsley.py Add scraper for Barnsley. 16 years ago
Berwick.py Add parser for Kirklees. Get rid of some unnecessary imports. 16 years ago
Birmingham.py Add scraper for Birmingham. 16 years ago
Brent.py Add scraper for Brent. 16 years ago
Brentwood.cgi Add Peter's scrapers for Highland, Glasgow, and Brentwood. 16 years ago
Brentwood.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. 16 years ago
Broxbourne.cgi Rename Broxbourne scraper so that it matches the database. 17 years ago
CGITemplate Add more debug. 16 years ago
Carmarthenshire.py Add scraper for Lichfield. Remove another unused import. 16 years ago
Christchurch.cgi Add Somerset scraper 17 years ago
Dacorum.cgi Fix the Dacorum perl parser. 16 years ago
EastHerts.cgi Move some cgi scripts so that I can copy them in with the right file 17 years ago
Eastbourne.py Add scraper for Eastbourne. The info and comment links won't work since they require you to have a cookie. If you go 16 years ago
Enfield.cgi Add back in the Enfield scraper which I probably deleted 17 years ago
Exmoor.py Add scraper for Exmoor. Fix name of Herefordshire. 16 years ago
FastWeb.py Oops - printing the sutton results twice... 16 years ago
Fife.py Add Forest of Dean and Fife parsers. 16 years ago
Flintshire.py Add scraper for Flintshire. I've included the OSGB x,y coordinates in the hope that we can start using these where 16 years ago
ForestOfDean.py Add Forest of Dean and Fife parsers. 16 years ago
Glasgow.cgi Add Peter's scrapers for Highland, Glasgow, and Brentwood. 16 years ago
Glasgow.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. 16 years ago
Gosport.py Add scraper for Gosport. 16 years ago
HTTPHandlers.py Add scraper for Gosport. 16 years ago
Halton.py Adding scraper for Halton. 16 years ago
Hampshire.py Add Hampshire scraper. 16 years ago
Harrow.py Adding scraper for Harrow. 16 years ago
Hastings.py Add scraper for Hastings. Sadly, no decent info urls again. Had to use the search page. The real info url is only 16 years ago
Herefordshire.py Add scraper for Exmoor. Fix name of Herefordshire. 16 years ago
Highland.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites 16 years ago
Highland.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. 16 years ago
Hounslow.py Add scraper for Hounslow. 16 years ago
IsleOfWight.py Add Scraper for the Isle of Wight. 16 years ago
Kensington.py Add Kensington and Chelsea Scraper. 16 years ago
KingstonUponThames.py Add scraper for Kingston upon Thames. 16 years ago
Kirklees.py Add parser for Kirklees. Get rid of some unnecessary imports. 16 years ago
Leicestershire.py Add scraper for Leicestershire County Council. 16 years ago
Lichfield.py Add scraper for Lichfield. Remove another unused import. 16 years ago
Maldon.py Add Pendle scraper (this is basically the same as Maldon). 16 years ago
Medway.py Add Medway scraper. 16 years ago
MultipartPostHandler.py add some Acolnet sites: 17 years ago
Newport.cgi Add perl scraper from Peter for Newport. 16 years ago
Newport.pl Add perl scraper from Peter for Newport. 16 years ago
NorthAyrshire.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites 16 years ago
NorthAyrshire.pl North Ayrshire, Redbridge: added 16 years ago
NorthHerts.cgi Move some cgi scripts so that I can copy them in with the right file 17 years ago
Ocella.py Add scraper for Gosport. 16 years ago
OtherFilesToCopy.csv Add scraper for Leicestershire County Council. 16 years ago
Planet.py Set up planet scraper. This get's us 5 new councils (though one, Tewkesbury, has no postcodes) 16 years ago
PlanningExplorer.py Change some PlanningExplorer scrapers to use date_registered rather than date_received. 16 years ago
PlanningUtils.py Add scraper for Herefordshire. 16 years ago
PublicAccess.py Some debug (mostly for westminster). 16 years ago
README adding fastweb 17 years ago
Redbridge.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites 16 years ago
Redbridge.pl North Ayrshire, Redbridge: added 16 years ago
RutlandLike.py Fix Lincoln parser. 16 years ago
Shetland.py Add scraper for Shetland Islands. 16 years ago
Shropshire.py Add Scraper for Lewes (which now has PublicAccess). 16 years ago
SitesToGenerate.csv Add scraper for Leicestershire County Council. 16 years ago
SouthOxfordshire.cgi expanded the api (still need to test some stuff) 17 years ago
SouthOxfordshireParser.py add the south oxfordshire scraper, and code to generate 17 years ago
SouthSomerset.cgi New version of South Somerset from Tom. 17 years ago
SwiftLG.py Some debug (mostly for westminster). 16 years ago
WAM.py Add scraper for Eastbourne. The info and comment links won't work since they require you to have a cookie. If you go 16 years ago
WestDorset.py Add python parser for West Dorset, and remove the non-working perl one. 16 years ago
Westminster.py Go back to the urllib2 version of Westminster. This works on disruptiveproactivity. 16 years ago
Westminster_pycurl.py Adding scraper for Halton. 16 years ago
createCGI.sh remove offending svn rm line... 17 years ago
generateCGIScripts.py adding fastweb 17 years ago

README 

In order to generate the contents of the CGI directory (../cgi-bin/)
run

./createCGI.sh

This script generates new cgi files in the CGI directory,
copies in some other files that are needed,
and commits all these changes to svn.

There is currently something very nasty in the permissions
column in the file OtherFilesToCopy.csv

Until version 2.6 of python, there appears to be no convenient
way to convert a string like 0755 into an octal. I am therefore
using integers to specify permissions of the copied files...

420 is 0644
493 is 0755

This is horrid, and must change in the future...