adrianshort
/
planningalerts
spogulis no https://github.com/adrianshort/planningalerts.git
1
0
Atdalīts 0
Kods Problēmas 0 Laidieni 0 Vikivietne Aktivitāte
Automatically exported from code.google.com/p/planningalerts
Nevar pievienot vairāk kā 25 tēmas Tēmai ir jāsākas ar burtu vai ciparu, tā var saturēt domu zīmes ('-') un var būt līdz 35 simboliem gara.
296 Revīzijas
2 Atzari
1.3 MiB
 
 
 
 
 
 
Koks: df565a5098
Atzari Tagi
${ item.name }
Izveidot atzaru ${ searchTerm }
no 'df565a5098'
${ noResults }
planningalerts/python_scrapers
duncan.parkes df565a5098 Add scraper for Weymouth and Portland. pirms 16 gadiem
..
Aberdeenshire.py Add parser for Kirklees. Get rid of some unnecessary imports. pirms 16 gadiem
AcolnetParser.py Fix Lewisham comments email address. pirms 16 gadiem
AcolnetParser_HTMLParser.py Add New version of the Acolnet scraper using BeautifulSoup instead of HTMLParser. pirms 17 gadiem
AmberValley.py Add scraper for Amber Valley. This was a really irritating one. There is no way to link to individual info pages, so pirms 16 gadiem
ApplicationSearchServletParser.py Daventry have replaced their nice url with an IP address... pirms 16 gadiem
AtriumePlanning.py Fix Dorset County Council scraper (the council seem to be using an IP address now rather than the domain they had pirms 16 gadiem
Barnsley.py Add scraper for Barnsley. pirms 16 gadiem
Berwick.py Add parser for Kirklees. Get rid of some unnecessary imports. pirms 16 gadiem
Birmingham.py Add scraper for Birmingham. pirms 16 gadiem
Brent.py Add scraper for Brent. pirms 16 gadiem
Brentwood.cgi Add Peter's scrapers for Highland, Glasgow, and Brentwood. pirms 16 gadiem
Brentwood.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. pirms 16 gadiem
Broxbourne.cgi Rename Broxbourne scraper so that it matches the database. pirms 17 gadiem
Broxtowe.py Add scraper for Broxtowe. pirms 16 gadiem
CGITemplate Add more debug. pirms 16 gadiem
Cairngorms.py Add scraper for the Cairngorms National Park. pirms 16 gadiem
Calderdale.py Add scraper for Calderdale. pirms 16 gadiem
Carmarthenshire.py Add scraper for Lichfield. Remove another unused import. pirms 16 gadiem
Christchurch.cgi Add Somerset scraper pirms 17 gadiem
Dacorum.cgi Fix the Dacorum perl parser. pirms 16 gadiem
EastHerts.cgi Move some cgi scripts so that I can copy them in with the right file pirms 17 gadiem
Eastbourne.py Add scraper for Eastbourne. The info and comment links won't work since they require you to have a cookie. If you go pirms 16 gadiem
Enfield.cgi Add back in the Enfield scraper which I probably deleted pirms 17 gadiem
Exmoor.py Add scraper for Exmoor. Fix name of Herefordshire. pirms 16 gadiem
FastWeb.py Oops - printing the sutton results twice... pirms 16 gadiem
Fife.py Add Forest of Dean and Fife parsers. pirms 16 gadiem
Flintshire.py Add scraper for Flintshire. I've included the OSGB x,y coordinates in the hope that we can start using these where pirms 16 gadiem
ForestOfDean.py Add Forest of Dean and Fife parsers. pirms 16 gadiem
Glasgow.cgi Add Peter's scrapers for Highland, Glasgow, and Brentwood. pirms 16 gadiem
Glasgow.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. pirms 16 gadiem
Gosport.py Add scraper for Gosport. pirms 16 gadiem
HTTPHandlers.py Add scraper for Gosport. pirms 16 gadiem
Halton.py Adding scraper for Halton. pirms 16 gadiem
Hampshire.py Add Hampshire scraper. pirms 16 gadiem
Harrow.py Adding scraper for Harrow. pirms 16 gadiem
Hastings.py Add scraper for Hastings. Sadly, no decent info urls again. Had to use the search page. The real info url is only pirms 16 gadiem
Herefordshire.py Add scraper for Exmoor. Fix name of Herefordshire. pirms 16 gadiem
Highland.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites pirms 16 gadiem
Highland.pl Add Peter's scrapers for Highland, Glasgow, and Brentwood. pirms 16 gadiem
Hounslow.py Add scraper for Hounslow. pirms 16 gadiem
IsleOfWight.py Add Scraper for the Isle of Wight. pirms 16 gadiem
Kensington.py Add Kensington and Chelsea Scraper. pirms 16 gadiem
KingstonUponThames.py Add scraper for Kingston upon Thames. pirms 16 gadiem
Kirklees.py Add parser for Kirklees. Get rid of some unnecessary imports. pirms 16 gadiem
Leicestershire.py Add scraper for Leicestershire County Council. pirms 16 gadiem
Lichfield.py Add scraper for Lichfield. Remove another unused import. pirms 16 gadiem
Maldon.py Add Pendle scraper (this is basically the same as Maldon). pirms 16 gadiem
Medway.py Add Medway scraper. pirms 16 gadiem
Mendip.py Add scraper for Mendip. pirms 16 gadiem
MultipartPostHandler.py add some Acolnet sites: pirms 17 gadiem
Newport.cgi Add perl scraper from Peter for Newport. pirms 16 gadiem
Newport.pl Add perl scraper from Peter for Newport. pirms 16 gadiem
NorthAyrshire.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites pirms 16 gadiem
NorthAyrshire.pl North Ayrshire, Redbridge: added pirms 16 gadiem
NorthHerts.cgi Move some cgi scripts so that I can copy them in with the right file pirms 17 gadiem
Ocella.py Add scraper for Gosport. pirms 16 gadiem
OtherFilesToCopy.csv Add scraper for Weymouth and Portland. pirms 16 gadiem
Planet.py Set up planet scraper. This get's us 5 new councils (though one, Tewkesbury, has no postcodes) pirms 16 gadiem
PlanningExplorer.py Change some PlanningExplorer scrapers to use date_registered rather than date_received. pirms 16 gadiem
PlanningUtils.py Add scraper for Mendip. pirms 16 gadiem
PublicAccess.py Some debug (mostly for westminster). pirms 16 gadiem
README adding fastweb pirms 17 gadiem
Redbridge.cgi Highland, North Ayrshire, Redbridge: updated based on changes made to planning websites pirms 16 gadiem
Redbridge.pl North Ayrshire, Redbridge: added pirms 16 gadiem
RutlandLike.py Fix Lincoln parser. pirms 16 gadiem
Shetland.py Add scraper for Shetland Islands. pirms 16 gadiem
Shropshire.py Add Scraper for Lewes (which now has PublicAccess). pirms 16 gadiem
SitesToGenerate.csv Add scraper for Weymouth and Portland. pirms 16 gadiem
SouthOxfordshire.cgi expanded the api (still need to test some stuff) pirms 17 gadiem
SouthOxfordshireParser.py add the south oxfordshire scraper, and code to generate pirms 17 gadiem
SouthSomerset.cgi New version of South Somerset from Tom. pirms 17 gadiem
SwiftLG.py Some debug (mostly for westminster). pirms 16 gadiem
WAM.py Add scraper for Eastbourne. The info and comment links won't work since they require you to have a cookie. If you go pirms 16 gadiem
WestDorset.py Add python parser for West Dorset, and remove the non-working perl one. pirms 16 gadiem
Westminster.py Go back to the urllib2 version of Westminster. This works on disruptiveproactivity. pirms 16 gadiem
Westminster_pycurl.py Adding scraper for Halton. pirms 16 gadiem
Weymouth.py Add scraper for Weymouth and Portland. pirms 16 gadiem
createCGI.sh remove offending svn rm line... pirms 17 gadiem
generateCGIScripts.py adding fastweb pirms 17 gadiem

README 

In order to generate the contents of the CGI directory (../cgi-bin/)
run

./createCGI.sh

This script generates new cgi files in the CGI directory,
copies in some other files that are needed,
and commits all these changes to svn.

There is currently something very nasty in the permissions
column in the file OtherFilesToCopy.csv

Until version 2.6 of python, there appears to be no convenient
way to convert a string like 0755 into an octal. I am therefore
using integers to specify permissions of the copied files...

420 is 0644
493 is 0755

This is horrid, and must change in the future...