uk_planning_scraper

Commit graph

Autor	SHA1	Nachricht	Datum
Graeme Porteous	779f0f8d91	Flatten application hash Instead of properties hashes within application hash we now prepends the hash keys so the values are accessible at `property_1_uprn`...	vor 5 Jahren
Graeme Porteous	97f5eb0c7e	Don't scrape properties by default	vor 5 Jahren
Graeme Porteous	9b86779fe7	Extract common Mechanize error handling	vor 5 Jahren
Graeme Porteous	452e39a867	Scape properties details	vor 5 Jahren
Graeme Porteous	1efb7c57d4	Extract summary scraping into separate method	vor 5 Jahren
Graeme Porteous	74e3121f28	Nuke whitespace	vor 5 Jahren
Adrian Short	45427b73ff	Bump version and publish gem	vor 5 Jahren
Adrian Short	dcd996772b	Bump version	vor 5 Jahren
Keith Payne	8b6f13e705	updates Kingston Upon Thames to idox system	vor 5 Jahren
Russell Garner	0908878f56	Fix tags for Leeds/Bradford westmidlands -> westyorkshire	vor 6 Jahren
Adrian Short	b8a303381e	Add lots of councils; bump version	vor 6 Jahren
NotQuiteMinerva	009a4b0932	Update authorities.csv Added: Bedford Blackpool Bracknell Forest Calderdale Cornwall County Durham Darlington Doncaster East Riding of Yorkshire Hull Milton Keynes Newcastle upon Tyne North East Lincolnshire North Lincolnshire North Somerset North Tyneside Northumberland Peterborough Plymouth Poole Rutland Solihull South Gloucestershire South Tyneside Southend-on-Sea Stockton-on-Tees Stoke-on-Trent Sunderland Swindon Thurrock Torbay West Berkshire Windsor and Maidenhead Wolverhampton York	vor 6 Jahren
Adrian Short	ae6c9f6809	Add Portsmouth and Southampton	vor 6 Jahren
Adrian Short	4c23d65bda	FIx CSV file format into 3 cols with header row. Fix #32	vor 6 Jahren
Adrian Short	a24582ee03	Add Northern Ireland multi-authority site #31	vor 6 Jahren
NotQuiteMinerva	74e873331b	Update authorities.csv Added N. Ireland from their centralised site - I haven't added all authorities as it's a long list and I wasn't sure it could cope with it. At the moment only Belfast has applications. The councils are listed below and can't be added separately, they all use this site for planning applications. Causeway Coast and Glens Mid and East Antrim Antrim and Newtownabbey Belfast Lisburn and Castlereagh City Council Ards and North Down Newry, Mourne and Down Armagh, Banbridge and Craigavon Mid Ulster Fermanagh and Omagh Derry City and Strabane District	vor 6 Jahren
Adrian Short	09b289d1fe	Use chained methods for scrape params #15	vor 6 Jahren
NotQuiteMinerva	1e0a321aad	Update authorities.csv Added Nottingham	vor 6 Jahren
Adrian Short	9cfbdd4a18	Remove system tags as they're added automatically	vor 6 Jahren
Adrian Short	dd8e0849e2	Add Aberdeen	vor 6 Jahren
Adrian Short	eb7e63275b	Bump version to 0.3.0	vor 6 Jahren
Adrian Short	96ac906d57	Sort authorities.csv; remove duplicate Wirral	vor 6 Jahren
Adrian McEwen	827eb6c93e	Add Halton (although it's not supported at present)	vor 6 Jahren
Adrian McEwen	1cc6c87ed8	Add St. Helens, Knowsley, Sefton and Wirral and tags for Liverpool	vor 6 Jahren
Ross Jones	22ea0b900d	Adds Wirral	vor 6 Jahren
Ross Jones	9e8fe1245c	Add Cheshire West and Chester	vor 6 Jahren
Adrian Short	8a070e1773	Raise exceptions rather than quit when things go wrong.	vor 6 Jahren
Adrian Short	9b0103030c	Fix URL	vor 6 Jahren
Jez Nicholson	863b7321c8	added a few LAs and use VCR	vor 6 Jahren
Adrian Short	44764bd5a8	Handle tags through methods not directly on the instance variable	vor 6 Jahren
Adrian Short	ca5588706b	Improve regex so it doesn't parse "Not Available" as a date (Lewisham) Should we even bother parsing all these fields from the search results pages given that we'll get them on the deeper scrape anyway?	vor 6 Jahren
Adrian Short	9f3af5e4a9	Tidy date format	vor 6 Jahren
Adrian Short	1aa0aac9f9	Create Application class; scrape into Application objects. #8	vor 6 Jahren
richard	d4ec031ddc	add south london	vor 6 Jahren
Adrian Short	a342ff53bd	Idox: searchCriteria.caseType and searchCriteria.developmentType are two separate things We need to test a group of Idox sites to see how these parameters are used in practice.	vor 6 Jahren
Adrian Short	fa2a705069	Tidy up	vor 6 Jahren
Adrian Short	2778252c22	Raise exception if search gives "too many results" error. Fix #14	vor 6 Jahren
Adrian Short	9229f1ff55	Add expected fields to search form if they're missing. Fix #6 This structure allows us to add other missing fields easily if necessary.	vor 6 Jahren
Adrian Short	7349fd6470	Make exception name nicer	vor 6 Jahren
Adrian Short	580b887708	Automatically add tags for software systems	vor 6 Jahren
Adrian Short	2e100bc3c8	Make scrapers private instance methods in Authority	vor 6 Jahren
Adrian Short	6d72d25166	Search field name variant for Idox eg Bolton	vor 6 Jahren
Adrian Short	8d15678700	Catch Idox internal error where search page fails to load	vor 6 Jahren
Adrian Short	4f4fc49c38	Add Greater Manchester authorities	vor 6 Jahren
Adrian Short	29f2c2a885	Update tags and README	vor 6 Jahren
pezholio	3a1499524f	Some initial tests	vor 6 Jahren
Adrian Short	7441f13d6b	Sometimes date fields have values that aren't dates Like "Not available" in Greenwich.	vor 6 Jahren
Adrian Short	ab9e76ca92	Bump version number	vor 6 Jahren
Adrian Short	1b0f87a371	Add all the London councils	vor 6 Jahren
Adrian Short	b9adc22816	Bugfix scopes	vor 6 Jahren

1 2

72 Commits (779f0f8d91719f1484e27a24e43652e320b2d486)