Scrapes planning applications data for InLinkUK from BT kiosks.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

16 lines
512 B

  1. require 'uk_planning_scraper'
  2. require 'scraperwiki'
  3. auths = UKPlanningScraper::Authority.all
  4. auths.each_with_index do |auth, i|
  5. puts "#{i + 1} of #{auths.size}: #{auth.name}"
  6. begin
  7. apps = auth.scrape({ validated_days: ENV['MORPH_DAYS'].to_i, keywords: 'inlink'})
  8. ScraperWiki.save_sqlite([:authority_name, :council_reference], apps, 'applications')
  9. puts "#{auth.name}: #{apps.size} application(s) saved."
  10. rescue UKPlanningScraper::SystemNotSupportedError => e
  11. puts "Error: #{e}"
  12. end
  13. end