Browse Source

Catch Idox internal error where search page fails to load

tags/v0.4.5
Adrian Short 6 years ago
parent
commit
8d15678700
1 changed files with 8 additions and 3 deletions
  1. +8
    -3
      lib/uk_planning_scraper/idox.rb

+ 8
- 3
lib/uk_planning_scraper/idox.rb View File

@@ -12,11 +12,16 @@ module UKPlanningScraper
puts "Getting: #{search_url}"
page = agent.get(search_url) # load the search form page

# Fill out and submit search form
form = page.form('searchCriteriaForm')
# Check that the search form is actually present.
# When Idox has an internal error it returns an error page with HTTP 200.
unless form = page.form('searchCriteriaForm')
puts "Error: Search form page failed to load due to Idox internal error."
return []
end
# form.action = form.action + '&searchCriteria.resultsPerPage=100'

# Fill out and submit search form

# Some councils don't have the received from/to dates on their form, eg Newham
form.send(:"date(applicationReceivedStart)", params[:received_from].strftime("%d/%m/%Y")) if params[:received_from]
form.send(:"date(applicationReceivedEnd)", params[:received_to].strftime("%d/%m/%Y")) if params[:received_to]


Loading…
Cancel
Save