From 496fcaf391b3413b8c5966fd749d4e840c2e1646 Mon Sep 17 00:00:00 2001 From: Adrian Short Date: Thu, 6 Sep 2018 15:59:46 +0100 Subject: [PATCH] Rename env vars per Morph spec Must start with MORPH_ --- README.md | 6 +++--- scraper.rb | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 5d81107..df70877 100644 --- a/README.md +++ b/README.md @@ -25,9 +25,9 @@ According to the principle of _one codebase, many deploys_, this scraper is [con |Name|Purpose|Default|Required?| |------------------|-----------------------------------------|----------| -|SCRAPER_DELAY |Minimum delay in seconds between HTTP requests to the server.|10|No| -|SCRAPER_USER_AGENT|User agent string sent as an HTTP request header.|_None_|Yes| -|SCRAPER_LOG_LEVEL |Controls the level of detail in the output logs according to [Ruby's `Logger` class](https://ruby-doc.org/stdlib-2.1.0/libdoc/logger/rdoc/Logger.html) constants.|1 _(Logger::INFO)_|No| +|MORPH_DELAY |Minimum delay in seconds between HTTP requests to the server.|10|No| +|MORPH_USER_AGENT|User agent string sent as an HTTP request header.|_None_|Yes| +|MORPH_LOG_LEVEL |Controls the level of detail in the output logs according to [Ruby's `Logger` class](https://ruby-doc.org/stdlib-2.1.0/libdoc/logger/rdoc/Logger.html) constants.|1 _(Logger::INFO)_|No| ## Running diff --git a/scraper.rb b/scraper.rb index f066649..2c24d29 100644 --- a/scraper.rb +++ b/scraper.rb @@ -17,13 +17,13 @@ def crawl_delay sleep DELAY_S end -DELAY_S = ENV['SCRAPER_DELAY'].to_f || 10 # seconds. Conservatively slow by default. Scrapes approx 360 pages per hour. -USER_AGENT = ENV['SCRAPER_USER_AGENT'] +DELAY_S = ENV['MORPH_DELAY'].to_f || 10 # seconds. Conservatively slow by default. Scrapes approx 360 pages per hour. +USER_AGENT = ENV['MORPH_USER_AGENT'] DATE_REGEX = /\d{2}-\d{2}-\d{4}/ $stdout.sync = true # Flush output buffer after every write so log messages appear immediately. logger = Logger.new($stdout) -logger.level = ENV['SCRAPER_LOG_LEVEL'].to_i || Logger::INFO +logger.level = ENV['MORPH_LOG_LEVEL'].to_i || Logger::INFO logger.info "Scraper starts. Let's do this." logger.info "Delay between requests is #{DELAY_S} seconds." logger.info "User agent is: #{USER_AGENT}"